It depends on if your running the server on the same machine as your client. If so, it would be better to allocate 4gb to single player client. The reason a server helps with client lag is because it handles the chunkloading and world gen stuff, so if on the same machine it would load you down.