I'm running client and server both on 1 GB memory (my computer has a total of 6). I've never experienced an out of memory error with FTB.
When I was still running an older version of Minecraft, though, I used to have a 64x texture pack installed, and boy that killed the RAM in a hurry. Even 2 GB wasn't enough to stop the errors, and that was with less than half as many mods as FTB has. So, for users who are running texture packs - a huge performance boost can be gained simply by turning that off.
Also, one thing I learned while running server and client together: installing Optifine allowed me to limit my framerate to 30 FPS instead of vanilla Minecraft's minimum of 40. This freed up enough CPU cycles for the server to stop stuttering, while it had only a very minor effect on the look and feel of the client (since I was getting slowdowns to begin with anyway).
Finally, when you are playing on a server, ask the admin what kind of view distance he specified in his server.properties file. Setting your own client's view distance to higher than that number will not show you additional landscape, but it will still cost a little bit additional performance and memory because it still increases the skybox; it's just filled with air. The default value for server is 9, which corresponds to a client view distance of "Normal +16", one tick more than Normal; the maximum possible for servers is 15, which corresponds to "Normal +112" in the client (in other words, one tick less than Far). Obviously, without Optifine you don't have that kind of fine control and should probably stay on "Normal" if you have performance issues. Chances are the server is on default view distance anyway.