There are a few factors to look at:
1). CPU speed. Faster is better. Multi-core not so much, since MC just started using multiple threads for dimensions but still runs on a single core. If you have a good OS and the configuration knowledge you might be able to dedicate a core to MC, leaving the rest to run the OS.
2). Ram. Remember, when allocating ram that you have to leave enough for the OS to function... Without the OS, nothing runs. Also, if you have "integrated graphics" you have to leave a sizable chunk of ram for that because it takes what it needs from system ram. If you have a separate GPU card it probably has it's own ram but if the mainboard has the Integrated GPU you'll probably need to disable it in your bios or the system will still reserve the ram.
3). Make sure that both your OS and Java versions are 64-bit! Having a 32-bit version of either (While 64-bit Java won't install on a 32-bit OS, a 32-bit Java installs just fine on a 64-bit OS) will limit you to running Vanilla and only the smallest of packs... This is because of 32-bit software has to run on 32-bit CPU hardware, which has ram limitations.
This being said, you're update path should be, in order of effectiveness:
1). 64-bit OS. This is the most important thing you can do, assuming you have a 64-bit CPU, of course. A 32-bit CPU cannot run any 64-bit software. If you use Mojang's launcher (Vanilla) or Curse, which links to Mojang's launcher, you don't need to worry about Java, since Mojang loads a runtime of the currently supported version (1.8u25). If you need Java get at least that version.
2). Ram. Depending on your system you might have 1.5 - 2GB that must be reserved for the OS and possible integrated graphics. I've seen people claim that they run fairly big packs on 2.5GB for Java, but it's usually at reduced performance settings like render distance, etc. Since 4GB seems to be about the most ram Java seems to be able to manage while running Minecraft, that's the current default setting Mojang gives their runtime Java config. I'd recommend upgrading to a minimum of 6GB ram. 8GB ram might actually be cheaper and more efficient depending on your hardware.
3). CPU. If you have an older CPU you should be able to get a faster one fairly cheaply. Disregard this step if you have a laptop. Most are designed to only allow everyday users to upgrade ram and maybe the hard disk. Even if your laptop CPU is socketed (mine are) you need a few years experience working on tiny electronic devices and mobile CPUs are not usually available at Fry's or Newegg. They're also expensive.