Figured I'd pipe in here since this is what I've done for a living for the past 20 years roughly (since 1994) (Unix/Linux Systems and Network Administration), everything from webhosting, to isps, to game servers, to corporate private networks, wans, etc. I really don't recommend running a server yourself unless you have some idea of how to function at the commandline in linux and actual server management in windows and at least some background in security.
First lets establish some correct definitions:
VPS/VDS Virtual Private Server/Virtual Dedicated Server (Hardware: Vendor) - Used fairly interchangeably now. A virtual server dedicated to you that has an allocated set of resources reserved for your machines operation. These typically run in a cluster of vmware/xen/hyperv machines that have hundreds of gigs of ram each, store the virutal machine files on a SAN, and have upwards of 8+ cores per processor and usually between 2-8 processors per server. Each may run just a few VDS/VPS or they may run hundreds. Unless overloaded fairly massively, this doesn't result in a performance decreace for the various hosts (often they still perform better than a dedicated server due to underlying higher end storage systems and such... but if overloaded they're abysmal performance wise.) They often can do automatic failover, snapshots of running systems for backups, and live migrations between various hosts in the cluster to help maintain nearly 100% uptime. (We usually aim for between 4 and 5 9s availability where i've worked ) They carve out the specifics of your system from the resources on a host and if need be it should be able to be freely moved to another host if something is amiss on the host it started on and often times will be automatically migrated to other hosts for performance reasons. You should remain blissfully unaware of this.
Dedicated Server - (Hardware: Vendor) The vendor owns the hardware you're using. You can typically install whatever you need on the system to accomplish your goals. You want a RAC.
Co-located Server - (Hardware: You) You provide a given piece of hardware with a specific specified foot print and they provide you rack space and bandwidth. Typically 1-4 RU is what they provide with some quantity of bandwidth. Make sure you have a RAC card of some type in the system and that it's accessible so you can get a remote console if the event of a networking failure. Whatever fails, you will be responsible for replacing. You will likely also be entirely responsible for backups and operating system updates. Some facilities have non-allowed content and services (many exclude for instance irc daemons since irc has a tendency to get denial of service, almost all exclude anything illegal, etc.)
Managed Server - (Software: Vendor) A server managed by vendor, can be any of the 3 types above, typically this means they will provide at least some level of software/operating system installation and on-going maintenance. Usually they break managed services into various levels of increasing price for increasing levels of management. At the high end they might help you optimize your database and webservers, at the low end they might install security updates once a month. Often times a managed server will come with a control panel as well if you're not familiar with server management a managed server with a control panel is a good way to go.
Vendor/Host - The people providing you the services.
I'm primarily going to talk about hosting on linux, because after 20 years of doing both I wouldn't consider hosting an application on windows that absolutely didn't require it. Minecraft doesn't require it.
Processor - 2 cores/processors tends to be a bit light, 4-8 seems to land pretty squarely in the minecraft realm. This lets you dedicate several cores to minecraft (and mods) and still have a core available for additional processes like databases or webservices you're providing (forums, teamspeak, etc.) Hyperthreaded cores are ok for reducing cpu time contention, but don't count them as part of the available cores. If i felt like counting them at all, i'd count them as .25 cores each. Cache matters, 4096kb+... 8192kb is better. Raw Ghz matters. Higher is better... but it's a secondary consideration until you max out ram.
Memory - RAM, I figure 2g + 128mb/concurrent person roughly until I hit 8g for "heavily modded" servers (my own pack has ~110 mods and runs well on 8g with upwards of 20-30 people concurrently). After that you gain more performance by dedicating ram to mysql for block logging, ramdisks, increasing cpu, or spending time optimizing your java performance. If I had a server with say 128g of ram, I'd likely dedicate 16g to java (minecraft), 8g to the database (for most minecraft purposes this is usually more than adaquete), and then allocate the rest to a ram disk.
Network - Very important. Verify you can get enough bandwidth from a host before you sign up with them. I figure .5Mb/s *BOTH DIRECTIONS* per concurrent user... I've seen it spike beyond that though with certain mods, but never on vanilla and even with mods ive never seen it sustain at >.5Mb/s per user. Best case is test first as always. Also be aware of bandwidth charges, you will get a certain amount as part of your package, but if you exceed that you will be charged an absurd amount of money per G you exceed it. If you THINK you might exceed it, call the host and ask for your package to be upgraded to a higher bandwidth quantity. It will save you money. You want at least 100mbit, don't take 10mbit these days (it indicates a poorly designed network and misuse of equipment/reuse of old equipment.) Gigabit is better (and pretty frequently offered.) Keep in mind these are local lan speeds, and what matters just as much is the hosts connectivity. Watch your hosts ping times for a week and see how much packet loss you're seeing, see what the average ping times are, see how many hops they are from you and from the backbone via traceroute. Lower on ping is better. Over 100 is unacceptable.
Storage - First, understand this: Comparing any solution except a ram disk as a single drive option is idiotic. You're virtually begging for data loss. On a vps/vds you absolutely should be on a raid no matter what since it should be stored on a SAN (even if you're unaware or can't see it due to the virtualization.) If not-- find a new vds/vps host. On a dedicated or co-located machine you should have a minimum of 2 drives in a mirrored configuration (raid 1)or 3-4 drives in raid 5. Increasing the number of drives increases the spindles (for non-ssd and stripes for ssd [with caveat]) which will improve performance usually. At some point you're better changing to a different type of disk/interface. Interface wise you generally follow this route: SATA II->SATA III->SAS/SCSI->PCIe->Direct (ram). Drive types you generally go: 5400->7200->10000-->SAS/SCSI(10000)->SAS/SCSI(15000)->SSD SATA->SSD SAS->SSD PCIe->Ramdisk. Now that all said, lets discuss raid and ssd drives. If you're using raid on a ssd you lose (or last i researched it this was true, ~8mo ago) the ability for the ssd's to use the TRIM command which is the primary mechanism ssd's prolong their life with. I've heard claims that the newer garbage collection makes up for some of the loss but I've seen no evidence of such yet and after this long and seeing the number of ssd failures in the field that I have... I'm a real sceptic. Meaning the drive will degrade what most people would consider "massively fast" (couple years) vs normal ssd life of 5+ years... that's fine as long as you plan for it though, planned failure is something you do with ram disks as well... If you're planning on using a ram disk you need to make some additional considerations, primarily what I refer to as "Planned Failure", without a doubt at some point... you will have something happen that will result in the server reseting unexpectedly. If you're using a ram disk and aren't backing up very frequently (5m-15m) you are going to lose all the data since your last backup. As long as you're doing frequent backups and have planned well for the eventual, you really can't find better performance than a ram drive. Putting all the world directory on the ram disk will only require a small ram disk and offers a fairly significant performance boost. If you're planning on including the entire minecraft directory and have dynmap, I hope you allocated plenty of space. Our world directory is about 1g, our minecraft (mostly due to dynmap) is 7.2g and it's not setup to be massively HD and most of the mystcraft worlds don't have significant renders. So long as you plan for and properly implement these things you will have no problems.