You were supposed to download the zip file and place the zip file inside the mods folder. Do not touch the FTBServer-1.6.4-965.jar file.
So, restore the FTBServer-1.6.4-965.jar to the original one you downloaded and download the LegacyJavaFixer and place it inside the mods folder.
Other...