Try starting the server with just these arguments and no others-
-Xms10240M -Xmx10240M -XX:MaxPermSize=256M
As @HaoSs stated this is most likely a issue with a item, setup, or system somebody made with thermal expansion.
I'm just assuming that you are using a 1.6 modpack you never said what...