No, you dont need to reduce the RAM. What do you have in the server.bat file? See, it looks like you set a max RAM at 4k or is that from the modpack?
For instance, here is what I put in my server.bat:
java -Xms2G -Xmx1G -XX:permSize=2G -jar minecraft_server.1.7.10.jar nogui
pause
Read here...