Little update :
I removed the chunk loader from the Nether, but that didn't work either.
I also noticed that my FPS in the Nether is normal, so something in the overworld is causing alot of lag.
Anybody know why?
EDIT : Sometimes when I launch the d20 pack and log into my world, it does...