Disabling mods that you don't find that you use much off is definitely not craze.
I got tired of the mods that were there for me, and didn't really feel that any of the packs fitted my needs. Took the Dw20 pack as a base, removed all the mods that I don't use, and added some that I wanted to play with (Big Reactors, Witchery, Rotarycraft etc)
All of a sudden I am playing with all the mods i wish to, but couldn't before due to my PC. In total, i have just about the same amount of mods, but removing some of the bigger ones, really did my system a favour.
(Also using the pack via MMC, seems to have boosted my FPS by a decent amount, but that might be pure coincidence.)