When I change a config, add a mod or anything even if it doesn't change world gen I make a new world; So no I don't think it's that. I never make a single test world, every time I launch MC I usually make a new world (I delete the content of my saves folder often).
To give a little more...