Never had a problem with whacking someone with a vajra. Though you pretty much are in creative mode ones you get a UU factory set up. I think I got about 300 Ultimate solars and 20 rock breakers fueling 3 recyclers. With that going on, I can do as I please without worrying about resources. Within the ~week it's been running I already have half an upgraded barrel full.
Though in terms of vajra accidents, don't use it when a server's lagging. Used a rock cutter to mine some stuff (diamonds, redstone, nikolite, ect.) if you switch to the vajra, it suddenly thinks you mined it with that instead and just drops the pieces instead of the blocks. Though while I love it as a weapon, it can be a real pain to use it like one sometimes if you don't want to break your surroundings. But that's what warded blocks are for!
Are you not playing with GT installed or did you stock up on iridium before the nerf?