Do they even work outside of Peaceful? No idea if anything changed has changed recently, but last time I tested them out they didn't. Diamond sword in the chest and all. Then, as soon as I opened the options menu and changed the difficulty, mobs started spawning for a split second and the chest started getting items.
Right, they don't work outside of peaceful, but that's the point I was making. You can get ghast tears and blaze rods and ender pearls without ever seeing a mob, but no one complains about it. This doesn't affect me at all, and I personally don't care, but I can't fathom how none of these "balance police" ever say a thing about it.