I think it would make sense to replace with a new block that has no use. Call it End Dirt, Filler, Residue, something like that. Although I suppose the advantage of dirt is that if you remove ExU from your world, you don't get an instant huge hole in the landscape.
I'd guess that it just needs the entire fence to be chunk-loaded in order for the boundary scan to work, so one-quarter of the player loading area, or more if extended by chunkloaders.
I set an Ender Quarry running in the Nether last night. To my surprise it dug up a small amount of cobblestone, about one and a half stacks compared to 15 stacks of netherrack in the short time that I watched it. Any ideas where the cobble is coming from? I've not seen any stone in the nether, and I'm pretty sure it hadn't chewed through anything that I had built.