There are a few factors with the ender quarry power consumption. First of all, power consumption is calculated per block broken so when you add a speed upgrade, power drain doesn't only increase due to the upgrade's own power drain multiplier but it also increases due to more blocks being broken per tick. This is why it consumes 30x as much power with Speed III despite there only being a 2x power drain multiplier.
Secondly, there's a base term per block broken which is unaffected by upgrades (this is why it only takes slightly more power to add Silk Touch), along with a hardness term which is proportional to the hardness of the block broken. This hardness term is multiplied by all the upgrade multipliers (multiplicatively, so an 80x and a 2x makes 160x).
I don't consider Fortune to be worth the massive extra power drain. Use silk touch and process the ores in your base (break them manually with Fortune or set up some MFR block breakers or other similar blocks). Then add a speed upgrade based on how much power generation you have available. So, for example, if you have less than 1 kRF/t, don't put any speed, and if you're generating at least 25 kRF/t or so, go Speed III (since the quarry becomes slightly less efficient when you upgrade it, if you don't have the power, excessive upgrades will actually make it slower). Note that you'll need quite a few ore processing machines. On my world I have a dozen fully upgraded SAG mills, a dozen fully upgraded redstone furnaces, roughly 15 MFR block breakers (since you can't upgrade them to be really fast), and one pulverizer and induction smelter for redstone ore --> cinnabar and cinnabar + ferrous ore --> shiny ingots (ferrous ore is rare so just one can keep up with the quarry). Note that the number of machines I have is a bit more than what is needed for a Speed III quarry's ores unless you put the quarry in a dense ore world or something, but the excess is nice to have when I just want to grind/smelt a ton of stuff (e.g. cobble to sand, or sand to glass).
Note that the machines you'll need for your processing also depends on your modpack. If your modpack adds a whole bunch of different ores, you will need more machines.
