This is what I had to do to fix it. I Removed the chest, placed an ME Interface, Inside the ME Interface I configured it to store 1 ore of each type. I placed an port Bus on the ME Interface and whitelisted the type of ore that I want to process. The Resonant Servo is set to whitelist the ore.
So the ore is stored inside the interface, which is then removed with a servo.
Dare anyone to create a more elegant idea.
[EDIT]
This won't work.
FIX:
Remove the Export Bus and place 2 of the ore inside and it will keep ore inside the interface long enough for the Servo to pull them out and place them into a mill and not go into the ME system.