Yes, you'll need to fill the router by some other means. If it is inserting into an inventory it will not extract on its own. You could accomplish that with two routers, the first with an ejector upgrade... but that's kinda silly quite frankly.
Be careful using pipes to feed frames into the routers, have some kind of spill over protection in place because once all of the Frame Housings are full the router will stop distributing them and the pipe will start spitting them out onto the ground -- you could limit this with gates but you'd still get spill over.
You should be able to interface with any of the faces below the Alveary to get the products (combs/bees/etc) out and if you pump bees into those 9 faces they should go in their appropriate slots. They're very smart about it. Even then, you might need to be a little more specific about which sides you interface with but I'm pretty sure that's only for Pneumatic Tubes which don't really apply anymore. Personally, I like the AE approach a little better, just feels smoother having a Fuzzy Export Bus, a Basic Import Bus, and a Barrel directly below the Alveary totally out of sight.