I tried doing that as well, I waited about four minutes before deciding it wasn't going to pick them up. The strange thing is that every once and a while, I'm talking like thirty to forty minute intervals it would pick up an apple or some sand. It's very strange, though I've never had the best of luck with forestry farms, this is even stranger than when I made a tree-farm using the arboretum and logger, and it would cut them down but would produce nothing at all. Haha.
EDIT: Seems I've fixed it, apparently my output wasn't valid so it wasn't collecting it or something. Not quite sure.