Try using the "upper left", "upper right" and "to cart", "from cart" selections, rather than color. I got that working well with AE at least.
I use an export bus inputting to a regulator on one side and a import bus directly attached to the cargo distributor on another side.
Edit: The regulator + export bus is for adding char coal that powers the cart.