If you're using barrels, you don't need colors at all- barrels will only accept one type of item, and tubes are smart enough to never put items into the wrong barrels. Just re-routing the tube from the first sorter into the right end of the barrels, placing a tube on top of each barrel, and leaving the second sorter at the right should solve all your problems. Or, as mentioned before, restriction tubes count as a tube that's about 5000 blocks long (I think), so you could move the second sorter one block back and place a restriction tube in front of it.