the unloader and loader need a boarding track on them to hold the cart in place while it transfers liquids. The nether link book you have shouldn't be set to be in front of the portal like that. it should be set to the booster by the cobblestone, what is happening is it is going through the portal to the booster and then straight through the portal again. Linking books do not link to other portals like the nether, but to exact coordinates and facing direction.
Your setup looks fine really. You just need to set the linking books straight. Stand on the booster by the cobble so that the unloader/loader is between you and the portal, create a linking book while you face the portal, use that link book to setup the portal for the other realm (not the one you created it in). After doing it for both books, and getting the boarding track down you should be fine. The default setting on the boarding track and un/loaders should be fine, also make sure they are liquid un/loaders. MystCraft portals can be really confusing when you first start to use them, but it gets easier I think.
Also the boarding track has an arrow on it. You want it to face towards the portal so it sends the cart the right way![]()
What I did to solve the problem was to set the linking book on a track away from the entrance. There you either use an elavated track (raise the track on a block, then replace the track on top of the block with another to act as a stopper)or control tracks to give the cart momentum. That track goes to the loader, then returns directly to the gate. Same on the other side. As the link is set up away from the actual portal it stops any 'bounce' back through.