Best lagfree sorting system? Pipes vs Tubes

[PhilHibbs]

Yes, I said that. I could do that for Cobblestone. And Marble. And Dirt. And Gravel. And Coal. And Apatite. And Redstone. And Nikolite. That covers the major items that you get significant streams of. I guess I could live with 8 slots in my input chest being permanently occupied, and the valuable stuff I could just put 8 in rather than 64, but then there's the macerated coal, every stack of 64 coal will generate 128 dusts. Actually I put a Timer on my Macerator outputs, so that gets bunched up automatically. Maybe I should add Obsidian as well, although mostly I'm quarrying Mystcraft ages that have no lava.

 

[AlanEsh]

I don't see why you'd be reluctant to just use the stacking feature in the Sorting Machine -- I mean I guess you could put a pre-Filter or several prior to the Sorter to pull out certain items before they get to your Sorter, but I don't know how that's much different from just doing it in the Sorting Machine. Diamond pipes are always a fun way to divert stuff prior to the big sorting mechanism too.
Anyway, I apparently am not really understanding the difficulty you are having, so...

 

[DoctorOr]

Right now, I have a hopper and BC pipe setup spewing honeycombs all over the ground, and I am trying to solve it with just buildcraft and I am becoming frustrated with it again.

Loops. Alternatively, iron pipes will cause items to "bounce" back.

One question though, if the sorting system locks up at the source, isn't that less lag producing than piles of entities all over the place?

But that's not even how redpower works. Redpower will back flow _any_ output block on the tube network, and doesn't decide to do so until the item reaches its destination and is determined to be full. This means the backflow fills all the other devices on the tube near the destination before returning to the source.

So, unless you are using strictly point to point tubes, a problem in one section can cascade and block other processes. Due to the nature of backflow, you may not even notice.

 

[Abdiel]

As a programmer I find it funny how there are 4 pages of people arguing that X causes more "lag" than Y, with not a single profiling experiment, not even a test in a controlled environment. All we have is "on my server (which has 3 billion other things running at the same time) I sometimes get "lag". And I use X-pipes. Therefore X-pipes lag."

I've never noticed either system being a significant source of performance problems, client- or server-side, if not horribly abused. This of course means getting rid of all your cobble/dirt/gravel/etc. right at the quarry, and sorting only the useful things. Pretty much any system will easily handle this kind of load even from multiple quarries. Every time I hear someone complain about overflow from quarries, it's because they insist on hoarding bajilions of stacks of cobble they will either never use, or can produce anywhere on the spot using an igneous extruder. (Yes, there are other things which could cause your pipes to overflow. But quarries are not one of them, if you use them properly.)

 

[KirinDave]

I've never noticed either system being a significant source of performance problems, client- or server-side, if not horribly abused.

While the plural of anecdote is not data, I do think a plurality of experience reports represents actionable data. And it's definitely the case that rp2 tubes have ruined our old server for a lot of people. I had to remove them and replace them. You and I don't notice because we have computers at a higher spec. As we're both aware, performance problems in tight loops that fit into arbitrary quanta like framerates often have stair-step like behavior as you pass beyond the performance threshold.

Actually, if you'd like to know what steams me: it's how many game crashes the Sortron causes. It's like no one ever wrote a real fortranforth (edit: derp, brainfart, f languages) program for it before. :\

 

[PhilHibbs]

Actually, if you'd like to know what steams me: it's how many game crashes the Sortron causes. It's like no one ever wrote a real fortran program for it before. :\

Or Forth. I've not had a Sortron crash yet, do you know what triggers it?

 

[KirinDave]

Or Forth. I've not had a Sortron crash yet, do you know what triggers it?

Recursion.

 

[DoctorOr]

As a programmer I find it funny how there are 4 pages of people arguing that X causes more "lag" than Y, with not a single profiling experiment, not even a test in a controlled environment. All we have is "on my server (which has 3 billion other things running at the same time) I sometimes get "lag". And I use X-pipes. Therefore X-pipes lag."

You presume much. These tests have been run, and redpower is a major source of lag because the blocks (filter, transposer, buffer, etc) cause inordinate (insane, really) levels of block updates. Minecraft works by sending the data for individual blocks, until it reaches a set threshold number of changes, then it sends _the_entire_chunk_. The threshold is set low enough that a small number of redpower blocks can trivially reach it. These block updates occur when the block processes an item onto, out of, or through the tube. Furthermore, each block causes _several_ updates every process. So a transposer extracting items every 2 seconds: Multiple block update every 2 seconds. A filter as inline on a tube: Every item that passes is several block updates. A filter receiving items to put in an inventory: Several block updates every item it puts in that inventory.

"Higher tech" blocks actually make less updates, so that a sorter or manager will cause less lag. This requires the availability of power.

 

[KirinDave]

You presume much. These tests have been run, and redpower is a major source of lag because the blocks (filter, transposer, buffer, etc) cause inordinate (insane, really) levels of block updates. Minecraft works by sending the data for individual blocks, until it reaches a set threshold number of changes, then it sends _the_entire_chunk_. The threshold is set low enough that a small number of redpower blocks can trivially reach it. These block updates occur when the block processes an item onto, out of, or through the tube. Furthermore, each block causes _several_ updates every process. So a transposer extracting items every 2 seconds: Multiple block update every 2 seconds. A filter as inline on a tube: Every item that passes is several block updates. A filter receiving items to put in an inventory: Several block updates every item it puts in that inventory.

"Higher tech" blocks actually make less updates, so that a sorter or manager will cause less lag. This requires the availability of power.

More on this subject. Some research suggests that Immibis's tubestuff and immibis infinitubes actually do most of the same things that RP2 tube systems do but with a critical eye towards this chunk update threshold. It's interesting. I'm actually surprised we don't see more people using his tubes and microblocks mods. They seem very well maintained and have a high degree of functionality. They also don't use "perfunctory power" in that once you put ANY power into them they run forever. They actually require a real supply of MJ to run.

 

[Guswut]

Spoiler: Recursion, you say?

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

Recursion.

More on this subject. Some research suggests that Immibis's tubestuff and immibis infinitubes actually do most of the same things that RP2 tube systems do but with a critical eye towards this chunk update threshold. It's interesting. I'm actually surprised we don't see more people using his tubes and microblocks mods. They seem very well maintained and have a high degree of functionality. They also don't use "perfunctory power" in that once you put ANY power into them they run forever. They actually require a real supply of MJ to run.

Although the infinitubes are fairly ugly, which is a fairly decent downside, eh?

 

[KirinDave]

Although the infinitubes are fairly ugly, which is a fairly decent downside, eh?

Microblock them away. But yeah they do sort of look like they're made of Ikea plastic and filled with pearl jam.

 

[Guswut]

Microblock them away. But yeah they do sort of look like they're made of Ikea plastic and filled with pearl jam.

Indeed, which could grow on me, but I hope not as I my medical coverage doesn't cover alien infestations.

And yeah, with the microblocks, everything is better.

 

[Abdiel]

Any idea why is it not in any FTB pack then? Permissions nonsense?

 

[Guswut]

Any idea why is it not in any FTB pack then? Permissions nonsense?

FTB normally tries to limit mods that duplicate each other's usage, which infinitubes would do in regards to RP2's tubes.

 

[KirinDave]

FTB normally tries to limit mods that duplicate each other's usage, which infinitubes would do in regards to RP2's tubes.

FTB has a soft spot for RP2 for historical reasons. Immibis has done a way better job keeping the mod up to date tho. And RP2 has gorgeous texture work, which is a big plus.

 

[DoctorOr]

Any idea why is it not in any FTB pack then? Permissions nonsense?

Politics. Immibis does many of the things redpower does, and does them better even to the point of accomplishing things previously claimed to be impossible.

 

[biomirth]

Any thoughts on how to buffer a sorting system so that it doesn't try to sort each individual item as soon as the quarry puts it in? The main problem is that everything pulls from the top left of the chest and also inserts into the top left. So as soon as a stack of cobble is pulled out and the next half-stack of cobble fulls up, a single cobble appears in the top left and gets sucked out and replaced by a single cobble etcetera. I don't really want to set Sorting Machine that pulls Cobblestone to only pull full stacks, but that would help somewhat. If I followed that path to its logical conclusion, I'd have little part-stacks of dozens of item types cluttering up my input enderchest.

I'm tempted to switch over to using turtles, as they buffer the input naturally. That would still leave my macerator and furanace output as individual items though, I'd like some smart stacking applied to that as well.

The way I handle this is to void cobble/dirt/gravel at the source. This means there are periods where the filter can work down any buildup in the input chest easily. If you are sucking cobble or any other large-scale source then you can add a second filter set to only pull that item, or pre-filter it at source. You can redstone-signal the first filter to "pull all but X" and that should keep them sorted.

 

[PhilHibbs]

I could do that. That makes the quarry set up a little more complex. I might, however, switch from using an Enderchest as a buffer to using a Turtle instead. I can easily program the turtle to throw full stacks into the chest immediately, and to also throw some of the fullest stacks if its own inventory is in danger of filling up e.g. when Slot 13 gets an item in it.

 

[PhilHibbs]

In case anyone is interested in my Sortron buffer, it's the signalBuffer program at the end of my code at the start of this thread. It pulls one slot every second from the attached chest. I have a black enderchest attached to my quarry, a sortron in between a black and a white enderchest in my base, and a white enderchest as the input to my sorting system, so my sorting system never sees the single stacks of cobble coming from the quarry, it just sees however much there is of whatever item was quarried 27 seconds ago.

 

[AlbertLooper]

Only thing really to avoid is a lot of machines in one chunk.

I wonder, does it matter if you put the same amount of machines in a single chunk or 2 separate ones next to eachother if you load them both anyway?

A player loads 10 chunks each direction around him, imagine they're all 'empty' and you have two scenario's. 1 chunk in front of you with '100 lag' or two chunks with '50 lag' each. Is there a difference in performance?