Mod Concept: Lagfree Automation

[Azgult]

So I'd like to guage interest on a mod idea I've come up with and may eventually end up implementing (not right now, as I'm busy with finals and such).

When I build thinks, I like to build them big. It therefore annoys me that the size of factories I make is essentially limited by my computing power. My factories have not yet reached the size that the lag is noticeable, however I am playing SP. I suspect that this is a serious problem for servers and/or for SP players with massive production facilities.

My concept to solve this is simple: Instead of hundreds of separate TileEntities connected by pipes/tubes and wires of various kinds, everything would get replaced with a single master TileEntity, which pre-calculates resource (incl. EU/MJ/whatever else) consumption and final items produced. This would save all computing power for handling progress in each machine separately, as well as that used in item transport. This input/output would happen at a user-set rate; by setting this rate lower even less CPU power would be used (I might also apply this to quarries, so that it mines sets of blocks at once, drastically reducing lag from them). Of course machines in such a factory would be blocked from being used otherwise.

Theoretically I may eventually take the concept further, so that factories can be made without a pre-set routine for making something, which then could serve as an anything-factory - that is to say you could request an item as the output and the factory would try to make it using the machines at its disposal in the best way.

Anyway, I'd like to hear from you whether you have had problems with automation-based lag before, how interested you would be in this concept, and of course whether you have any ideas on how to improve the idea.

I'd like to point out again that this is in the very early concept stages. I am not yet sure if I will implement it (although if there is a lot of demand for it, I probably will), and certainly won't start working on it for a few months due to school. Furthermore I have only had limited experience with minecraft mods in the past, and would need a fair bit of time to work myself into the way automation is handled in other mods - and that before I can write a single line of code.

EDIT: Forgot to ask, has anyone heard of this done before? I'd be kinda wasting my time if so

 

[Antice]

Will start with the last item on your post firts, No it has not been done before afaik. and I've been around for a while too.
As for interest. sure. I am certain that any item transport system you come up with that removes lag would be most interesting to the guys over at buildcraft.

However. one caveat. I don't think that things like quarries should be changed too much. they might break blocks one at a time, but that is actually the least lag inducing way to do it as is.
What could be changed however is the way items are taken out of the quarry. if it had it's own internal buffer system, that only outputted full stacks at a time for instance. would help reduce pipe lag by a de facto reduction in the number of item entities that goes trough them while still maintaining the troughput.
Redpower does something like this when using a filter. When it pulls items out of inventories you end up pulling an entire stack at a time instead of single items. I guess you could do something similar, except your transport system could maybe package multiple entities and entity types into a single container that is then shipped trough the system as a single entity. I've noticed that for long transport legs it's less laggy to use a storage cart on a rail rather than a pipe due to the lower amount of entities involved in moving a chest full of stuff at once rather than as separate entities. it all depend on wither you have to move item data from one array to another in the program to achieve the desired goal.

honestly tho. the entities are just moved the way they are moved due to looks. the optimal item transport system would be a multiblock structure that put all the items into one massive array that machines could then collectively share. (ender chests are pretty good at reducing lag too)
I dunno how interesting such a beast would be to actually play with to be honest. some of the charm of automation mods in general is to watch items zipping back and forth between the various machines while some industrial sounding music plays in the background.

This is just my 2øre worth. I'm not a programmer, and have only a basic grasp of the concepts involved.

 

[Zjarek_S]

honestly tho. the entities are just moved the way they are moved due to looks. the optimal item transport system would be a multiblock structure that put all the items into one massive array that machines could then collectively share..

I recommend checking Applied Energistics out: http://ae-mod.info/ . It is quite new mod, but very promising.

 

[dc0110]

What could be changed however is the way items are taken out of the quarry. if it had it's own internal buffer system, that only outputted full stacks at a time for instance. would help reduce pipe lag by a de facto reduction in the number of item entities that goes trough them while still maintaining the troughput.
Redpower does something like this when using a filter. When it pulls items out of inventories you end up pulling an entire stack at a time instead of single items.

What happens when you don't have a full stack of something? no matter how big your quarry is, i think there's slim chance of getting exact full stacks of everything.
Good to work these kind of things out though

 

[Antice]

What happens when you don't have a full stack of something? no matter how big your quarry is, i think there's slim chance of getting exact full stacks of everything.
Good to work these kind of things out though

it could push the items out when it's done digging maybe? or it could collect for a set amount of time and then pulse the stacks out regardless of wither they are full or not at that point in time. the basic idea is to just avoid sending single items of everything all the time. especially things like cobble is pretty lag inducing since there is going to be a lot of it in the pipe at once.

 

[dc0110]

it could push the items out when it's done digging maybe? or it could collect for a set amount of time and then pulse the stacks out regardless of wither they are full or not at that point in time. the basic idea is to just avoid sending single items of everything all the time. especially things like cobble is pretty lag inducing since there is going to be a lot of it in the pipe at once.

Maybe an accessible GUI/Inventory, so you can manually collect the remaining items, there are maybe to many variables to factor in for the "auto eject when done" scenario

 

[King Lemming]

EDIT: Forgot to ask, has anyone heard of this done before? I'd be kinda wasting my time if so

It can't work as a purely singular tile entity, you need to assemble many small ones in some way. This is exactly how Thermal Expansion conduits and liquiducts work and why they are efficient. However, the interface constraints on a singular tile entity that would handle everything and the genericized coding requirements would actually make it less efficient than multiple specialized systems.

One of the major goals of TE is to show that a tech mod can be extremely cpu-friendly, and we're doing that. So while I wouldn't call it a waste of time on your part, it is not an entirely new concept.

 

[MilConDoin]

I think the item tesseracts are among the least lag inducing possible ways to move items around.
And for the quarry: Let is use an internal storage (maybe 9 slots), iterating over these slots once per second, sending the content of the current slot out into the network/tesseract/... Problem here would be, if you have a big variation in the stuff you cave out together with quarrying faster than once per second. Since this should be rare, just halt the quarry momentarily until the next second, when a new slot will become free.

Why did I know beforehand, that the King would show up in this thread? Your crusade against lag is too deep ingrained, it seems. Hail to the King, Baby.

 

[Azgult]

It can't work as a purely singular tile entity, you need to assemble many small ones in some way. This is exactly how Thermal Expansion conduits and liquiducts work and why they are efficient. However, the interface constraints on a singular tile entity that would handle everything and the genericized coding requirements would actually make it less efficient than multiple specialized systems.

One of the major goals of TE is to show that a tech mod can be extremely cpu-friendly, and we're doing that. So while I wouldn't call it a waste of time on your part, it is not an entirely new concept.

As to implementation specifics, it has been quite a while since I even looked at the decompiled minecraft code and I've never even worked with forge, so I'm basically just making educated guesses about the internal workings right now. I am fairly certain though that the main implementation problem would be extracting all recipies for a certain machine in a reliable (and if possible mod-independant) way. Regardless, implementation (including how precisely quarrys could buffer items) is not really what I wanted to discuss here (I have several good ideas floating around, otherwise I wouldn't even consider asking if people would be interested).

As for thermal expansion, I must say that it is one of the main inspirations for the idea; liquiducts/conduits make BC waterproof/conductive pipes look like they were designed by toddlers. I must confess that I haven't tried using tesseracts for item distribution yet, only for long-distance transport.

Nontheless, from a purely technical point of view, all systems (that I know of) have a time complexity of O(n), which could theoretically be reduced to O(1) with precalculation.

What I am still wondering is exactly how common automation-based lag is; I thought I had a lot of it earlier, but it turned out I derped and managed to make the RP equivalent of a storage loop (I had items oscillating between a target and backflowing every tick). Right now my tritium+deuterium factory for 4 fusion reactors is running fine, but is also munching a fair amount of my CPU.

Lastly, as to the aesthetic value of pipes/tubes, I have to agree. Kinda. Yes, it is very nice to see the items flowing through, BUT in the end most people end up building a factory and forgetting about it, the only item flow they may still see is the input/output flows.

Bleh, I've talked too much >.>

 

[Golrith]

It can't work as a purely singular tile entity, you need to assemble many small ones in some way. This is exactly how Thermal Expansion conduits and liquiducts work and why they are efficient. However, the interface constraints on a singular tile entity that would handle everything and the genericized coding requirements would actually make it less efficient than multiple specialized systems.

One of the major goals of TE is to show that a tech mod can be extremely cpu-friendly, and we're doing that. So while I wouldn't call it a waste of time on your part, it is not an entirely new concept.

Hmm, that sounds like you are looking into item transport via TE version of pipes. After all, you've given us liquid and power, that just leaves items.
Especially with (so it seems) RedPower causing lag, we need a CPU friendly item transport system. Tesses are great for getting items from A to B over distances, but pipes are still needed (unless you are rich and can afford a ton of Tesses).

Hopefully your coding methods will filter into other mods, so they are more CPU friendly too. The more we can help the CPU, the more mods we can kill it with

 

[King Lemming]

Nontheless, from a purely technical point of view, all systems (that I know of) have a time complexity of O(n), which could theoretically be reduced to O(1) with precalculation.

I'm afraid that there are a couple of incorrect assumptions in here - all systems are not O(n) - there are quite a few already in MC which are O(N^2), and while you can precalculate any given system state to create an O(1) algorithm on paper, that only works for static systems. When your system changes, you get to recalculate your state, which can be far worse than that O(N) ever could be. Even amortized over time, it may easily surpass the pre-optimized state. This is especially likely given the nature of Minecraft, when chunks load and unload constantly depending on tons of different factors - spawn chunks in particular would be a huge monkey wrench in this one.

Even conduits, which actually do pre-calculation to reduce the steady state computations as much as possible are still a variant of O(N). And this makes sense - Minecraft is single threaded and you have to iterate through each critical point and interface with the tiles directly.

Again, not a bad idea, but if you really want to help with CPU overhead, pick a system and dive in, then optimize that particular game mechanic as much as possible. Maybe even throw in a commit to Forge.