[Help] Most Efficient AE Storage System?

[Jack0928PC]

First let me start this off by saying I am a hoarder, my chests practically define the term. I know that AE is the system for me for the simple fact that it takes me 10 minutes sometimes to even find an item in one of my hundred chests sitting around my base. It will be excellent to have everything in one place for me to find. Now with that said I am not coming into this blind. I do have background knowledge on Applied Energistics, nothing special, just the gist of things. I always used to think that the best way to set this up was to just have multiple chests with 64k storage inside of them and that was it, and that was how I used AE (Obviously not the smartest or most organized way of doing things). But I have been reading into it a bit and it seems that many people seem to just recommend using 1k or 4k storage units. I took this into consideration and gave it a shot. I made myself a 4k storage unit, formatted it to all my ores and ingots and other misc items in that category and started to fill it up with some of the stuff I had laying around. To my dismay, I ran out of bytes... Not even half way done with storing my ores and ingots yet either (Picture Below). So what is the best way of doing things in AE? Not just storing ores and such but for other items as well. I know to store cobblestone, Dirt, Gravel and other items that you get absurd amounts of in a DSU. However what do I do with the other items? Should I store them all in barrels and just connect them with storage busses? Or is there a better option. I would really like to hear your input on this. Thanks, -Jack

 

[netmc]

Well, the fewer number of different items in a storage cell, the more of the items it can hold. Just 1 item of 63 different items will take about half the storage space of the cell. I personally would use an ME drive and store the disks that way instead of in chests. I would put a bunch of 1k cells in it, then start dumping in my items. Any items that I have a lot of, would go into their own pre-formated cell, then used with the io manager(?) to transfer all the items from the network into the new cell. This would free up considerable amount of space from the quickly filling 1k drives. Anything that has a bunch of items (cobble, dirt, sand, etc.) would go into DSUs (or 64k cells).

In the case of what is in your screen shot, I would put them gems in their own pre-formated 4k cell. Redstone would also go in its own. same for copper, tin, iron, and maybe silver. Everything else would go in 1 1k/4k cell.

 

[hiroshi42]

Well, first, what is efficiency, space or materials? I generally have a large (64k) storage for ingots and whatnot that I know I am going to have a lot of, but I've heard that the most material efficient method is to store anything you are going to have several hundred or more of in a jabba barrel with a storage bus or into it's own dedicated 1k or 4k storage cell

 

[Jack0928PC]

Well, the fewer number of different items in a storage cell, the more of the items it can hold. Just 1 item of 63 different items will take about half the storage space of the cell. I personally would use an ME drive and store the disks that way instead of in chests. I would put a bunch of 1k cells in it, then start dumping in my items. Any items that I have a lot of, would go into their own pre-formated cell, then used with the io manager(?) to transfer all the items from the network into the new cell. This would free up considerable amount of space from the quickly filling 1k drives. Anything that has a bunch of items (cobble, dirt, sand, etc.) would go into DSUs (or 64k cells).

In the case of what is in your screen shot, I would put them gems in their own pre-formated 4k cell. Redstone would also go in its own. same for copper, tin, iron, and maybe silver. Everything else would go in 1 1k/4k cell.

So from what im taking from this is to make a couple drives to store all the 1k cells. Then format the cells to 1 item? Also are you saying I should make a 4k cell for all the items in which I cumulate a couple thousand off. Your response seems fairly self explanatory, but I just want to clear up exactly what I should be doing. Thanks

 

[netmc]

That's about right. I personally start with about 6 1k drives, and anything i have over 1k of goes into their own 1k drive or shared 4k drive. Once it hits about 3k our so, I start putting it in its own 4k. At about 10k, I drop it in a dsu.

This is of course dependant on the amount of quartz I have available to make drives.

I eventually upgrade to 4k drives for all my random stuff, and DSUs for all my high quantity items.

 

[BreezyTaco]

I was just wondering how I should set up my applied energistics system on my recent world, this thread helped a lot!

 

[kaovalin]

Screw efficiency, give each item its own personal 64k drive! But seriously, preformat your drives people. 63 items in a single drive completely kills the storage capacity of the drive. I have extra cells so I plan to horde as few men have horded before.

 

[netmc]

One of the other tricks I use... When I offload a bunch of items into its own pre-formated cell, it leaves partial disks everywhere. so, I take the last 3-4 random junk drives out of my ME drive system, then dump them in the io import. This will pull out any items possible, and place them back in the network. This will leave me with a mostly empty drive in many cases, and several of the drives that were orange (already containing 63 types, but not out of space), will fill into red. I then repeat that with all the remaining disks and condense the storage into as few drives as possible, freeing up type storage on the last few drives. When I am down to the last drive or two with free space (green status), I will make a couple new drives, or upgrade an empty one to the next size up.

 

[kaovalin]

One of the other tricks I use... When I offload a bunch of items into its own pre-formated cell, it leaves partial disks everywhere. so, I take the last 3-4 random junk drives out of my ME drive system, then dump them in the io import. This will pull out any items possible, and place them back in the network. This will leave me with a mostly empty drive in many cases, and several of the drives that were orange (already containing 63 types, but not out of space), will fill into red. I then repeat that with all the remaining disks and condense the storage into as few drives as possible, freeing up type storage on the last few drives. When I am down to the last drive or two with free space (green status), I will make a couple new drives, or upgrade an empty one to the next size up.

That reminds me of a great tutorial that showed how to perform this defragmentation automatically. AE really needs to get on defragmentation as an automated block. Wouldnt mind if it needed some drives in it to function either.

 

[midi_sec]

i can't imagine people running AE networks that use one disc per item on a server with the population the one i play on has.

a lot of setting up your network depends on your play environment. don't be that guy that hogs all the tps because you want your "supar awsome ME unit" that does everything and controls everything all at the same time and stores everything and sometimes makes me espress...no. we hunt those people down and take their things.

 

[netmc]

That reminds me of a great tutorial that showed how to perform this defragmentation automatically. AE really needs to get on defragmentation as an automated block. Wouldnt mind if it needed some drives in it to function either.

I remember seeing this as a feature request FAQ that was denied.

 

[kaovalin]

i can't imagine people running AE networks that use one disc per item on a server with the population the one i play on has.

a lot of setting up your network depends on your play environment. don't be that guy that hogs all the tps because you want your "supar awsome ME unit" that does everything and controls everything all at the same time and stores everything and sometimes makes me espress...no. we hunt those people down and take their things.

Well its my server so the TPSs are all belong to us now . If I was playing on a big server, I would expect it to be able to handle anything I've ever built. I dont make things sloppily, but I do make them to scale once I get the materials. Once I get AE going, I make it a point to never use a crafting table again from then on.

 

[midi_sec]

If I was playing on a big server, I would expect it to be able to handle anything I've ever built. I dont make things sloppily, but I do make them to scale once I get the materials. Once I get AE going, I make it a point to never use a crafting table again from then on.

now imagine that of the 80 people online, 30 people on are doing exactly that. their entire network is one sprawling mass. they can all autocraft quantum armor, or some other absurd block, and are all doing it.

tps tanks when you allow people to build ME networks resembling octopii.

 

[kaovalin]

now imagine that of the 80 people online, 30 people on are doing exactly that. their entire network is one sprawling mass. they can all autocraft quantum armor, or some other absurd block, and are all doing it.

tps tanks when you allow people to build ME networks resembling octopii.

Im sorry but I really disagree. I shouldnt have to care what other people are doing on the server, unless they are building something the community can use so I dont have to. If I see a server that says it can handle 80 people then I expect it to handle 80 people doing realistic builds in a modded environment. If it cant, then its the fault of the administrators for overstating its ability. Its not a vanilla server, mods add ticks.

Its also the server owner's fault for choosing a mod pack that was outside the parameters of the experience they wanted for their users. People arent always going to play in expected ways, and too many rules eventually start cutting into the fun for everyone.

That being said, I think we ultimately agree but are picturing different scales and different levels of build competence. Because build competence will kill TPS faster than scale. When I originally stated a disk for every item, I dont think anyone would do so, that was clear hyperbole.

 

[midi_sec]

sorry, but i completely disagree.

it is possible to build an ME network in such a way that it does not ping the entire network (even portions unused by that operation) to smelt a single piece of stone.

 

[kaovalin]

sorry, but i completely disagree.

it is possible to build an ME network in such a way that it does not ping the entire network (even portions unused by that operation) to smelt a single piece of stone.

Sure, which is why you tell AE to smelt stacks at a time to save on how often it has to perform said operations (1/64th if you're keeping track). Or even better, have a TE igneous extruder pump directly into a barrel that is functioning as storage devices for the AE network and get around the whole issue. I fail to see how I cant have a factory that does everything if I build it responsibly. I also dont see why you think everything will exclusively use AE. Who would want the energy overhead?

 

[midi_sec]

because that is how it is. everybody wants their AE to do everything for them. harvest my wood, store my ingots, grind my ores, run my gregtech machines, keep track of my bees, process my combs, oh yeah by the way could you autocraft me some alveary blocks? thx

hypothetically tho, if you're not processing ore your ME cables connecting to your ore processing machines should not be lighting up while you're not processing ore, if they are lighting up, those are unnecessary calculations for blocks that you caused the server to perform. a more responsible way to build would be to subnetwork your AE, or at the very least, use a piece of dark cable and a lever to shut down unused portions of your network.

edit: again, this is just for large servers. maybe mine is the exception rather than the rule, because we do seem to have quite a few advanced groups of people. the economy is a driver to build more and bigger, so they've found ways to squeeze the most out of smaller footprint setups.

 

[kaovalin]

because that is how it is. everybody wants their AE to do everything for them. harvest my wood, store my ingots, grind my ores, run my gregtech machines, keep track of my bees, process my combs, oh yeah by the way could you autocraft me some alveary blocks? thx

hypothetically tho, if you're not processing ore your ME cables connecting to your ore processing machines should not be lighting up while you're not processing ore, if they are lighting up, those are unnecessary calculations for blocks that you caused the server to perform. a more responsible way to build would be to subnetwork your AE, or at the very least, use a piece of dark cable and a lever to shut down unused portions of your network.

edit: again, this is just for large servers. maybe mine is the exception rather than the rule, because we do seem to have quite a few advanced groups of people. the economy is a driver to build more and bigger, so they've found ways to squeeze the most out of smaller footprint setups.

As an example, I have a stage where my income from quarrying/automatic mining comes into proprocessing before it hits my AE system. This is where it decides if it needs to be ground up, smelted, etc. Then from here it goes into the AE storage. Directly attached to my MAC I have several IO things with all the recipes for each type of action. One for smelting, one for grinding, etc. Attached to those IO things are tesseracts going to a group of machines for said function. My processing is done in bulk and is distributed among the machines using routers. The output of these groups are put back through a tesseract into the direct AE input.

Given said setup, is this too much for what your server has allowed? I only mean to get specific because I don't know for sure how complex of a system is looked down on outside my own experience.

Additionally, I made my server a lot happier when I got rid of the whole EU universe. It was the main driver for making a huge automated factory in 1.4.7. I also intend to do more communal builds so everyone can benefit from my factory constructing ability. Less lag for all that way really.

 

[midi_sec]

your setup sounds like it could be good, i'd have to poke around though.

the bad thing about AE and large servers is even assuming an ideal scenario with every person with an advanced AE network using an ideal layout for max friendliness to the server, you'd still run into a certain threshold of networks where the sheer number of calculations done by all AE networks globally starts to bog the server down. you can't hold sysops accountable for that.

kind of one of those "not everyone can be an astronaut" type things, not everyone needs a massive AE on a large server

 

[kaovalin]

your setup sounds like it could be good, i'd have to poke around though.

the bad thing about AE and large servers is even assuming an ideal scenario with every person with an advanced AE network using an ideal layout for max friendliness to the server, you'd still run into a certain threshold of networks where the sheer number of calculations done by all AE networks globally starts to bog the server down. you can't hold sysops accountable for that.

kind of one of those "not everyone can be an astronaut" type things, not everyone needs a massive AE on a large server

Well that is sort've to my point, maybe the server is overstating its ability to service that many users effectively. No situation is ideal, it should be assumed things are not ideal. I can sympathise though that large servers will have the worst time balancing everyones playstyles and expectations, but I still believe its sysops who sets the tone of the server. The whole mod's point is automation and storage after all.

A great way to help players play by the rules without making it seem like it is to give specific goals to have them take targeted approaches to things. Each accomplishment can be rewarded. You know, basic psychology

Edit: I do hope my replies dont come across as turse. I just find this to be a really interesting exchange since I dont play with a server with that many users. I'm sure there are things I've never thought about that might make my server better. In the end, that's all I look to do for my server.