Problem FTB Unleashed 1.1.3 Insane RAM usage? (server kills?)

[Revamp1984]

• What OS are you running? I am not able to tell you.
• Are you using minecraft hosting provider or a dedicated/local server not designed for minecraft? Hosting Provider (Creeperhost)
• What version of FTB are you using? (Don't just say latest tell us the version) FTB Unleashed 1.1.3 /w mcpc-plus-1.5.2-R1.1-forge738-B635.jar & TickThreading-1.5.2.186
• Did you add any mods to the server pack? None added, all Enabled
• What are your server specs?
  Spider
  RAM 3072MB HDD 40GB CPU 2 cores Data transfer Unlimited* Platform Virtual Server
• A good detailed paragraph about the problem

A long story short, with FTB Unleashed 1.1.3 (without MCPC & TickThreading) my RAM usage went sky high.
Regardless If we had 1 or 5 people on the server, it would start at 50% and would very rapidly reach 87- 90%
This would in our case result in a server kill (the server kills itself, no crash logs)
On 1st sight we thought it was a RAM problem, so we upgraded our package from Zombie to Spider.
This did not do the trick, so MCPC+ and TickThreader where installed, how ever this on its own did not solve anything, the server kept killing itself after several hours running.

TT was suggesting world leaks, even in worlds we did not enter yet.
As a result we destroyed all the portals, and replaced them with linking books to minimize risk of creatures being stuck in a portal.
After which we also put a chunkloader in every world known this to make sure everything was properly loaded / unloaded when needed.

Unfortunately also this did not solve the problem, although TT was no longer reporting near as much possible leaks.
RAM usage stays at an all time high, again it does not matter if we are alone on the server or with a group, I am currently alone and on 87% RAM usage.
I have reinstalled the server, with the latest TT and MCPC together with a fresh install of FTB 1.1.3.
TT no longer gives me (as I type is) any warnings about world leaks, if it where not for running the console in debug mode, everything would seem fine (apart from the high RAM usage)
If I leave the server running long enough without a save & restart, the server will kill itself.

What it does give me is the following message:

03:09:43 [SEVERE] [TickThreading] Exception ticking TileEntity c powercrystals.minefactoryreloaded.tile.machine.TileEntityHarvester @1283637085 in Overworld/0 at x, y, z: 3037, 126, 3230
03:09:43 java.lang.NullPointerException
03:09:43 at powercrystals.minefactoryreloaded.tile.machine.TileEntityHarvester.activateMachine(TileEntityHarvester.java:145)
03:09:43 at powercrystals.minefactoryreloaded.tile.base.TileEntityFactoryPowered.func_70316_g(TileEntityFactoryPowered.java:133)
03:09:43 at nallar.tickthreading.minecraft.tickregion.TileEntityTickRegion.doTick(TileEntityTickRegion.java:90)
03:09:43 at nallar.tickthreading.minecraft.tickregion.TickRegion.run(TickRegion.java:35)
03:09:43 at nallar.tickthreading.minecraft.ThreadManager$2.run(ThreadManager.java:101)
03:09:43 at nallar.tickthreading.minecraft.ThreadManager$1.run(ThreadManager.java:43)
03:09:43 at nallar.tickthreading.util.FakeServerThread.run(FakeServerThread.java:16)
03:13:33 [SEVERE] [TickThreading] Exception ticking TileEntity c powercrystals.minefactoryreloaded.tile.machine.TileEntityHarvester @1283637085 in Overworld/0 at x, y, z: 3037, 126, 3230
03:13:33 Stack trace unavailable for java.lang.NullPointerException-java.lang.NullPointerException. Add -XX:-OmitStackTraceInFastThrow to your java parameters to see all stack traces.

Now I know this has to do with the Harvester that I have in my base, yet I can not seem to find what exactly is wrong with it, nether can i find how to fix this, as I do not know what is wrong with it.
If anyone can explain this to me, this would be very helpful.

And then my final question, is this MFR Harvester issue what is creating the insane RAM usage?
Or is it something in FTB Unleashed 1.1.3 that is causing it?
I would love to get some help on this one, as so far working with CreeperHost we haven't found a solution for this yet.

Greetz!

 

[DZCreeper]

Stop using Tick Threading, the mod author Nallar has stopped working on it and has no plans to continue working on it. It is open source which means others can work on it, but at this time no one has stepped forward.

To fix world leaks, just place a chunk loader in each dimension. Personally your memory issues sound like someone left a mob spawner turned on and chunk loaded. Before you remove Tick Threading, use its features to locate any chunks with high entity counts.

 

[Revamp1984]

How would one turn on a vanilla spawner? As far as I'm aware these will only spawn when someone is close by?
I will use TT to look for high entity counts, thanks for the tip, although this high RAM usage started before anyone had a mob spawner.

Anyone any idea's on the MFR warning?

 

[DZCreeper]

Vanilla spawners can be controlled via light levels but players nearby are needed. I was talking about Soul Shards or MFR Autospawners which can work with players much farther away or even offline.

The MFR warning is not actually MFR, its Tick Threading causing that and its a known problem which will not be fixed because the mod is no longer being updated.

 

[Revamp1984]

Okay, so assuming there are no spawners that are pouring over etc.
What can cause this insane RAM usage, and inevitable server kill?
I'm assuming having to /save-all /restart every 3 hours is not how its supposed to be.

 

[cjm721]

With using a creeperhost VPN I assume you do not know your start parameters or how to change them, but depending on how survivability ratios are changed and which GC you are using.

If you think its just a mod rename your world folder and see how your usage is. If its back low its something built probably a spawner or something to that extent. Tickthreading's profiler is still good even with the bugs that are in it. Just use profile e <time> <dimention> (or /profile e 7 to look at all worlds) and that will help you track down the laggy spot.

 

[Revamp1984]

1st of, you speak about a laggy spot, yet we do not have "lag" its just the RAM usage going skyhigh.

Rite, ive used /profile e7 and this is what it comes up with

https://dl.dropboxusercontent.com/u/81141130/2013-08-15_07.57.26.png
https://dl.dropboxusercontent.com/u/81141130/2013-08-15_07.57.28.png

Now, what I gather out of this is basically that my MFR Harvester and my Packager line is intensive?
And that my Cow-shack and Pipes are also intensive?
Or at least these things are more intensive then the other stuff.

How can this 1 platform, cause such high RAM usage? And eventually kill the server? (Remember, kill it, not crash it)

Is this just a clear sign that we should upgrade even further Hosting Package wise?
Should I alone need 4+ GB Ram to run this simple platform?

 

[cjm721]

Well before you go spending more memory try running a profiler on the ram and see/post the results of that.

If you got JDK or the server based java installed (No clue with creeper host and what they will allow access wise) you should have access to visualvm (named jvisualvm usally) which you can google how to use. Or YourKit is another one.

 

[Revamp1984]

Well before you go spending more memory try running a profiler on the ram and see/post the results of that.

If you got JDK or the server based java installed (No clue with creeper host and what they will allow access wise) you should have access to visualvm (named jvisualvm usally) which you can google how to use. Or YourKit is another one.

This is going way over my head atm, ive downloaded YourKit, but as soon as I load it up, im like a pensioner looking at the Iphone5 and I go uhhhhhhh

 

[cjm721]

I have not used yourkit just know alot of people recommend it. Visual VM is preaty simple. Open it select an active running jar then go to profile tab > memory > start

 

[TechStack]

Revamp1984 Your not alone.
I'm using 1.1.3 on creeperhost and have recently upgraded to a higher tier server ( Enderman package) and i've seen the server use upwards of 8 GB 11GB virtual memory. Eventually it will still crash like you suggest but it usually starts lagging bad first so i restart it before that is needed. I'm lucky to let it run for 24 hrs before problems start to happen.
I did notice when i installed 1.1.3 that creeperhost listed it as unsupported .. i wonder if this is why.
when ram usage is high /TPS reports numbers as low as 6 of 20 when things get out of hand. most of the TPS lag is showing as overall usually not a isolated in a single world.. i think it may be minecraft /java trying to manage all of its memory allocated and garbage collections kicking in.

if i get time today i may try VisualVM i'll let you know if i sort it out [DOUBLEPOST=1376590575][/DOUBLEPOST]Revamp1984 Your not alone.
I'm using 1.1.3 on creeperhost and have recently upgraded to a higher tier server ( Enderman package) and i've seen the server use upwards of 8 GB 11GB virtual memory. Eventually it will still crash like you suggest but it usually starts lagging bad first so i restart it before that is needed. I'm lucky to let it run for 24 hrs before problems start to happen.
I did notice when i installed 1.1.3 that creeperhost listed it as unsupported .. i wonder if this is why.
when ram usage is high /TPS reports numbers as low as 6 of 20 when things get out of hand. most of the TPS lag is showing as overall usually not a isolated in a single world.. i think it may be minecraft /java trying to manage all of its memory allocated and garbage collections kicking in.

if i get time today i may try VisualVM i'll let you know if i sort it out

 

[Revamp1984]

Hi, thanks for your reply

I just got suggested by CH that reinstall the server again, but this time leave out TT, MultiParts and WRCBE, so I guess that will be an all nighter.
I have been debating going for the dedicated server, but with your reports of RAM taking up to 11GB, I doubt that this is how it should be.
At the moment I'm managing with server restarts every hour, hopefully tomorrow things will run better.

Let me know if anything comes out of VisualVM, even though I don't understand it, I am certainly interested.

Edit: Hourly restarts are not fast enough with more then 3 players online

 

[TechStack]

I've got VisualVM working with my server now. not to bad to setup just need to go into the control panel in CH under Minecraft -> system properties and set it to enable Enable Profiler ( it says the port you'll need on the next line)

Then install the visualVM tool add a server with your Hostname / IP and then right click on that and add a JMX connection
It will pop a new window with your server IP : just add the port given from creeperhost to the end of that line and hit OK
then you should be able to connect to your server & profile the Java app.
I"ve been able to see what is taking the most CPU , still trying to see how to profile memory but i'm not running into out of memory problems as fast as you are.

I can understand why CH is saying to remove WRSCBE & Multipart -- one of the two caused a hard crash for me last week that forced me to use an older backup

 

[Revamp1984]

Alright, so here is my next step.
Ive reinstalled 2x
1st time I put in MCPC+ and let it run, made sure to turn of Multipart etc.
Within 10 min server crashed

I then decided to take out MCPC+ and install FTB clean, take out the add-ons requested and boot it up again.

The next msg is spammed in the console window

[ForgeModLoader] Received a tiny packet for a network mod that does not accept tiny packets

Googled it, but can not seem to find anything, anybody got any thoughts about this one?

 

[TechStack]

any crash reports in your crash-reports folder?

 

[Revamp1984]

None, server kills itself because of the MEM usage going sky-high (92%+ = kill) so it doesn't crash.
The only notable things I can find are that GateCopy message and

[WARNING] [IC2] API ERROR: ic2.core.block.machine.tileentity.TileEntityMatter@240d5d24 (0:380,65,-1757) didn't implement demandsEnergy() properly, no energy from injectEnergy accepted (512) although demandsEnergy() requested 512

The GateCopy gets spammed, as I only see the ic2 one, once or twice

 

[TechStack]

OK interesting. I also get the IC2 API error i don't think its causing much harm but is very annoying I've tracked it down to matter fabs (from the cords)

have you considered changing your settings in the control pannel to only allow minecraft to use say 2 GB of your total 3 GB allowed in your VPS. it may allow it to keep running long enough to get the out of error message possibly ... obviously this is just for debugging and you wouldn't want to keep it that way.

Its a setting in the CP for CH.

 

[Revamp1984]

Ill have a go at that right now!
Any idea on the GateCopy?

 

[TechStack]

GateCopy ? sorry i'm not seeing that in the thread...

[edit] oh do you mean "[ForgeModLoader] Received a tiny packet for a network mod that does not accept tiny packets" ? i did find one hit on google but it wasnt any use it was for the tekit pack ... but they did point to Minefactoryreloaded

 

[Revamp1984]

Random: Setting a max for the MEM (so not selecting adaptive) keeps my MEM at 78% max.