Huge TPS drop and Block Lag - Unleashed 1.1.3 w/MCPC+

Cozza

New Member
Jul 29, 2019
320
0
0
Hey Guys,

Over the last couple of days my server has experienced a very large TPS drop and many players are experiencing block lag.

I can't seem to track it down trying everything I can possibly think of. The server just seems to be backed up with ticking entities.

If anyone can help out that would be great!

Bukkit timings: http://aikar.co/timings.php?url=6010165
aPerf readout of overworld entities: http://imgur.com/IWXxQQJ
VisualVM snapshot : https://dl.dropboxusercontent.com/u/82904851/Snapshot.nps

java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1~deb7u1)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
Startup:
Code:
command = "{JAVA}" -server -Xmx20480M -Xms2048M -Xmn1024m -XX:TargetSurvivorRatio=80 -XX:MaxTenuringThreshold=31 -XX:+UseG1GC -XX:MaxGCPauseMillis=40 -XX:GCPauseIntervalMillis=400 -XX:+AggressiveOpts -XX:PermSize=1024m -XX:+TieredCompilation -XX:ReservedCodeCacheSize=2048m -XX:+UseCodeCacheFlushing -XX:CompileThreshold=1000 -jar "{JAR}" nogui
Specs:
Xeon E3-1270 V3
32GB Ram
1GB/s Uplink
The server floats between 8-14 TPS with even 1 player online. CPU sits at around 20% with 10 players online.
 

cjm721

New Member
Jul 29, 2019
734
0
1
Well definitely something with rail-craft. You could get Tickprofiler (Just the profiling aspects of tickthreading) for 1.5.2 and that should help you track it down to location. Go there and remove it.
 

cjm721

New Member
Jul 29, 2019
734
0
1
I go to where those MFSU's are and turn them off. I suspect there is an infinite power loop there (there should never be 64k calls to MFSUs. The Spigiot timming did say it was tile entities causing the issue.
 

Cozza

New Member
Jul 29, 2019
320
0
0
Thanks cjm, power loops seem to be the big hitter.

There was a power loop about 500 cables away from the MFSU's on an array of about 300 electric engines. Fixed the loop and my players are reporting a much better experience.

I've been finding them all over the place. Especially with redstone energy cells.

Thanks for the help.
 

rroethof

New Member
Jul 29, 2019
2
0
0
Hey Cozza,

Might I ask you how you have managed to get MCPC+ working with Unleashed 1.1.3 ?
At this moment we are working on our new server, but it is still based on Ultimate due to MCPC, I would like to have Unleashed instead of Ultimate due to the newer version of MC and more options.

With regards,

Ronny
 

cjm721

New Member
Jul 29, 2019
734
0
1
Same way you got ultimate work. Look on md_5's build server and you will see the 1.5 versions.
 

rroethof

New Member
Jul 29, 2019
2
0
0
I did just that, downloaded the "mcpc-plus-1.5.2-R1.1-forge738-B639.jar" and added that into my script.
But it keeps complaining about the FML

Code:
2013-08-27 06:33:41 [INFO] [ForgeModLoader] Forge Mod Loader version 5.2.23.738 for Minecraft 1.5.2 loading
2013-08-27 06:33:41 [INFO] [ForgeModLoader] Java is OpenJDK 64-Bit Server VM, version 1.7.0_25, running on Linux:amd64:3.2.0-51-generic, installed at /usr/lib/jvm/java-7-openjdk-amd64/jre
2013-08-27 06:33:41 [WARNING] [ForgeModLoader] The coremod codechicken.core.asm.CodeChickenCorePlugin does not have a MCVersion annotation, it may cause issues with this version of Minecraft
2013-08-27 06:33:41 [WARNING] [ForgeModLoader] The coremod codechicken.nei.asm.NEICorePlugin does not have a MCVersion annotation, it may cause issues with this version of Minecraft
2013-08-27 06:33:41 [WARNING] [ForgeModLoader] The coremod powercrystals.core.CoreLoader does not have a MCVersion annotation, it may cause issues with this version of Minecraft
2013-08-27 06:33:41 [WARNING] [ForgeModLoader] The coremod miscperipherals.asm.MiscPeripheralsPlugin does not have a MCVersion annotation, it may cause issues with this version of Minecraft
2013-08-27 06:33:41 [INFO] [MiscPeripheralsASM] Initialized
Exception in thread "main" java.lang.RuntimeException: java.lang.NoClassDefFoundError: cpw/mods/fml/common/asm/ASMTransformer
    at cpw.mods.fml.relauncher.FMLRelauncher.setupHome(FMLRelauncher.java:192)
    at cpw.mods.fml.relauncher.FMLRelauncher.relaunchServer(FMLRelauncher.java:147)
    at cpw.mods.fml.relauncher.FMLRelauncher.handleServerRelaunch(FMLRelauncher.java:45)
    at net.minecraft.server.MinecraftServer.main(MinecraftServer.java:1621)
    at org.bukkit.craftbukkit.Main.main(Main.java:21)
Caused by: java.lang.NoClassDefFoundError: cpw/mods/fml/common/asm/ASMTransformer
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:788)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:447)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at cpw.mods.fml.relauncher.RelaunchClassLoader.findClass(RelaunchClassLoader.java:195)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at cpw.mods.fml.relauncher.RelaunchClassLoader.registerTransformer(RelaunchClassLoader.java:157)
    at cpw.mods.fml.relauncher.RelaunchLibraryManager.handleLaunch(RelaunchLibraryManager.java:272)
    at cpw.mods.fml.relauncher.FMLRelauncher.setupHome(FMLRelauncher.java:172)
    ... 4 more
Caused by: java.lang.ClassNotFoundException: cpw.mods.fml.common.asm.ASMTransformer
    at cpw.mods.fml.relauncher.RelaunchClassLoader.findClass(RelaunchClassLoader.java:273)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 19 more
Caused by: java.lang.NoClassDefFoundError: net/minecraftforge/event/Event
    at net.minecraftforge.transformers.EventTransformer.buildEvents(EventTransformer.java:59)
    at net.minecraftforge.transformers.EventTransformer.transform(EventTransformer.java:35)
    at cpw.mods.fml.relauncher.RelaunchClassLoader.runTransformers(RelaunchClassLoader.java:387)
    at cpw.mods.fml.relauncher.RelaunchClassLoader.findClass(RelaunchClassLoader.java:260)
    ... 21 more
Caused by: java.lang.ClassNotFoundException: net.minecraftforge.event.Event
    at cpw.mods.fml.relauncher.RelaunchClassLoader.findClass(RelaunchClassLoader.java:273)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 25 more
Caused by: java.lang.AbstractMethodError
    at cpw.mods.fml.relauncher.RelaunchClassLoader.runTransformers(RelaunchClassLoader.java:387)
    at cpw.mods.fml.relauncher.RelaunchClassLoader.findClass(RelaunchClassLoader.java:260)
    ... 27 more
 

Cozza

New Member
Jul 29, 2019
320
0
0
Just an update to my original issue. Myself and players are still experiencing issues with Block lag and rubberbanding, ping tests and tps are all fine. TPS is fairly consistent at 20.

I'm focusing my attention on the current issues with IC2 and believe that may be the cause. Here's the research i've done so far:

With IC2 364 the console is getting spammed with:

[Server] WARNING API ERROR: ic2.core.block.machine.tileentity.TileEntityMatter@79140b4d (0:494,51,-114) didn't implement demandsEnergy() properly, no energy from injectEnergy accepted (512) although demandsEnergy() requested 512.

The issue appears to crop up when a mass fabricator is using scrap/scrapboxes. 364 also appears to have bad scrap->UU matter conversion. Essentially the mass fab is acting like it has scrap when there is none, so when there is scrap we start receiving the errors. (Mentioned here, and confirmed by player over on the IC2 forums)

Now this is where it starts to get interesting:

The FTB Team is currently testing IC2 Build #384. It's been documented by Nallar and bloodmc that builds #378-385 are not working correctly in MCPC+, EU is not flowing through cables in certain scenarios. (here).

So digging further into the rabbit hole, there's is a few more mentions of the issue. Here and here bloodmc is telling us to use the latest IC2 build (#385) and is also confirming builds #364 & 374 flowing energy correctly. So okay, we shoot for #385 even though bloodmc has told us #378+ is broken and working at the same time?

Here's our next issue. Build #385 appears to be somewhat incompatible with Advanced Solars. This is what the console looks like as soon as you boot the server. This console spam is also occurring in #384 which the FTB team is currently testing for Unleashed 1.1.4.

I'm completely lost as to how this can be fixed.
 

cjm721

New Member
Jul 29, 2019
734
0
1
I have the IC2 Mass fab spam also, but have not really noticed any adverse effects due to it. As for incompatibility I say tell the FTB team to use a more update to day version (if they do use an incompatible version I won't update to it personaly)
 

Cozza

New Member
Jul 29, 2019
320
0
0
I have the IC2 Mass fab spam also, but have not really noticed any adverse effects due to it. As for incompatibility I say tell the FTB team to use a more update to day version (if they do use an incompatible version I won't update to it personaly)

Hi cjm,

How many mass fab's are normally running? I often see 4-5 throwing the error and many of my players are putting well over 10k eu/t through their mass fabs. Is it possible the issue is there for you but not as severe?

The issue also is that build #364 has the bug of mass fabs always running like they have scrap. (The cause of the API error) Build #364 is what Unleashed 1.1.3 is currently using.

While the FTB Team is testing build #384, the console is flooded with Advanced Solar errors on that build and build #385 which is the last IC2 1.5.2 version. There just doesn't seem to be an IC2 build without problems.

I'm going to test without MCPC+ tonight and see what happens with the latest IC2 builds.
 

Vauthil

New Member
Jul 29, 2019
1,491
-14
1
#384 and #385 are functionally identical except for an internal version number increment, FYI. All the #385 build did was make it version 1.117 instead of version 1.116. #386 then officially began the transition into actual 1.6.x builds.

Unfortunately you're dealing with a bit of a Sophie's choice here due to the interactions at play. I can understand that. It's a complicated situation. I expect to hear a fair amount of rawr on this. It's unfortunately the nature of the beast, particularly given MCPC+'s mission of making disparate and often downright hostile APIs work together in tandem. Mod authors willing to adjust in that environment are the exception and not the rule. I think a lot of people didn't understand/don't remember that back in the 1.2.5 era, many mod authors had flat out delegated third parties to write the bukkit-compatible ports to their mods, and washed their hands of having to support those ports. When bukkit made it abundantly clear they have no interest in meeting the forge community anywhere near the middle, a lot of that got dropped in the aftermath, making the MCPC team's task Herculean when not downright Sisyphean. And that's where we're at now with a lot of these issues.
 

Cozza

New Member
Jul 29, 2019
320
0
0
I agree Vauthil,

While I haven't tested without MCPC yet and I don't blame the FTB team for any of the issues im having. I am very worried about what is going to happen if 1.1.4 rolls out with IC2 build #384, simply because of the amount of "feedback" you will hear from MCPC+ users.

My server actually thrived this month and has been slowly on the decline due to what appears to be "mystery block lag and rubber banding". Everything I test comes up good but the issue is still present.

Where MCPC has very good advantages which makes an admins life much easier. It always seems to be plauged with bugs, some of which are utterly gamebreaking and your up to the mercy of the MCPC team to try and fix it.

Like I said, i'll be testing without MCPC tonight and seeing what happens.
 

cjm721

New Member
Jul 29, 2019
734
0
1
I say about 20 Mass fabs with two of mine both running over 50kEU/t. The posting to log as long as it is not over 10 per second usually causes no issues.
 

cjm721

New Member
Jul 29, 2019
734
0
1
#384 and #385 are functionally identical except for an internal version number increment, FYI. All the #385 build did was make it version 1.117 instead of version 1.116. #386 then officially began the transition into actual 1.6.x builds.

Unfortunately you're dealing with a bit of a Sophie's choice here due to the interactions at play. I can understand that. It's a complicated situation. I expect to hear a fair amount of rawr on this. It's unfortunately the nature of the beast, particularly given MCPC+'s mission of making disparate and often downright hostile APIs work together in tandem. Mod authors willing to adjust in that environment are the exception and not the rule. I think a lot of people didn't understand/don't remember that back in the 1.2.5 era, many mod authors had flat out delegated third parties to write the bukkit-compatible ports to their mods, and washed their hands of having to support those ports. When bukkit made it abundantly clear they have no interest in meeting the forge community anywhere near the middle, a lot of that got dropped in the aftermath, making the MCPC team's task Herculean when not downright Sisyphean. And that's where we're at now with a lot of these issues.

I remember back then and both sides blamed the other for saying they did not want to collaborate on designs. So as to the truth we probably will never know as none of us were in those discussions to see what was really said so we can't blame that on the Forge or Bukkit.

The one thing that is going to become an issue is when Bukkit becomes the official server jar, that is a question of when not if (unless Mojang wants to throw alot of money out the window). Vanilla Minecraft right now already has several classes for dealing with communication with Bukkit plugins and as the versions advance I except to see more and more integration. As that comes we also have Mojang's team that is supposely working on the Mod API (Minecon being the source of that info) and then a bit more heresay on that info is that the Bukkit team was asked to start work on it who already have a stable clean API for plugins (I will admit there is alot I can't do and even less now in recent updates with the changes to NMS) but what it does lack does help alot with stability. The day the MOD API will come out will probably not be until 1.8 at least unless all the teams decide to work on it instead of adding new features, but when it does if forge is not already collaborating with the team that is making it then I am guessing alot of mod authors are just going to quit due to the massive changes there is going to be. Now with the recent update to MCP/forge that allows for a deobfuscated run time is first step into making moding Minecraft not only easier but more compatible from version to version.

As for server owners running default server I can get about 20-30 players on and playing before server goes bellow 15TPS but with MCPC I can up near 100 players. It does not help alot of mods are not written to be used in mass scale at all which is partially what made TickThreading so much of a performance boost because it separated lots of the servers tasks (which where al in one thread) so that you could use multiple cores of you CPU and at the same time since none of the mods FTB uses (not 100% sure about this) are designed to be multi-threaded there was alot of issues with running TT at first but now for 1.4.7 has been mostly ironed out. So from an owners perspective what am I do to? If I run default server then I can only have about 25 people on at once which is I was funding the server via donations would be right on the line of not being able to fund itself, or do I swich to a program which works reasonably well so I can 4-5x the amount of people I have on. At the same time if I want to run a public server then what rules do I want to set. If I make it no griefing/stealing what will I use to protect it / log what people do? If I state that the server is no griefing then I am expected if it ever does happen to reprimand the person that did it (if it can even be found out) or prevent it in the first place which leads to using mods like MyTown which just switched authors. In some of the mod packs there is MFFS which is you use v2 is very good at protecting from explosions if you have the power while v3 (the UE version) is very good at preventing people from doing anything but has no real protection from tnt/nuke cannons), or You can give everyone a mystcraft age but then you have to deal with if they want to show it to someone and they person backstabs them etc. So do I whitelist my server then which will cause a dramatic reduction of the amount of people that play and ie funding for it. The other route which is what I have gone is made the server raid / grief / pvp allowed so I can deal with fixing mod bugs and watching for people who want to use fly/xray mods, but by doing that I use a considerable amount of the community because most people don't want to use their stuff but at the same time I don't have to deal with the constant Person A broke ___ rule.

Either way its a double edged sword for using alternative Server jars and extra mods but at the same time if noone did then alot of servers would not be able to support the communities they have behind them and personally I believe FTB and modded Minecraft in general would not be where it is today without them.

Also on that note I have had rather good success with tieing multiple servers together so I can just type a command to switch between them which allows for "multithreading" of servers but if something goes wrong I am my only source of help, but as I was talking about before its how you weigh the pros and cons as I can deal with Trading post not working (got permission to edit the mod server side so I might be able to fix that soon), and a few things just not always visually updating..

One thing I always have found ironic is that when 1.6 updated I was sitting on IRC in forge's channel when someone from forge dev team (don't think it was lex) asked if special source was updated to they could start working on forge for 1.6 and md_5 posted the link. What is the ironic part about that is md_5 makes special source but also makes Spigiot and is the 5th highest contributor to MCPC+, so the person forge was waiting for so they could start 1.6 development is also the person that makes the performance Bukkit server (Spigot).


Random note which is kinda neat to look at: http://metrics.griefcraft.com/plugin/Spigot#Server+Software
(Spigot is what MCPC is built on and made by mostly the same people)[DOUBLEPOST=1377752592][/DOUBLEPOST]
I agree Vauthil,

While I haven't tested without MCPC yet and I don't blame the FTB team for any of the issues im having. I am very worried about what is going to happen if 1.1.4 rolls out with IC2 build #384, simply because of the amount of "feedback" you will hear from MCPC+ users.

My server actually thrived this month and has been slowly on the decline due to what appears to be "mystery block lag and rubber banding". Everything I test comes up good but the issue is still present.

Where MCPC has very good advantages which makes an admins life much easier. It always seems to be plauged with bugs, some of which are utterly gamebreaking and your up to the mercy of the MCPC team to try and fix it.

Like I said, i'll be testing without MCPC tonight and seeing what happens.

Did not see your post as I was writing my rather long post above. It might be that someone has an MFFS running as that can cause packet spam which can easily cause the issue your seeing also.
 

cjm721

New Member
Jul 29, 2019
734
0
1

Ok I had not seen the most of that post.

...Read the entire thread this time....

Getting off topic of this thread but looking at that post changes my perspective a bit but the one thing I am still stuck on is either Bukkit is a bunch of pomps pricks or cpw/lex really never new that minecraft.net had the server jar which is what mc-dev is based on (which I think they knew about) seems as if the Bukkit side was taking that forge needed how to decompile while they were really asking for the de-obfuscater remappings which I am rather confused on why the entire Bukkit team seemed to think that info was public. Something about that entire topic seems off to me and what my memory is telling me about that situation as I was making bukkit plugins back then and looking at forge/modloader and trying to figure out how to run it (the decompile -> deobfu really screwed me up back then and now that I have had over a year of coding experiance in java I can do it now for forge.

Considering I had never seen the Bukkit team be so unprofessional in any thread like that (not like I watched their forums closely either) posts like gravity would get you kicked out by most places and especially after Bukkit was merged into Mojang (when will they get integrated as they still don't have direct access...). As for Bukkit working with forge that peaty clearly states Bukkit wont spend any time with anything they have not coded. Which none of it answers what’s going to happen when the mod API comes out which is where alot of my concerns are because where is forge going to be? That is one of the reasons I am hesitant to really start working on my own standalone mod if I know its going to need entirely recoded on a different system.

Thankyou for the link Vauthil going to give me alot more to think about.
 

Cozza

New Member
Jul 29, 2019
320
0
0
Did not see your post as I was writing my rather long post above. It might be that someone has an MFFS running as that can cause packet spam which can easily cause the issue your seeing also.

This would show up on TickProfiler though, right?

An interesting note, TickProfiler shows the server running at around 13-14 TPS during a 60 seconds profile where running /tps shows 20 TPS fairly consistently.
 

cjm721

New Member
Jul 29, 2019
734
0
1
This would show up on TickProfiler though, right?

An interesting note, TickProfiler shows the server running at around 13-14 TPS during a 60 seconds profile where running /tps shows 20 TPS fairly consistently.

/tps is not very accurate and Nallar has a post somewhere explaining why. And if its packet spam then it might not show up on a profile. But from that profile that should show the chunks that are taking the most cpu.