Causes of massive tick rate lag on my server

[Brian Cherrick]

Keep in mind that I play solo on my server . I have restarts scheduled every 12 hours, and immediately after a restart it is fine, but the following screenshot was taken about 7 hours after the restart, and tick rate is down to 12, with tick time up to 82.8 ms.



I have 4 gigs on the server (which I'm only using 55% of), and 7kish dedicated to the client, which I'm using 46% of.

Could any of the following be the cause of it?

1. A hefty amount of crafters?
2. The tier 5, and tier 6 vein ore miner outside?
3. The tree farm outside?
4. The turbine in the basement? (do keep in mind, it is completely encased with casings ... no glass)
5. Could it be my Mekanism turbine?
   
6. Could it be either of the big reactors powering the turbines?
   
7. Could it be any of the refined storage cables running around the base? (I don't have many of the remote networks set up)
8. Could it be anything related to my Draconic fusion crafting setup?
9. Could it be the Tier 8 energy ball that rests over my temple?
10. Any of the solar panels from ET? (currently have 1 T5, and 3 T4's)
11. Mob Farm?
12. Any of the mods? (list below)
13. Power transportation around the base, which is Draconic Wireless.

I have done the following in an effort to alleviate the issue, but to no avail:

1. Removed my crop farm in favor of cloches
2. Removed several vein ore miners, even though I had the power to operate there.
3. Removed my water mill for gp setup.

I am really unsure of what to do, and the server owners suggestion after moving me from one node to another is to basically start from scratch. Not sure if his hardware can't handle the pack (which for any server operator I'd have a hard time believing they can't run 1 pack with 1 person on it).

Eliminating Compact Machines, Lost Cities, and TF would help a little. But even without those 3 installed my tick time is still around 70 ms. Not sure if it is worth noting or not, but I don't have any TPS issues when playing offline, which leads me to believe it is server equipment, and not anything I have set up per se.

Open to any and all suggestions.

 

[GreenZombie]

If this is your server, "WarmRoast" is a java profiler that can help you pin down which mod is causing grief.

Now, as to your described symptoms, it sounds most like you have a mod that is leaking memory in some way - because the performance slows down over the course of hours. If any particular set of machines is responsible for the tps issues, then the problem should manifest as soon as you visit the area and its loaded.

Unless you have scripts that run on startup and kill excess entities from the world.

 

[Psychicash]

Second GreenZombie. Also I'd like to say that, it looks like you have a bunch of machines in that room that you don't... interact directly with. I would suggest that you hide these machines in a room that is closed off. Could help the rendering. Also breaking and moving the blocks could help.

 

[Inaeo]

I'll have to give WarmRoast a look.

 

[Nuclear_Creeper0]

I know refined storage can cause server issues. Most public servers ban the use of the crafters because they cause excessive amounts of lag, cough DeVco cough.

 

[Brian Cherrick]

If this is your server, "WarmRoast" is a java profiler that can help you pin down which mod is causing grief.

Now, as to your described symptoms, it sounds most like you have a mod that is leaking memory in some way - because the performance slows down over the course of hours. If any particular set of machines is responsible for the tps issues, then the problem should manifest as soon as you visit the area and its loaded.

Unless you have scripts that run on startup and kill excess entities from the world.

Second GreenZombie. Also I'd like to say that, it looks like you have a bunch of machines in that room that you don't... interact directly with. I would suggest that you hide these machines in a room that is closed off. Could help the rendering. Also breaking and moving the blocks could help.

Where does this Warmroast program get installed at? It downloads as a jar file, and the directions aren't really clear as to what to do.

 

[Brian Cherrick]

In the meantime, I'm ditching the crafters for the big block crafter (ala AE1) that direwolf uses in his lets play to see if that alleviates any of the issue, or if it persists.

 

[GreenZombie]

Warmroast program get installed at? It downloads as a jar file, an

The jarfile is the program. The usage instructions are in the readme.md file on the github page you download it from:

https://github.com/sk89q/WarmRoast/blob/master/README.md

The basic usage is:
* Figure out the path to where Java is installed. This should be a Java Development Kit (JDK) not a Java Runtime Environment (JRE) install.
* Run Warmroast using a command like:
java -Djava.library.path=PATH_TO_JDK/jre/bin -cp PATH_TO_JDK/lib/tools.jar:warmroast-1.0.0-SNAPSHOT.jar com.sk89q.warmroast.WarmRoast --thread "Server thread"
* At the prompt - choose the Java VM that corresponds to your MC Server.

Now, warm roast is profiling your server.

* To see the results, leave warmroast running and navigate your browser to http://localhost:23000

You should see a screen something like this:

If the names make no sense, then follow the readmes instruction for downloading MCP for your version of Forge and adding its location to the command line passed to warmroast.

My JDK is installed program files, so I run warm roast with this command:

java -Djava.library.path=C:\Program Files\Java\jdk1.8.0_65/jre/bin -cp C:\Program Files\Java\jdk1.8.0_65/lib/tools.jar;warmroast-1.0.0-SNAPSHOT.jar com.sk89q.warmroast.WarmRoast --thread "Server thread" --mappings .\mcp\conf

 

[Brian Cherrick]

Turns out that warmroast isn't compatible with my server, but something similar is, Tickprofiler. So we shall see how things go.

On a side note what is a good hosting company that is competitively priced for a 3g to 4g server. I currently pay about 20 for 4g, but don't mind a little more if quality is higher.

Might even consider swapping server providers.

 

[GreenZombie]

Warmroast is *always* compatible. While it was nominally developed for profiling Minecraft, its a java VM profiler and could be used to profile any java app.

 

[Brian Cherrick]

Warmroast is *always* compatible. While it was nominally developed for profiling Minecraft, its a java VM profiler and could be used to profile any java app.

Just going by what I was told. But it takes time to manifest itself. So in time I can see what is happening.

I'm still not against switching providers, Just don't know the best one to go with.

 

[Brian Cherrick]

What I ran with TickProfiler .. not sure how to read it, but the provider things it's something in the world

Entity Profiler > https://pastebin.com/PUSaY6Gp
Packet Profiler > https://pastebin.com/Kiwa54Ve
Utilization Profiler > https://pastebin.com/cSAXJQjc
Contention Profiler > https://pastebin.com/UXppQTVN
Lag Spike Profiler > https://pastebin.com/PveH2FfW
Chunk Profiler > https://pastebin.com/Ms3Yfdfn
TPS > https://pastebin.com/VyMsxFzZ

Misc. Server Skipping Ticks > https://pastebin.com/BvwDVs3j

 

[GreenZombie]

Well, the Server tick log just confirms that the server knows its lagging too
Id say this is the typical pattern of the garbage collector taking too long but frankly any kind of mod wasting cpu cycles is going to manifest like this.
Question: What is the command line being used to launch your server?

In terms of your TPS report there do not appear to be a large number of Entities. But there are a lot of Tile Entities - over 7000. Have you been using carpenters / chisel and bits or something? You should avoid using Tile Entity style blocks for decorative purposes as they impose a processing cost for just existing and pretty much any block that is not a variant on a cube, stairs or tile is going to be a tile entity : Especially non cube blocks that can look like other blocks.

I don't know how to read the contention profiler, the network profiler doesn't indicate anything and the entity profiler doesn't indicate that you have a bunch of sheep in a pen causing AI pathing issues.

 

[Brian Cherrick]

Well, the Server tick log just confirms that the server knows its lagging too
Id say this is the typical pattern of the garbage collector taking too long but frankly any kind of mod wasting cpu cycles is going to manifest like this.
Question: What is the command line being used to launch your server?

In terms of your TPS report there do not appear to be a large number of Entities. But there are a lot of Tile Entities - over 7000. Have you been using carpenters / chisel and bits or something? You should avoid using Tile Entity style blocks for decorative purposes as they impose a processing cost for just existing and pretty much any block that is not a variant on a cube, stairs or tile is going to be a tile entity : Especially non cube blocks that can look like other blocks.

I don't know how to read the contention profiler, the network profiler doesn't indicate anything and the entity profiler doesn't indicate that you have a bunch of sheep in a pen causing AI pathing issues.

What do you mean, what command line?

Tile Entities ... don't really use carpenter/chisel ..... Only non-solid block type things I had are Draconic energy cube, vein ore miners (although they're comprised of solid blocks), fusion crafting, lots of Draconic wireless thingies around, and lots of RS cables. Barrels as well.

That's what really jumps out at me. Now if you want, I can zip up my world if you want to browse at it and see what you can find.

 

[GreenZombie]

What do you mean, what command line?

Tile Entities ... don't really use carpenter/chisel .....

You say you play solo on your server, and you have scheduled restarts etc.

Where exactly is this server hosted? I mean, I play on a server. Its a linux box that I have in my house, and I have installed McMyAdmin and put together bash scripts that install modpacks using @Reika's curse mudpack downloader and so on... There is a web interface, but because I made it myself I know how java is being used to run my server.

This is clearly not your setup. How much access do you have to your server and how? How did you get a specific modpack installed?

Surely the people who run your server can provide you with some kind of access to the command line they are using to run the forge server .jar file?
As a 3rd party server, how much memory have they given you/are you paying for - and are you sure its enough for your server?

 

[Brian Cherrick]

You say you play solo on your server, and you have scheduled restarts etc.

Where exactly is this server hosted? I mean, I play on a server. Its a linux box that I have in my house, and I have installed McMyAdmin and put together bash scripts that install modpacks using @Reika's curse mudpack downloader and so on... There is a web interface, but because I made it myself I know how java is being used to run my server.

This is clearly not your setup. How much access do you have to your server and how? How did you get a specific modpack installed?

Surely the people who run your server can provide you with some kind of access to the command line they are using to run the forge server .jar file?
As a 3rd party server, how much memory have they given you/are you paying for - and are you sure its enough for your server?

Responded to you via PM's

 

[Psychicash]

Well, the Server tick log just confirms that the server knows its lagging too
Id say this is the typical pattern of the garbage collector taking too long but frankly any kind of mod wasting cpu cycles is going to manifest like this.
Question: What is the command line being used to launch your server?

In terms of your TPS report there do not appear to be a large number of Entities. But there are a lot of Tile Entities - over 7000. Have you been using carpenters / chisel and bits or something? You should avoid using Tile Entity style blocks for decorative purposes as they impose a processing cost for just existing and pretty much any block that is not a variant on a cube, stairs or tile is going to be a tile entity : Especially non cube blocks that can look like other blocks.

I don't know how to read the contention profiler, the network profiler doesn't indicate anything and the entity profiler doesn't indicate that you have a bunch of sheep in a pen causing AI pathing issues.

So chisel and bits works like a chest... as a tile entity? I was wondering about that. But the lag I used to get from Carpenter's blocks isn't nearly as bad as with C&B... i'm really curious about this.

 

[Drbretto]

I am going to have to check out this warmroast thing... I'm sure I'm completely destroying my server.

But I'm afraid it's going to tell me that I can't have my things anymore

 

[GreenZombie]

So chisel and bits works like a chest... as a tile entity? I was wondering about that. But the lag I used to get from Carpenter's blocks isn't nearly as bad as with C&B... i'm really curious about this.

I don't know what dark magic C&B does, BUT: Minecraft has two rendering pipelines. One deals with blocks - first it processes all the solid blocks - because it knows that solid blocks with solid blocks around them cannot be seen so it can ignore them. It can also ignore blocks that only are exposed "on the other side" from the camera. So MC an stitch together some surfaces, assign them texture coordinates, and draw the entire solid block world in a single call. Then another pass does the transparent blocks.

Then - then MC needs to do with tile entities. Tile entities can be drawn any which way, can be arbitrarily transparent so Minecraft must always draw all of them. Each tile entity has its own render function, so MC can't batch them into a single draw call. I am sure there are some optimization opportunities here that MC does make use of, but the work involved in rendering TEs is orders of magnitude harder than rendering blocks, and normal minecraft is not particularly optimized to deal with them anyway as Notch could (rightly) assume that tile entities were much rarer than blocks - 10s per chunk at most.

Mods that introduce multiblocks and build them out of TEs? A small 10x10x10 structure adds 1000 TEs to the chunk its in. C&B performs quite well, so I think that internally its a small instance of a optimal renderer - rather than each TE rendering separately It might be rendering multiple on screen TEs in a single pass and short circuiting the other TEs paint calls. At least that's what I hope it is doing.

 

[Brian Cherrick]

I don't know what dark magic C&B does, BUT: Minecraft has two rendering pipelines. One deals with blocks - first it processes all the solid blocks - because it knows that solid blocks with solid blocks around them cannot be seen so it can ignore them. It can also ignore blocks that only are exposed "on the other side" from the camera. So MC an stitch together some surfaces, assign them texture coordinates, and draw the entire solid block world in a single call. Then another pass does the transparent blocks.

Then - then MC needs to do with tile entities. Tile entities can be drawn any which way, can be arbitrarily transparent so Minecraft must always draw all of them. Each tile entity has its own render function, so MC can't batch them into a single draw call. I am sure there are some optimization opportunities here that MC does make use of, but the work involved in rendering TEs is orders of magnitude harder than rendering blocks, and normal minecraft is not particularly optimized to deal with them anyway as Notch could (rightly) assume that tile entities were much rarer than blocks - 10s per chunk at most.

Mods that introduce multiblocks and build them out of TEs? A small 10x10x10 structure adds 1000 TEs to the chunk its in. C&B performs quite well, so I think that internally its a small instance of a optimal renderer - rather than each TE rendering separately It might be rendering multiple on screen TEs in a single pass and short circuiting the other TEs paint calls. At least that's what I hope it is doing.

My question is what is a tile entity exactly?

My base at the time what is relatively standard. Had a few big reactors a few turbines Some Farms outside. Draconic energy storage and a mob farm that wasn't active. Along with your standard machines