r/java • u/danielepolencic • 3d ago

Java JVM CPU and Memory Requests and Limits in Kubernetes

Running Java on Kubernetes? CPU and memory limits affect much more than scheduling.

A 4 GiB heap requires a container larger than 4 GiB. Fractional CPU limits can also change the processor count that HotSpot sees.

In our new article, you will learn:

  • Why JVM heap size and container memory are different boundaries
  • How CPU limits affect garbage collection, worker pools, and application performance
  • Which JVM and container metrics to collect when validating resource settings under load

The article also includes practical experiments and an interactive configuration calculator.

Read: https://learnkube.com/java-jvm-kubernetes-requests-limits

34 Upvotes

85% Upvoted

18 comments sorted by

24

u/cogman10 3d ago

I skimmed the article, and didn't really notice a mention of JVM off heap allocations. Those are a pretty big deal that can make right sizing the JVM tricky.

This, IMO, is a weakness in JVM auto-configuration. The JVM just does a poor job in selecting reasonable memory limits in a container environment. It's forced us to explicitly set the values because everything is just a little wrong.

MaxRamPercentage seems like it's a good thing, but it fails to account for the fact that not everything grows at the same rate. 10% headroom might not be enough when talking about a container with a 256MB max, but it may be extreme overkill if you are talking about a container with a 64GB max.

It'd be really nice if the JVM could automatically and dynamically adjust it's XMX setting to max out memory usage. What I'd like would be able to say something like "JVM, Start with an XMX that's 75% the container's memory, watch and see how your native memory use grows. Slowly scale up that XMX until the system's free memory is around 100MB"

12

u/gaelfr38 3d ago

This. Off heap is the real trouble.

2

u/A_random_zy 2d ago

I am wondering if any of it is already in works. I don't follow the dev of jvm that closely but I'd be interested of someone knows something.

1

u/aoeudhtns 1d ago

https://openjdk.org/jeps/8359211

https://openjdk.org/jeps/8377305

The G1GC JEP mentions accounting for native/off-heap memory; the ZGC JEP doesn't seem to mention it but I'd be surprised if it doesn't.

Neither of these are Candidate status yet though, so let's hope.

We also have this: https://openjdk.org/jeps/8354416

Which will help us debug these issues more easily.

2

u/A_random_zy 1d ago

Wow thanks for sharing this 😃

1

u/FuzzyZocks 3d ago

Best resource to understand off heap sizing? We did work to understand this in Cassandra but did not get to finish with other work ongoing. Interested to pick this back on my own.

7

u/cogman10 3d ago edited 3d ago

Oh that's a hard one. The problem is it comes in all sorts of flavors.

For example, metaspace is offheap. So if your app has a lot of classes in it or a lot of class generation, you can blow up your off heap memory. Native threads allocate some offheap memory for their stacks. Most networking will allocate small native buffers which are off heap. Most GCs in the JVM have some offheap memory. G1GC used to allocate around 20% of XMX offheap (it's gotten a lot better). And of course JNI often has routes to off heap.

All of these things or none of them can be a problem depending on the application which is why it's so tricky.

I've mostly built what I know about offheap from experiencing issues with it in our apps. The most helpful tool for that is jemalloc which has a debug mode that can create allocation graphs.

1

u/Life_Sink9598 3d ago

Do you typically see issues with off-heap memory from JVM usage or from libraries utilizing native (= off-heap) memory?

The standard tool for understanding JVM memory usage is Native Memory Tracking in summary mode, with the caveat that using it also increases off-heap memory usage a little bit.

1

u/cogman10 2d ago

It can be either. I've seen the JVM specific problems mostly around G1GC's overhead (which has gotten better). The native libraries where I've seen it most are generally compression libraries (including the JDK's Deflate).

But for a few apps that have had a large number of platform (non-virtual) threads being used, I've seen those also cause problems.

It is definitely application specific.

1

u/ElderCantPvm 2d ago

Interesting, this reminds me of how the ZFS ARC cache self adjusts its size

2

u/TallGreenhouseGuy 3d ago

Buildpacks take this into account in the memory formula, which could be a good starting point:

https://paketo.io/docs/reference/java-reference/#memory-calculator

2

u/lazystone 3d ago

And if you use Spring Boot, then it already uses Buildpacks

1

u/burl-21 2d ago

And it should soon add low memory profile too, see https://github.com/paketo-buildpacks/jvm-vendors/pull/31

2

u/Turbots 2d ago

Use buildpacks to build your Java containers, it has a really good memory calculator in there to compute the JVM heap and offheap sizes really well, and you can tweak them very well too.

Or just the memory calculator in your own docker file:

https://github.com/cloudfoundry/java-buildpack-memory-calculator

1

u/k_brn 3d ago

But do you really need Kubernetes if you can run on two 1 GB VMs with a reverse proxy and get better availability with way less management overhead?

3

u/thisisjustascreename 3d ago

The answer to "Do you really need Kubernetes?" is almost always no. But lots of people know how to run it and lots of orgs are already using it.

7

u/Cilph 3d ago

If you just have the one app, you likely dont need it. If you need to deploy multiple apps over multiple solated environments for multiple tenants and set up secure firewalls between each and support blue-green deployment at the same time, then it quickly becomes reasonable to use k8s.