r/java • u/jaccomoc • 22h ago
Vert.x Event-Loop vs Virtual Threads: Jactl Suspend/Resume Benchmark
https://jactl.io/blog/2026/09/02/jactl-virtual-threads-benchmarkThe article benchmarks Jactl script execution with differing number of blocking operations comparing throughput with Jactl built-in suspend/resume on Vert.x Event-Loops and throughput with Jactl on Virtual Threads. The benchmarks are run on both Java 21 and Java 25 with some surprising (to me at least) results.
8
u/yawkat 20h ago
Three issues with this:
- AFAICS this is a closed model load, which is a batch processing workload. In a typical vertx application you have an open model load instead. See this article: https://www.scylladb.com/2021/04/22/on-coordinated-omission/
- Because this is JMH, it does not mirror "batchy" behavior of network interfaces, which is the typical use case for vertx.
- When tasks block, they are typically waiting for IO work, not
sleeping. This has implications for scheduling.
2
u/jaccomoc 20h ago
Thanks for the feedback. I agree that the test is not ideal but I am not sure about you saying that the load is a closed model load. The test queues (for Vert.x) all requests or creates virtual threads for all requests and then waits for them all to finish. Isn't that an open model load?
3
u/pron98 59m ago edited 36m ago
The improvement in JDK 25 wasn't in virtual threads, it was in Thread.sleep ;) (although between 22 and 25 there were also improvements to IO on virtual threads). Timed and untimed parks use different mechanisms, and we can't draw conclusions from one to the other; it was the timed park specifically that wasn't very efficient in JDK 21, but it's also used mostly in benchmarks and tests. Different kinds of IO also go through different mechanisms, and we can't extrapolate from one to another.
This demonstrates the danger in extrapolating and drawing conclusions from benchmarks: they measure themselves, and to extrapolate anything you need to know exactly which variables matter and when. Between concurrent algorithms that behave completely differently under low and high contention, code with slow and fast paths, optimising compilers that depend on the call graph, branch prediction, a cache hierarchy, and memory management that depends on the global behaviour of the program and on how long it's running (malloc/free is actually much more sensitive to this than Java's moving GCs [1]), it's virtually impossible to draw any conclusions from any benchmark, in any language, unless you know exactly how the mechanism is implemented, how the compiler is implemented, and how memory management is implemented.
Also, benchmark code often behaves so differently from real code, that the code takes different paths, and some improvements matter only to real programs and not to most benchmarks, and some matter almost exclusively to benchmarks. We usually try to focus on real programs, but sometimes we say, you know what, let's make the benchmarks look better, too...
[1]: I've seen benchmarks where C++ outperforms Java only because there is nothing else in the program besides the benchmark code, which happens to only hit the malloc/free fast path and never trigger the slow path; if the program also had other code, the results would have been reversed.
1
u/jaccomoc 23m ago edited 15m ago
That's good to know, thanks. To be honest, I did have doubts about using sleep() as a proxy for other blocking operations but it was easier to do that than to include a real web service call or equivalent. And, running a web server would mean that measuring max throughput on a single machine would be impacted by the performance of the web server as well (unless it was on a separate server in which case the test would no longer be self-contained).
I have updated the article with a Postscript section with this additional information.
2
u/New-Departure-5969 7h ago
benchmarks are fun but in prod the event loop usually dies from accidental blocking (jdbc, sync stuff), not from the loop itself. we had a vert.x api around 12k tps / ~100ms p99.99 and the real win was ditching blocking jdbc for the vert.x pool. virtual threads help, but saturated pools still wreck p99.
2
u/Known-Volume1509 3h ago edited 3h ago
It doesn't say what Vert.x version is this? Vert.x 5 supports virtual thread verticles. Curious if you run your tests against that, even though it's no longer event-loop.
Also, isn't this horribly costly?
For the Jactl Vert.x tests, the
sleep()call will create aContinuationby throwing an exception and capturing the execution state so that it can be resumed once the sleep completes. When the script suspends, the script returns and the event-loop thread is then able to process the next event in its queue.
2
u/jaccomoc 3h ago
The article lists the versions of everything. It shows Vert.x as version 4.5.33. I didn't benchmark against 5.x because I am still retaining Java 8 compatibility for the moment and Vert.x 5.x needs Java 11 as a minimum.
Throwing an exception is not expensive if you don't fill in the stack trace (and the benchmarks support this). See Jactl Continuations and Virtual Threads in Java 8 for a description of how all this works.
21
u/s0ftware-dev 21h ago
TLDR virtual threads are better.