Comments for https://horstmann.com/unblog/2022-04-15/index.html
Comments for: https://horstmann.com/unblog/2022-04-15/index.html
- CClever Code @clever_code
Thanks for article.
Tomcat 10.1.0-M16 (beta) Released
The notable changes in this release are:
Refactor synchronization blocks locking on SocketWrapper to use ReentrantLock to support users wishing to experiment with project Loom.
Almost same code yours in SocketProcessorBase.java
- KIn reply tosystem⬆:Kien Vo @KienVo
Hi Horstmann,
I tested the virtual threads in my local machine comparing with "old" platform threads and the result surprised me that virtual threads are much worse (multiple times) than platform threads.
Below is my test function (the result is the same for both local or remote API). Could you give any advice as I'm quite confused now?static void handleUrlVirtualThreads() { int nThreads = 10_000; // try (var executor = Executors.newFixedThreadPool(100)) { try (var executor = Executors.newVirtualThreadPerTaskExecutor()) { IntStream.range(0, nThreads).forEach(i -> executor.submit(() -> { try { var url = new URL("http://localhost:8090/hello"); //http://asia-southeast1.google.api.expert/ip try (var in = url.openStream()) { return new String(in.readAllBytes(), StandardCharsets.UTF_8); } } catch (IOException e) { throw new RuntimeException(e); } })); } }
- TTheodore Ravindranath @TheodoreRavi
Did you shutdown the Executor and awaitTermination? Otherwise, it just creates the tasks and leaves them running and moves out of that method handleUrlVirtualThreads.
Also, if the response of the 3rd party site is very fast, then you may not see much benefit as 100 threads can process 10_000 calls easily.More importantly, I think the benefit of virtual threads comes in when there are assorted tasks with different work profile.
For example, imagine 100 threads are blocked with calls to a very slow website. Now, imagine you get 50 more tasks which just need to access files on harddisk.
In the case of executor, only when the threads start to become unblocked, executor can start working on the other 50 "easy" tasks which just need local file access.
However, with virtual threads, all 150 operations will be started simultaneously, and the 50 fast ones will be completed almost immediately.- CCay Horstmann @cayhorstmann
Actually, the executor is shut down in the try-with-resources statement. It's easy to get blacklisted when sending 10,000 requests in quick succession, so I tried it with 1,000 with the http://asia-southeast1.google.api.expert/ip service. Virtual threads: 3 seconds, platform threads: 7 seconds.
- T
- CCay Horstmann @cayhorstmann
Try it again--I just refreshed it.
- OIn reply tosystem⬆:Oliver @oliver
Looks like this change is also included in Tomcat 9.1.74 now: "Refactor synchronization blocks locking on SocketWrapper to use ReentrantLock to support users wishing to experiment with project Loom."
- TIn reply tosystem⬆:Theodore Ravindranath @TheodoreRavi
Very useful article.. thanks so much.
As I mentioned in my other comment, I feel being non-blocking really shines when there are lots of long-waiting tasks mixed with lots of small tasks.
The blocking API / thread pools will wait for some of the long-waiting tasks to complete before taking up the small tasks. Whereas non-blocking will simply go through with the smaller tasks.
I believe that is the key benefit of Virtual threads.- CCay Horstmann @cayhorstmann
The sweet spot of virtual threads is lots of tasks that mostly block. The key advantage of virtual threads is to program in a natural Java style, with branches, loops, method calls, and exception handling, instead of a reactive style with callbacks. Don't use virtual threads for non-blocking tasks.
- CCay Horstmann @cayhorstmann
Absolutely. That's why the creators of virtual threads stress that they increase throughput. It's not the only way to increase throughput. A reactive architecture will do the same. But at the cost of an unnatural programming model.
- In reply tocayhorstmann⬆:TTheodore Ravindranath @TheodoreRavi
"The sweet spot of virtual threads is lots of tasks that mostly block" - while I agree with this, isn't it also true that they let other tasks through while native threads would really be blocked.
Imagine lot of web controller requests, doing assorted operations, if 200 native threads were blocked on 50 DB connections, all 200 threads would be stuck, whereas virtual threads would allow other customer requests which don't involve the DB to go through. That's why virtual threads really shine when there is an assorted work load. The point is, there are more requests which are not waiting for the same 50 DB connections, which will go through in case of Virtual threads.
And yes, we don't have to do non-blocking and reactive to achieve this - in case of virtual threads, we can work with the traditional/convenient iterative model.