Virtual Threads - Java 19 İle Geliyor

Giriş

Buraya nasıl geldik. Açıklaması şöyle. Yani önce Green Threads daha sonra Fiber denendi.

In the very early versions of Java, when the multithreading API was designed, Sun Microsystems were faced with a dilemma: should we use User Mode threads or map the Java thread one to one with the OS thread. All benchmarks back then showed that User Mode threads were severely inferior, increasing memory consumption without giving much in return. However this was benchmarked 20 years ago and things were quite different back then. We didn’t have such high load requirements and the Java language was still not very mature. Now the situation is different and we had few attempts of introducing user mode threads into the language. For example Fibers.

Unfortunately due to the fact that they were implemented as a separate class it was very hard to migrate your whole codebase to it and eventually they disappeared and never got merged into the language.

Loom Nedir

Project Loom'um amacı Virtual Threads. Açıklaması şöyle

Project Loom introduces lightweight user-mode threads called Virtual Threads as instances of java.lang.Thread.

Loom tek proje değil. Açıklaması şöyle

There are several Java projects that have a very specific task to achieve. Those are for example Valhala, Panama, Amber and of course Loom. Loom’s goal is to overhaul the concurrency model of the language. They aim to bring virtual threads, structured concurrency and few other smaller things (for now).

Loom da bir çok JEP teklifinden oluşuyor. Açıklaması şöyle

Per the JDK release process, features in Project Loom were broken down into several JEPs (JDK Enhancement Proposals) and made available in different JDK releases.

Loom Tarihçesi

Açıklaması şöyle. Java 19 ile preview olarak geldi. Java 21 ile dile dahil oldu

Project Loom (2017):

The Loom project was started to provide a functionality of virtual threads in java.

Java 19 (2022):

Virtual threads were proposed as a preview feature by JEP 425

Java 21 (2023):

Virtual threads are finalised and released.

Virtual Thread Nasıl Çalışır

Açıklaması şöyle

We all know blocking a thread is evil and negatively affects your application’s performance. Well, not in this case. When a virtual thread blocks on I/O or some blocking operation in the JDK, such as BlockingQueue.take(), it automatically unmounts from the platform thread.

The JDK’s scheduler can mount and run other virtual threads on this now-free platform thread. When the blocking operation is ready to complete, it submits the virtual thread back to the scheduler, which will mount the virtual thread on an available platform thread to resume execution.

This platform thread doesn’t have to be the same from which the virtual thread was unmounted. As a result, we can now build highly concurrent applications with high throughput without consuming an increased number of threads (by default, Executors for virtual threads will use as many platform threads as the number of processors available).

Virtual Thread İle Çalışırken şunlara dikkat etmek lazım

In order to switch to Virtual threads we don’t have to learn new things, we just have to unlearn a few.

• Never pool Virtual Threads, they are cheap and it makes no sense
• Stop using thread locals. They will work, but if you spawn millions of threads you will have memory problems. According to Ron Pressler: “Thread locals should have never been exposed to the end user and should have stayed as an internal implementation detail”.

Bu durumda artık klasik Thread Pool'a gerek kalmıyor. Açıklaması şöyle

Thread-per-Request Model with Virtual Threads?

One should not pool virtual threads as they are not expensive resources. One can create millions of them to handle network operations. They should be spun up on-demand and killed when their task is through, and are thus suited for short lived tasks.

These properties of virtual threads give near-optimal CPU utilization and a significant increase in performance in terms of throughput and not speed. Now that we have all the supporting data, it is safe to say that a virtual thread per request model in a Java server application is safe and more efficient than pooling platform threads.

Şeklen şöyle

Yöntem 1 - Thread. startVirtualThread metodu

Thread.startVirtualThread metodu yazısına taşıdım

Yöntem 2- Thread.ofVirtual() metodu

Thread.ofVirtual metodu yazısına taşıdım

Yöntem 3 - Executors.newVirtualThreadPerTaskExecutor() metodu

Executors.newVirtualThreadPerTaskExecutor metodu yazısına taşıdım

ForkJoinPool

Virtual Threads ForkJoinPool ile çalıştırılır. Yukarıdaki çıktıdan da görülebilir. Normalde ForkJoinPool işlemci sayısı kadar thread ile başlar. Bu sayı jdk.virtualThreadScheduler.parallelism ile değiştirilebilir. Açıklaması şöyle

Moreover, you can control the initial and maximum size of the carrier thread pool using the jdk.virtualThreadScheduler.parallelism, jdk.virtualThreadScheduler.maxPoolSize and jdk.virtualThreadScheduler.minRunnable configuration options. These are directly translated to constructor arguments of the ForkJoinPool.

Unloading İşlemi - Blocking Çağrılar İçin Gerekir

Virtual thread I/O  veya bir blocking işlem yapacaksa unload edilir (örneğin BlockingQueue.take())

İşlem hazır olunca virtual thread tekrar load edilir.  Açıklaması şöyle. Yani Unload işlemi için JDK'daki bir çok kod tekrar yazılmış.

To implement virtual threads, as mentioned above, a large part of Project Loom’s contribution is retrofitting existing blocking operations so that they are virtual-thread-aware. That way, when they are invoked, they free up the carrier thread to make it possible for other virtual threads to resume.

For example, whenever code blocks on a semaphore, lock, or another Java concurrency primitive, this won’t block the underlying carrier thread but only signal to the runtime that it should capture the continuation of the current virtual thread, put it in a waiting queue and resume once the condition on which the blocking happened is resolved.

Virtual Threads - Thread Pinning

Virtual Threads için JDK'daki bir çok kod baştan yazılmış. Böylece thread unloading işlemi 

yapılabiliyor.  Ancak bir kaç işlemde istisna var. Buna Thread Pinning deniliyor

sleep metodu

Örnek

Açıklaması şöyle

Similarly, Thread.sleep now only blocks the virtual thread, not the carrier thread.

sleep() metodu artık şöyle. Burada Thread'in VirtualThread olup olmadığı kontrol ediliyor.

public static void sleep(long millis) throws InterruptedException {
  if (millis < 0) {
    throw new IllegalArgumentException("timeout value is negative");
  }

  if (currentThread() instanceof VirtualThread vthread) {
    long nanos = MILLISECONDS.toNanos(millis);
    vthread.sleepNanos(nanos);
    return;
  }

  if (ThreadSleepEvent.isTurnedOn()) {
    ThreadSleepEvent event = new ThreadSleepEvent();
    try {
      event.time = MILLISECONDS.toNanos(millis);
      event.begin();
      sleep0(millis);
    } finally {
      event.commit();
    }
  } else {
    sleep0(millis);
  }
}

VirtualThread.sleepNanos() en sonunda şu metodu çağırıyor. Continuation.yield() kullanılıyor

@ChangesCurrentThread
private boolean yieldContinuation() {
  boolean notifyJvmti = notifyJvmtiEvents;
  // unmount
  if (notifyJvmti) notifyJvmtiUnmountBegin(false);
  unmount();
  try {
    return Continuation.yield(VTHREAD_SCOPE);
  } finally {
    // re-mount
    mount();
    if (notifyJvmti) notifyJvmtiMountEnd(false);
  }
}

Açıklaması şöyle. Yani Virtual Thread unload edilirse, her şeyi heap'te bir yerde saklanıyor

That is to say, Continuation.yield will transfer the stack of the current virtual thread from the stack of the platform thread to the Java heap memory, and then copy the stacks of other ready virtual threads from the Java heap to the stack of the current platform thread to continue execution. Performing blocking operations such as IO or BlockingQueue.take() will cause virtual thread switching just like sleep. The switching of virtual threads is also a relatively time-consuming operation, but compared with the context switching of platform threads, it is still much lighter.

Yani şöyle yapabiliriz. 1000 tane thread yaratıp hepsine sleep() yapsak bile tüm işlem yine de 4 saniye sürecektir.

var e = Executors.newVirtualThreadPerTaskExecutor();
for (int i = 0; i < 1000; i++) {
  e.submit(() -> { sleep(4000); });
}
e.shutdown();
e.awaitTermination(1, TimeUnit.DAYS);

Files & DNS

Açıklaması şöyle. Yani bu işlemlerde Carrier Thread sayısı otomatik artırılıyor

Yes, there are more APIs that cause thread pinning and blocking of the carrier thread, namely all file operations (such as reading from a FileInputStream, writing to a file, listing directories, etc.), as well as resolving domain names to IP addresses using InetSocketAddress.

That might look worrying, but Loom does take some remedial steps. If you take a look at the source code of FileInputStream, InetSocketAddress or DatagramSocket, you'll notice usages of the jdk.internal.misc.Blocker class. Invocations to its begin()/ end() methods surround any carrier-thread-blocking calls.

...

In other words, the carrier thread pool might be expanded when a blocking operation is encountered to compensate for the thread-pinning that occurs. A new carrier thread might be started, which will be able to run virtual threads.