JEP 491 - Synchronize Virtual Threads without Pinning.

Giriş

Açıklaması şöyle

Improve the scalability of Java code that uses synchronized methods and statements by arranging for virtual threads that block in such constructs to release their underlying platform threads for use by other virtual threads. This will eliminate nearly all cases of virtual threads being pinned to platform threads, which severely restricts the number of virtual threads available to handle an application's workload.

synchronized block kullanıyorsak Virtual Thread pinned oluyor. Bu JEP ile artık olmuyor

Virtual Threads - Thread Pinning

Thread Pinning 

Virtual Threads için JDK'daki bir çok kod baştan yazılmış. Böylece thread unloading işlemi yapılabiliyor.  Ancak bir kaç işlemde istisna var. Buna Thread Pinning deniliyor

1. Object.wait()

2. synchronized blok içinde

3. native metod çalıştırırken

4. UDP Sockets

5. Files & DNS

Açıklaması şöyle

Pinning describes the condition where a virtual thread, which maps the platform thread, is stuck to the carrier thread. It means that the virtual thread can’t be unmounted from the carrier threads because the state of it can’t be stored in the heap memory. The pinned thread prevents others from utilizing the same platform thread.

1. synchronized Thread Pinning

Açıklaması şöyle

Only one thread can enter it at a time. The other threads attempting to enter will be blocked until the current (running) thread exits. They would need an uninterrupted access to shared resources to prevent race conditions as well as an accurate state of the data.

Örnek

Şu kod carrier thread'i bloke eder. 

var e = Executors.newVirtualThreadPerTaskExecutor();
for (int i = 0; i < 1000; i++) {
    e.submit(() -> { new Test().test(); });
}
e.shutdown();
e.awaitTermination(1, TimeUnit.DAYS);

class Test {
  synchronized void test() {
    sleep(4000);
  }
}

synchronized yerine  ReentrantLock kullanılır. Şöyle yaparız

var e = Executors.newVirtualThreadPerTaskExecutor();
for (int i = 0; i < 1000; i++) {
    e.submit(() -> { new Test().test(); });
}
e.shutdown();
e.awaitTermination(1, TimeUnit.DAYS);

class Test {
  private ReentrantLock lock = new ReentrantLock();
  void test() {
    lock.tryLock();
    try {
      sleep(4000);
    } finally {
      lock.unlock();
    }
  }
}

MySQL-J JDBC sürücüsündeki bir açıklama şöyle

Synchronized blocks in the Connector/J code were replaced with ReentrantLocks. This allows carrier threads to unmount virtual threads when they are waiting on IO operations, making Connector/J virtual-thread friendly. Thanks to Bart De Neuter for contributing to this patch. ”

2. native method Thread Pinning

Açıklaması şöyle

This method lets you use code from other languages into your JAVA project, like C or C++. Native methods are invoked within the JAVA virtual machine but executed outside its control.

3. Sockets Thread Pinning

TCP socketlerini okurken Virtual Threads bloke olmuyor, ama UDP socketlerinde oluyor. Açıklaması şöyle

However, not all Java constructs have been retrofitted that way. Such operations include synchronized methods and code blocks. Using them causes the virtual thread to become pinned to the carrier thread. When a thread is pinned, blocking operations will block the underlying carrier thread-precisely as it would happen in pre-Loom times.

Eğer carrier therad'in bloke olma durumu varsa, virtual thread sayısı artırılır. En fazla kaç thread olabileceği  jdk.virtualThreadScheduler.maxPoolSize ile atanabilir

pinned thread Tracing

Açıklaması şöyle

Use the following JVM parameters as options to trace the pinned thread

-Djdk.tracePinnedThreads=full Prints a complete stack trace when a thread jams while pinned, highlighting native frames and frames holding monitors.

-Djdk.tracePinnedThreads=short Limits the output to just the problematic frames.

Virtual Threads - Java 19 İle Geliyor

Giriş

Buraya nasıl geldik. Açıklaması şöyle. Yani önce Green Threads daha sonra Fiber denendi.

In the very early versions of Java, when the multithreading API was designed, Sun Microsystems were faced with a dilemma: should we use User Mode threads or map the Java thread one to one with the OS thread. All benchmarks back then showed that User Mode threads were severely inferior, increasing memory consumption without giving much in return. However this was benchmarked 20 years ago and things were quite different back then. We didn’t have such high load requirements and the Java language was still not very mature. Now the situation is different and we had few attempts of introducing user mode threads into the language. For example Fibers.

Unfortunately due to the fact that they were implemented as a separate class it was very hard to migrate your whole codebase to it and eventually they disappeared and never got merged into the language.

Loom Nedir

Project Loom'um amacı Virtual Threads. Açıklaması şöyle

Project Loom introduces lightweight user-mode threads called Virtual Threads as instances of java.lang.Thread.

Loom tek proje değil. Açıklaması şöyle

There are several Java projects that have a very specific task to achieve. Those are for example Valhala, Panama, Amber and of course Loom. Loom’s goal is to overhaul the concurrency model of the language. They aim to bring virtual threads, structured concurrency and few other smaller things (for now).

Loom da bir çok JEP teklifinden oluşuyor. Açıklaması şöyle

Per the JDK release process, features in Project Loom were broken down into several JEPs (JDK Enhancement Proposals) and made available in different JDK releases.

Loom Tarihçesi

Açıklaması şöyle. Java 19 ile preview olarak geldi. Java 21 ile dile dahil oldu

Project Loom (2017):

The Loom project was started to provide a functionality of virtual threads in java.

Java 19 (2022):

Virtual threads were proposed as a preview feature by JEP 425

Java 21 (2023):

Virtual threads are finalised and released.

Virtual Thread Nasıl Çalışır

Açıklaması şöyle

We all know blocking a thread is evil and negatively affects your application’s performance. Well, not in this case. When a virtual thread blocks on I/O or some blocking operation in the JDK, such as BlockingQueue.take(), it automatically unmounts from the platform thread.

The JDK’s scheduler can mount and run other virtual threads on this now-free platform thread. When the blocking operation is ready to complete, it submits the virtual thread back to the scheduler, which will mount the virtual thread on an available platform thread to resume execution.

This platform thread doesn’t have to be the same from which the virtual thread was unmounted. As a result, we can now build highly concurrent applications with high throughput without consuming an increased number of threads (by default, Executors for virtual threads will use as many platform threads as the number of processors available).

Virtual Thread İle Çalışırken şunlara dikkat etmek lazım

In order to switch to Virtual threads we don’t have to learn new things, we just have to unlearn a few.

• Never pool Virtual Threads, they are cheap and it makes no sense
• Stop using thread locals. They will work, but if you spawn millions of threads you will have memory problems. According to Ron Pressler: “Thread locals should have never been exposed to the end user and should have stayed as an internal implementation detail”.

Bu durumda artık klasik Thread Pool'a gerek kalmıyor. Açıklaması şöyle

Thread-per-Request Model with Virtual Threads?

One should not pool virtual threads as they are not expensive resources. One can create millions of them to handle network operations. They should be spun up on-demand and killed when their task is through, and are thus suited for short lived tasks.

These properties of virtual threads give near-optimal CPU utilization and a significant increase in performance in terms of throughput and not speed. Now that we have all the supporting data, it is safe to say that a virtual thread per request model in a Java server application is safe and more efficient than pooling platform threads.

Şeklen şöyle

Yöntem 1 - Thread. startVirtualThread metodu

Thread.startVirtualThread metodu yazısına taşıdım

Yöntem 2- Thread.ofVirtual() metodu

Thread.ofVirtual metodu yazısına taşıdım

Yöntem 3 - Executors.newVirtualThreadPerTaskExecutor() metodu

Executors.newVirtualThreadPerTaskExecutor metodu yazısına taşıdım

ForkJoinPool

Virtual Threads ForkJoinPool ile çalıştırılır. Yukarıdaki çıktıdan da görülebilir. Normalde ForkJoinPool işlemci sayısı kadar thread ile başlar. Bu sayı jdk.virtualThreadScheduler.parallelism ile değiştirilebilir. Açıklaması şöyle

Moreover, you can control the initial and maximum size of the carrier thread pool using the jdk.virtualThreadScheduler.parallelism, jdk.virtualThreadScheduler.maxPoolSize and jdk.virtualThreadScheduler.minRunnable configuration options. These are directly translated to constructor arguments of the ForkJoinPool.

Unloading İşlemi - Blocking Çağrılar İçin Gerekir

Virtual thread I/O  veya bir blocking işlem yapacaksa unload edilir (örneğin BlockingQueue.take())

İşlem hazır olunca virtual thread tekrar load edilir.  Açıklaması şöyle. Yani Unload işlemi için JDK'daki bir çok kod tekrar yazılmış.

To implement virtual threads, as mentioned above, a large part of Project Loom’s contribution is retrofitting existing blocking operations so that they are virtual-thread-aware. That way, when they are invoked, they free up the carrier thread to make it possible for other virtual threads to resume.

For example, whenever code blocks on a semaphore, lock, or another Java concurrency primitive, this won’t block the underlying carrier thread but only signal to the runtime that it should capture the continuation of the current virtual thread, put it in a waiting queue and resume once the condition on which the blocking happened is resolved.

Virtual Threads - Thread Pinning

Virtual Threads için JDK'daki bir çok kod baştan yazılmış. Böylece thread unloading işlemi 

yapılabiliyor.  Ancak bir kaç işlemde istisna var. Buna Thread Pinning deniliyor

sleep metodu

Örnek

Açıklaması şöyle

Similarly, Thread.sleep now only blocks the virtual thread, not the carrier thread.

sleep() metodu artık şöyle. Burada Thread'in VirtualThread olup olmadığı kontrol ediliyor.

public static void sleep(long millis) throws InterruptedException {
  if (millis < 0) {
    throw new IllegalArgumentException("timeout value is negative");
  }

  if (currentThread() instanceof VirtualThread vthread) {
    long nanos = MILLISECONDS.toNanos(millis);
    vthread.sleepNanos(nanos);
    return;
  }

  if (ThreadSleepEvent.isTurnedOn()) {
    ThreadSleepEvent event = new ThreadSleepEvent();
    try {
      event.time = MILLISECONDS.toNanos(millis);
      event.begin();
      sleep0(millis);
    } finally {
      event.commit();
    }
  } else {
    sleep0(millis);
  }
}

VirtualThread.sleepNanos() en sonunda şu metodu çağırıyor. Continuation.yield() kullanılıyor

@ChangesCurrentThread
private boolean yieldContinuation() {
  boolean notifyJvmti = notifyJvmtiEvents;
  // unmount
  if (notifyJvmti) notifyJvmtiUnmountBegin(false);
  unmount();
  try {
    return Continuation.yield(VTHREAD_SCOPE);
  } finally {
    // re-mount
    mount();
    if (notifyJvmti) notifyJvmtiMountEnd(false);
  }
}

Açıklaması şöyle. Yani Virtual Thread unload edilirse, her şeyi heap'te bir yerde saklanıyor

That is to say, Continuation.yield will transfer the stack of the current virtual thread from the stack of the platform thread to the Java heap memory, and then copy the stacks of other ready virtual threads from the Java heap to the stack of the current platform thread to continue execution. Performing blocking operations such as IO or BlockingQueue.take() will cause virtual thread switching just like sleep. The switching of virtual threads is also a relatively time-consuming operation, but compared with the context switching of platform threads, it is still much lighter.

Yani şöyle yapabiliriz. 1000 tane thread yaratıp hepsine sleep() yapsak bile tüm işlem yine de 4 saniye sürecektir.

var e = Executors.newVirtualThreadPerTaskExecutor();
for (int i = 0; i < 1000; i++) {
  e.submit(() -> { sleep(4000); });
}
e.shutdown();
e.awaitTermination(1, TimeUnit.DAYS);

Files & DNS

Açıklaması şöyle. Yani bu işlemlerde Carrier Thread sayısı otomatik artırılıyor

Yes, there are more APIs that cause thread pinning and blocking of the carrier thread, namely all file operations (such as reading from a FileInputStream, writing to a file, listing directories, etc.), as well as resolving domain names to IP addresses using InetSocketAddress.

That might look worrying, but Loom does take some remedial steps. If you take a look at the source code of FileInputStream, InetSocketAddress or DatagramSocket, you'll notice usages of the jdk.internal.misc.Blocker class. Invocations to its begin()/ end() methods surround any carrier-thread-blocking calls.

...

In other words, the carrier thread pool might be expanded when a blocking operation is encountered to compensate for the thread-pinning that occurs. A new carrier thread might be started, which will be able to run virtual threads.