Jump to content

Virtual thread

Add links
From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Haraldrudell (talk | contribs) at 18:00, 5 April 2022 (→Definition). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.
  • Comment: You may edit the existing article to add any material that is not already present. If this is a request to rename, please discuss via a Move Request, Robert McClenon (talk) 06:23, 5 April 2022 (UTC)

In computer programming, virtual threads are threads that are scheduled by a runtime library instead of natively by the underlying operating system (OS). Virtual threads allows for tens of millions of preemptive tasks and events without swapping on a 2021 consumer-grade computer.[1], compared to low thousands of operating system threads[2]. Preemptive execution[3] is important to performance since constructs that are not preemptive, such as coroutines or the largely single-threaded Node.js, introduce delays in responding to asynchronous events such as every incoming request in a server application[4]. Java servers have featured extensive and memory consuming software constructs allowing dozens of pooled operating system threads to preemptively execute thousands of requests per second without the use of virtual threads. Key to performance here is to reduce the initial latency in thread processing and minimize the time operating system threads are blocked[5]

Virtual threads increase possible concurrency by many orders of magnitudes while the actual parallelism achieved is limited by available execution units and pipelining offered by present processors and processor cores. In 2021, a consumer grade computers typically offer a parallelism of tens of concurrent execution units[6]. For increased performance through parallelism, the language runtime need to use all present hardware[7], not be single-threaded or feature global synchronization such as global interpreter lock

The many magnitudes of increase in possible preemptive items offered by virtual threads is achieved by the language runtime managing resizable thread stacks[8]. Those stacks are smaller in size than those of operating system threads. The maximum number of threads possible without swapping is proportional to the amount of main memory[9]

In order to support virtual threads efficiently, the language runtime has to be largely rewritten to prevent blocking calls from holding up an operating system thread assigned to execute a virtual thread[10] and to manage thread stacks[11]. An example of a retrofit of virtual threads is Java Loom[12]. An example of a new language designed for virtual threads is Go[13]

Definition

The term Virtual threads has been used ambiguously over the years. For example Intel[14] in 2007 spoke about their hyper-threading as virtual threads. Virtual threads were commercialized with Google’s Chrome browser in 2008[15] where virtual threads may hop physical threads. Virtual threads are truly virtual, created in user-space software

  1. Virtual Threads are preemptive
    1. This is important for response performance, that the virtual thread can react to events without programmer intervention or before concluding a current task
  2. Virtual Threads can hop over all processors and cores
    1. This allows for using all available hardware, a 10x increase on today’s computers
    2. In the go1.18 implementation, there are virtual thread queues per execution unit. There are additional virtual threads not allocated to an execution unit and an execution unit can steal virtual threads from another execution unit[16]
  3. Unlike co-routines, there is no yield or other intervention by the programmer
    1. Unlike coroutines, if a virtual thread is in an infinite loop, it does not block the program. Execution continues at a higher cpu load, even if there are more looping threads than available execution units
  4. Today’s implementations of Virtual Threads have smaller, managed stacks
    1. This allows for many magnitudes more threads than from using OS threads
  5. Because allocating a virtual thread is akin to allocate memory structures, they can be allocated very quickly, like 600,000 per second. This is not possible for OS threads and would crash the host
    1. The quicker ramp-up lessens the need for thread-pools of pre-launched threads to cater for sudden increases in traffic
  6. Like OS threads, virtual threads share memory map and can therefore freely share and access memory objects subject to synchronization. Data can be shared as zero-copy references

Important distinctions is concurrency, in which a single resource is shared in small time increments so that it appears always available and parallelism, in which instructions are executing truly at the same time leading to a magnitude higher performance, complex issues around synchronization, and a higher level of software engineering

Implementations

Java Project Loom

Project Loom: Virtual threads is a lightweight user-mode scheduled alternative to standard OS managed threads. Virtual threads are mapped to OS threads in a many-to-many relationship

See also

References

  1. ^ Rudell, Harald (2022-03-19). "massivevirtualparallelism".
  2. ^ baeldung (2022-01-02). "Maximum Number of Threads Per Process in Linux | Baeldung on Linux". www.baeldung.com. Retrieved 2022-03-30.
  3. ^ "Go 1.14 Release Notes - The Go Programming Language". go.dev. Retrieved 2022-03-30.
  4. ^ Node.js. "Don't Block the Event Loop (or the Worker Pool)". Node.js. Retrieved 2022-03-30.
  5. ^ "Principles to Handle Thousands of Connections in Java Using Netty - DZone Performance". dzone.com. Retrieved 2022-03-30.
  6. ^ "MacBook Pro 14-inch and MacBook Pro 16-inch". Apple. Retrieved 2022-03-30.
  7. ^ "Frequently Asked Questions (FAQ) - The Go Programming Language". go.dev. Retrieved 2022-03-30.
  8. ^ "JEP draft: Virtual Threads (Preview)". openjdk.java.net. Retrieved 2022-03-30.
  9. ^ Rudell, Harald (2022-03-22). "Maximum number of virtual threads in Go".
  10. ^ Szczukocki, Denis (2020-03-18). "Difference Between Thread and Virtual Thread in Java | Baeldung". www.baeldung.com. Retrieved 2022-03-30.
  11. ^ "Why you can have millions of Goroutines but only thousands of Java Threads". rcoh.me. 2018-04-12. Retrieved 2022-03-30.
  12. ^ "Main - Main - OpenJDK Wiki". wiki.openjdk.java.net. Retrieved 2022-03-30.
  13. ^ "The Go Programming Language". go.dev. 2022-03-22. Retrieved 2022-03-30.
  14. ^ "Intel Technology Journal" (PDF).
  15. ^ "Threading and Tasks in Chrome". chromium.googlesource.com. Retrieved 2022-04-05.
  16. ^ Lu, Genchi (2021-07-22). "Java's Thread Model and Golang Goroutine". Medium. Retrieved 2022-04-05.


Retrieved from "https://en.wikipedia.org/w/index.php?title=Virtual_thread&oldid=1081165666"