Windows Subsystem for Linux -- Pico Process Overview
Overview
This post discusses pico processes, the foundation of WSL. It explains how pico processes work in Windows and goes into the history of how they came to be, the abstractions we decided to implement and the various use cases beyond WSL that emerged. Armed with this context, the next series of posts will dive into specific areas of how exactly WSL works.
Drawbridge
The pico process concept originated in MSR as part of the Drawbridge project. A goal of this project was to implement a lightweight way to run an application in an isolated environment, with the application’s OS dependencies decoupled from the underlying host OS (e.g., XP application running on Windows 10). Normally, this would be done by running the application and OS in a virtual machine, but this comes with significant resource overhead. Instead, the Drawbridge project aimed to run the target application and OS entirely within the user-mode address space of a single process on the host OS. With its reduced overhead, when compared to a VM, this approach allows for greater density of application workloads on a single host while still providing much of the same isolation and compatibility guarantees. Check out the official Drawbridge project page for more detailed information for how they pulled this off.
In Drawbridge, the “library OS” was the target OS of the application workload. To support a library OS that differs from the underlying host OS in addition to running the library OS in user-mode, the MSR folks needed the host OS to get out of the way and stop trying to manage the user-mode address space inside this process. They coined this type of process a “pico process” on the host side to indicate that it is a smaller version of a normal host process. A kernel-mode driver was responsible for supporting the pico process and acting as the broker between the host OS kernel and the library OS in user-mode.
Of course, proper support for this on top of a publicly released and supported version of Windows would require core Windows kernel changes to add the facilities necessary for pico processes. The MSR team brought this idea to us over in the kernel team toward the beginning of 2013, and we all agreed it would be great feature to add. In fact, we realized that this approach aligned well with some internal discussion we had been having around longer-term strategy to bring back the idea of subsystems to facilitate future architectural changes within Windows. This initial support for pico processes first appeared in Windows 8.1 and Windows Server 2012R2 but was limited to Drawbridge. Pico process support was later expanded to other Windows features.
Minimal Process
As we started implementing official support for pico processes, we decided to split the abstraction into two layers:
- Minimal process: This is the most rudimentary type of process. Specifically, a process marked as a minimal process tells the rest of the host to get out of the way and not manage it. From the Windows kernel point of view, it is simply an empty user-mode address space.
- Pico process: This is a minimal process with an associated pico provider kernel-mode driver to manage that empty user-mode address space.
Unlike traditional NT processes, when creating a minimal process, the user-mode address space is untouched and no threads are created to run in that process. Various locations in the kernel were surgically updated to skip user-mode address space setup, including:
- The user-mode binary ntdll.dll is not mapped into the process by default.
- The process environment block (PEB) is not created.
- There is no initial thread created, and thread environment blocks (TEBs) are not automatically created whenever a thread is created for the pico process.
- The shared user data section is not mapped into the process. This is a block of memory mapped read-only into all user-mode address space for efficient retrieval of common system-wide information.
- Various places that assumed a process would always have a PEB and/or TEBs were updated to be able to handle processes without them.
While the Windows kernel does not actively manage a minimal process, it still provides all of the underlying OS support you would expect – thread scheduling, memory management, etc.
You may be wondering – why did we separate the notion of “minimal” and “pico” processes? The idea of an empty minimal process seemed useful on its own, separate from anything related to supporting a pico process. While we had nothing specific in mind around this time back in 2013, eventually a couple of scenarios did surface in Windows 10 that are now using minimal processes directly:
- Memory Compression: Memory compression is a Windows feature that compresses unused memory to keep more data resident in RAM. It also reduces the amount of data written to and read from the pagefile, thus improving performance. Windows memory compression utilizes the user-mode address space of a minimal process.
- Virtualization based Security (VBS): Using underlying virtualization capabilities, VBS isolates the user-mode address space of critical user-mode processes from the rest of the OS to prevent tampering, even from the kernel or kernel-mode drivers. A minimal process is created to indicate to management tools (e.g., Task Manager) that VBS is running.
Pico Processes and Providers
A pico process is simply a minimal process that is associated with a pico provider kernel-mode driver. This pico provider surfaces the entire kernel interface as far as the user-mode portion of the process is concerned. The Windows kernel passes all system calls and exceptions that originate from the user-mode portion of a pico process to the pico provider to handle as it sees fit. This allows the pico provider to model a different user/kernel contract separate from what Windows would normally provide.
Early during boot, a kernel-mode driver registers with the Windows kernel as a pico provider, and the kernel and provider exchange a set of interfaces specific to the needs of a pico provider. For example, the pico provider provides function pointers for the kernel to call when dispatching a user-mode system call or exception, and the kernel provides function pointers for creating pico processes and threads.
Regardless of what behaviors and abstractions the pico provider exposes to user-mode, it ultimately will rely on the Windows kernel for underlying support of thread scheduling, memory management and I/O. Of course, portions of the Windows kernel had to be updated to support new scenarios where needed.
Windows Kernel Changes
Later posts will detail the Windows kernel changes in more detail, but here is a quick sampling:
- Improved fork support: Yes – the Windows kernel has supported “fork” for a long time (going back to earlier POSIX and SFU application support), but it is not exposed in the Win32 programming model that the rest of Windows is programmed against. We have improved the fork implementation to meet some new requirements as part of the WSL work.
- Fine-grained memory management: Windows normally manages the user-mode address space in 64KB chunks, but was updated to allow management at single-page 4KB granularity for pico processes.
- Case-sensitive file names: Again, yes – the Windows kernel and NTFS have long supported case-sensitive file names, but it is disabled by default and not exposed in the Win32 programming model. Changes were made to allow individual threads to opt-in to case-sensitivity operations to support a broader range of WSL scenarios.