Wine-NSPA – Client Scheduler Architecture

This page documents the client-side scheduler hosts and the current consumers routed through them.

Overview
Thread model
API surface
Current consumers
Validation and current controls
Relationship to the rest of Wine-NSPA
References

1. Overview

The architectural prerequisite was upstream spawn-main: the Unix bootstrap thread no longer becomes the application’s Win32 main thread. Instead, the bootstrap thread parks in sched_run() and becomes a per-process scheduler host, while the app main thread is created separately and continues through normal Win32 startup.

On top of that split, Wine-NSPA uses a client-side scheduler substrate:

one always-present default-class sched thread, named wine-sched
one lazy RT-class sched thread, named wine-sched-rt, created only when an RT consumer actually registers work and NSPA_RT_PRIO is configured
ntdll_sched_* entry points for poll, timer, async, synchronous call, and cancel

The purpose is not to replace wineserver dispatch. Gamma remains wineserver-side. This scheduler is the client-process sidecar used to host small helper loops and timer dispatchers without adding more dedicated helper threads per subsystem.

2. Thread model

The thread model has two classes.

Class	Thread name	Spawn policy	Scheduler policy	Purpose
Default	`wine-sched`	always present after spawn-main	`SCHED_OTHER`	general poll/timer/async/call hosting
RT	`wine-sched-rt`	lazy, first RT-class registration only	`SCHED_FIFO` at `NSPA_RT_PRIO - 1`	precision timer consumers that used to own dedicated RT helper threads

Two details matter:

The default sched thread is intentionally pinned to SCHED_OTHER. It is a general callback host and must not become an RT thread accidentally.
The RT sched thread is optional. If RT promotion is unavailable, RT-class registrations return STATUS_NOT_SUPPORTED and the caller falls back to its legacy dedicated-thread path.

The scheduler implementation itself uses:

PI mutexes around each sched-instance user list
a non-blocking wake pipe so producers never deadlock on a full signal fd
generation-stamped handles for ABA-safe cancel
self-call detection in ntdll_sched_call() so the sched thread does not deadlock by waiting for work it must dispatch itself

3. API surface

The API surface is:


NTSTATUS ntdll_sched_register_poll( int fd, int events,
                                    poll_callback callback,
                                    void *private,
                                    sched_handle_t *handle );

NTSTATUS ntdll_sched_register_timer( const LARGE_INTEGER *timeout,
                                     async_callback callback,
                                     void *private,
                                     sched_handle_t *handle );

NTSTATUS ntdll_sched_async( async_callback callback, void *private );
NTSTATUS ntdll_sched_call( call_callback callback, void *private );
NTSTATUS ntdll_sched_cancel( sched_handle_t handle );

NSPA adds class routing through NTDLL_SCHED_CLASS_DEFAULT and NTDLL_SCHED_CLASS_RT. Consumers that need RT dispatch call the class-aware registration helpers; consumers that only need a general callback host stay on the default instance.

The important semantics:

register_* returns a generation-stamped handle that can be canceled later
cancel is best-effort and returns STATUS_NOT_FOUND on a stale or already consumed handle
async is fire-and-forget
call is synchronous to the caller, but self-calls from the sched thread run inline to avoid deadlock

4. Current consumers

4.1 Async close queue on `wine-sched`

The first real consumer is the local-file async close queue. For eligible fully-shareable local-file handles, NtClose no longer pays unix close() and server close_handle latency inline on the caller thread. Instead it pushes a bounded queue entry to the default sched thread.

The rules are conservative:

queue capacity: 64
eligibility: FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE only
pre-flush on same-path reopen to close the local race window
fallback inline on queue-full, disabled sched routing, or restrictive sharing

This is a latency and consolidation feature, not a semantic change. Restrictive sharing closes still go inline immediately so any blocked opener sees the close at the same point as before.

4.2 `local_timer` and `local_wm_timer` on `wine-sched-rt`

The next consumers are the timer dispatchers that used to own separate RT helper threads:

nspa_local_timer
nspa_local_wm_timer

When RT is available, both route onto the shared RT sched instance instead of running dedicated pthread loops. The priority class is unchanged from the legacy design: SCHED_FIFO at NSPA_RT_PRIO - 1. The win is consolidation and shared infrastructure, not a different scheduling policy.

When both migrations are active together, the process loses one helper thread relative to the pre-migration layout because two legacy loops collapse onto one shared wine-sched-rt host.

4.3 Observability sampler on the default class

NSPA_SCHED_OBS_INTERVAL_MS enables a periodic sampler hosted on the default class. It is not a production fast path, but it is active and useful because it exercises the timer and cancel paths continuously with a real in-tree consumer.

Current built-in output is written to /dev/shm/nspa-obs.<pid> and includes stats such as:

close-queue depth
RT-probe liveness and firing counters when the RT probe is enabled

This sampler remains default OFF.

5. Validation and current controls

The public status here is based on targeted validation of the current consumers, not on a new full-suite publish.

The scheduler consumers no longer expose per-feature opt-out gates in the public surface. Async close routing, local_timer, and local_wm_timer all run on the normal path when their own eligibility checks pass and RT is available. The one remaining public control here is the optional sampler:

Item	Default	Purpose
`NSPA_SCHED_OBS_INTERVAL_MS`	OFF	opt-in scheduler-host sampler for observability only

Targeted 2026-05-02 results:

spawn-main carry: Smoke 0 / 1 / 2 / 3 PASS
Ableton playback after spawn-main: wineserver observed at 0.0% CPU during playback
combined sched-RT migrations: 10/10 RT-probe regression check PASS
combined sched-RT migrations: Ableton boot, library scan, project load, and playback PASS
combined sched-RT migrations: net -1 thread per process relative to the pre-migration layout

6. Relationship to the rest of Wine-NSPA

This page is client-side infrastructure. It composes with, but does not replace:

Gamma channel dispatcher: still wineserver-side and still the server RPC path for requests that need wineserver authority
NT-local stubs: the sched threads host some of those client-local dispatchers, especially timers
io_uring: separate client-side async I/O substrate; not routed through ntdll_sched

The main decomposition consequence is that the client side has a cleaner place to host helper loops. That shrinks the number of ad-hoc per-subsystem threads and moves more timing-sensitive work out of the wineserver process without changing wineserver ownership of cross-process semantics.