ReferencePage 10 of 14
Native Streamer
The in-process NVST streamer in the Qt app, its video backends per platform, and its diagnostics
On this page8 sections
The native streamer is the production desktop streaming path. The Qt app loads opennow-streamer-ffi as an in-process shared library. The streamer owns NVST transport, decode, audio, gameplay input, source-stream Matroska recording, and GPU frame publication. NVST transport means RTSPS signaling, a dedicated Mjolnir SRTP video socket, and an ICE/DTLS/SCTP bundle for audio, microphone, RTCP, and input. The Qt app has no Chromium or browser WebRTC fallback.
No Electron child process
Older docs described an experimental Electron-supervised Rust child process with a GStreamer webrtcbin backend and web fallback. That path is retired with Electron. Packaged Qt builds embed the streamer library beside the executable and opennow-core.
The core can also start a session over a native WebRTC transport, called WebRTC compatibility mode. It runs inside the same embedded streamer and uses the same decoders and GPU presentation. See Protocol notes.
How Qt uses it#
- Qt supplies one complete CloudMatch session context to the embedded engine.
- The runtime reserves its sockets and performs the NVST session lifecycle (OPTIONS, DESCRIBE, SETUP, ANNOUNCE, PLAY, keepalive, TEARDOWN).
- Qt lends current QRhi native graphics objects to the runtime.
- Platform decode publishes a native GPU frame into a one-frame mailbox.
- Qt imports the converted texture into the scene graph so QML overlays compose above video in the same window.
There is no separate streamer window, helper application, or packaged standalone streamer binary in Qt releases. The standalone opennow-streamer host remains a development tool only.
Settings that matter#
| Setting | Default | Purpose |
|---|---|---|
streamClientMode | "native" | Desktop streaming mode; the Qt app always streams natively |
transportMode | "nvst" | Native media transport. Legacy values such as "webrtc" are rewritten to "nvst" |
nativeVideoBackend | "auto" | Settings → Stream → Advanced → Video backend. Applies to the next stream. A forced backend that is unavailable fails instead of switching to another API |
codec | "auto" | Codec preference. The core resolves Auto against the streamer's capability report before CloudMatch allocates a seat |
colorQuality | "8bit_420" | Bit depth and chroma. Choices the current backend, codec, or membership cannot use are disabled with a reason |
enableHdr | false | HDR request for new sessions; needs an HDR-capable display and a 10-bit HEVC or AV1 decoder. A store version whose catalog entry does not list HDR streams in SDR, with a one-time Streaming in SDR notice |
audioOutputDevice | "" | Opaque output device ID from native enumeration; empty means system default |
showNativeStreamerStats | false | Native stats overlay preference |
enableCloudGsync | false | Cloud G-Sync/VRR request |
webrtcCompatibilityMode | "auto" | Settings → Network → Advanced → WebRTC compatibility mode: Automatic, WebRTC, or NVST for new sessions |
On Windows the stream view uses DirectX 11. The picker also lists DirectX 12 and Vulkan, but marks them as not supported by this stream view. On Linux, Auto never falls back to software decoding; pick Software (CPU) yourself when no hardware decoder works. Leave Auto selected unless you are isolating a known GPU issue.
The color picker lists 4:4:4 choices with the note "Coming soon". They are only enabled on Windows and macOS with an Ultimate membership, and only when the runtime reports a matching hardware decode profile.
Platform video paths#
| Platform | Native backend paths |
|---|---|
| Windows | Media Foundation hardware decode on D3D11; DirectX 11 only in the stream view |
| macOS (Apple Silicon) | VideoToolbox hardware decode; Metal presentation |
| Linux | Vulkan Video, VA-API, CUDA, or V4L2 hardware decode; FFmpeg software on request; Vulkan presentation |
HDR needs a 10-bit HEVC or AV1 hardware path on the platform and an HDR-capable output. H.264 and software decoding are SDR only. On Linux, HDR output needs a single-screen Wayland session with color management; X11 stays SDR.
Recovery#
The streamer requests retransmission of lost video packets, uses forward error correction data from the server, and asks for a new keyframe (IDR) when a frame cannot be repaired. If video stops, the app retries the session a bounded number of times (8 attempts) before it reports a failure.
- After unrecoverable loss, the streamer reports the lost frame range to the server (reference-frame invalidation) and holds back frames that depend on it until a recovery frame arrives. An IDR request remains the backstop.
- When the chosen codec, chroma, or HDR mode cannot be decoded, launch falls back to a profile that can be (for example 4:4:4 to 4:2:0, or HDR to SDR) instead of failing. Each fallback is recorded in diagnostics.
- Game audio passes through a timestamp-ordered jitter buffer (45 to 200 ms), and the microphone sends plain Opus on the bundle's fixed payload type 111 and SSRC 1.
Packaging#
The Qt CMake build always compiles opennow-streamer-ffi with Cargo's release profile and links it into opennow-qt. CPack installs that library beside the Qt executable and opennow-core.
| Target | Qt packaging behavior | Artifacts |
|---|---|---|
| Windows x64 / ARM64 | Qt package embeds the streamer library beside OpenNOW and opennow-core. setup.exe installs per user into %LOCALAPPDATA%\OpenNOW; the MSI and portable ZIP carry the same binaries. Not Authenticode-signed, so SmartScreen can warn | OpenNOW-Qt-<version>-Windows-<arch>-setup.exe, .msi, portable .zip |
| Linux x64 / ARM64 | AppImage recommended, with a .zsync sidecar for external delta-update tools. DEBs bundle Qt, QML plugins, SDL3, and the media runtime under /opt/opennow. The Flatpak (io.github.opencloudgaming.OpenNOW, x86_64) is a local or workflow build only and has no in-app updates | OpenNOW-Qt-<version>-Linux-<arch>.AppImage, .AppImage.zsync, .deb; Flatpak via workflow or local build |
| macOS Apple Silicon | DMG or ZIP installs to Applications on macOS 13 or later; Intel Macs are not included. Nightlies publish the DMG only | OpenNOW-Qt-<version>-Darwin-arm64.dmg, .zip |
Look for assets named OpenNOW-Qt-… in GitHub Releases. Linux packages include a bundled FFmpeg fallback; GPU driver interfaces are discovered at runtime. The AppImage and DEB launcher adds the common VA-API driver directories to the search path when LIBVA_DRIVERS_PATH is not set.
Local JSON protocol#
The embedded engine speaks a versioned local JSON protocol (protocol 7 in v1.0.3; dev uses 8) through the FFI command/response/event callbacks. The FFI ABI version is 15 in v1.0.3 and 18 on dev; it was 11 in v1.0.2, 12 from v1.0.3-nightly.983.1, 13 from v1.0.3-nightly.1026.1, 15 from v1.0.3-nightly.1107.1, and 16 from v1.0.4-nightly.1515.1. Important user-facing capabilities include:
- session start/stop and telemetry
- microphone enable/toggle/state
- audio device enumeration
- local playback mute
- clipboard text submission
- controller rumble events
See the upstream streamer README and FFI README for command fields and ABI rules.
Diagnostics#
Settings → About & support → Copy diagnostics → Export builds a bounded, redacted report. The diagnostics folder is diagnostics/ inside the data directory:
| Platform | Default data directory |
|---|---|
| Windows | %APPDATA%\OpenNOW |
| macOS | ~/Library/Application Support/OpenNOW |
| Linux | $XDG_CONFIG_HOME/OpenNOW, or ~/.config/OpenNOW |
OPENNOW_DATA_DIR replaces the data directory. The export can include native-streamer.log and qt-native.log and their .previous copies. Neither log contains SRTP keys, ICE credentials, or packet payloads.
When reporting black screens, say whether the entire application window is black or only the video surface is black. Include GPU, driver, package name, and the diagnostics export. Audio alone does not prove that video is arriving.
For environment overrides and transport internals, see Native Streamer Architecture.
Related pages#
- Architecture
- Configuration
- Troubleshooting
- Native Streamer Architecture
- Protocol notes (Electron-era WebRTC history and the WebRTC compatibility mode)