Skip to content
← Back to docs
← demain.technology

Dual-Display Wayland Architecture: the tactile surface and Virtual the display output Output

The Portal OS platform runs a single Wayfire compositor instance that manages two logically distinct displays: the tactile surface, the physical 2240×1400 touchscreen on the Portal OS hardware laptop, and the display output, a virtual 2560×1080 headless output that serves as the rendering canvas for AR glasses. This dual-display design decouples the spatial application surface from the physical touch panel, allowing the compositor to composite a wide-aspect-ratio canvas optimized for the AR glasses optical field of view while simultaneously driving the laptop’s built-in screen for keyboard and touch interaction.

System Overview: Two Outputs, One Compositor

Section titled “System Overview: Two Outputs, One Compositor”

The architecture is best understood as a single-process Wayfire compositor owning the DRM backend, with a plugin dynamically attaching a headless virtual output. The physical the tactile surface panel handles direct user interaction (keyboard, touch fusion), while the display output is never scanned out to any physical panel, it is captured frame-by-frame via the zwlr_export_dmabuf_v1 protocol, encoded to H.265, and broadcast over Wi-Fi to the glasses.

graph TB
    subgraph "Wayfire Compositor (portal.service)"
        WC["Wayfire 0.11<br/>WLR_BACKENDS=drm"]
        
        subgraph "Physical Display"
            EDP["the tactile surface<br/>2240×1400<br/>Touchscreen Panel"]
            KB["portal-input.service<br/>(layer_shell OVERLAY)<br/>Rive keyboard + touch"]
        end
        
        subgraph "Virtual Display"
            PVO["portal-virtual-output<br/>plugin"]
            HEAD["HEADLESS-1 (disabled)"]
            GLASS["the display output<br/>2560×1080<br/>position=0,0"]
            SPAT["portal-spatial plugin<br/>(zone placement)"]
            WARP["portal-spatial-warp plugin<br/>(perspective correction)"]
        end
        
        WC --> EDP
        WC --> PVO
        PVO -->|creates| GLASS
        WC -.->|disables| HEAD
        WC --> GLASS
        GLASS --> SPAT
        GLASS --> WARP
    end
    
    subgraph "Streaming Pipeline"
        PS["portal_stream<br/>dmabuf capture"]
        EGL["EGL UBWC Linearizer"]
        V4L2["v4l2h265enc<br/>(IRIS HW encoder)"]
        RTP["rtph265pay → udpsink"]
        
        GLASS -->|zwlr_export_dmabuf_v1| PS
        PS --> EGL
        EGL --> V4L2
        V4L2 --> RTP
    end
    
    subgraph "Network"
        AP["the wireless interface (5GHz AP)"]
        GLS["AR glasses Glasses"]
        RTP -->|UDP 192.168.50.255:5000| AP
        AP --> GLS
    end
    
    subgraph "Input Bridge"
        UIN["/dev/uinput<br/>'Portal OS Virtual Pointer'"]
        UIN -->|map_to_output| GLASS
        KB -->|fusion → uinput| UIN
    end

The compositor service runs Wayfire with WLR_BACKENDS=drm and binds the DRM device at /dev/dri/card0, establishing DRM master and performing atomic modeset on the tactile surface. The portal-virtual-output plugin then injects a headless output into Wayfire’s multi-backend, which Wayfire picks up as a new Wayland output. All window placement, spatial zone logic, and perspective warp rendering operate exclusively on the display output.

The portal-virtual-output Plugin: Headless Output Injection

Section titled “The portal-virtual-output Plugin: Headless Output Injection”

The virtual output is not configured statically in wayfire.ini, it is dynamically created at runtime by the portal-virtual-output Wayfire plugin. This C++ plugin operates at the wlroots backend level, creating a headless backend, adding it to Wayfire’s multi-backend, and then emitting the new_output signal so Wayfire’s output layout manager registers it.

The plugin’s init() method performs three critical steps: (1) it calls wlr_headless_backend_create() on the compositor’s event loop, (2) attaches the headless backend to the multi-backend via wlr_multi_backend_add(), and (3) creates a 2560×1080 output through wlr_headless_add_output(). The output is initially named HEADLESS-1, a name that collides with the default headless output that wlroots’ headless backend auto-creates at 1280×720. The wayfire.ini configuration explicitly disables this default [output:HEADLESS-1] with mode = off to prevent a phantom output from claiming the (0,0) coordinate origin.

sequenceDiagram
    participant WF as Wayfire Core
    participant PVO as portal-virtual-output
    participant HB as Headless Backend
    participant MB as Multi-Backend
    participant OL as Output Layout

    WF->>PVO: init()
    PVO->>HB: wlr_headless_backend_create(loop)
    PVO->>MB: wlr_multi_backend_add(backend, headless)
    PVO->>HB: wlr_headless_add_output(2560, 1080)
    HB-->>PVO: wlr_output* (named "HEADLESS-1")
    PVO->>MB: wl_signal_emit(new_output, virtual_output)
    MB->>OL: register output
    OL->>WF: layout configured (the display output at 0,0)
    WF-->>WF: HEADLESS-1 disabled (mode=off)

The manual signal emission via wl_signal_emit_mutable is necessary because Wayfire only auto-discovers outputs during initial backend setup, outputs added later by a plugin must be explicitly announced. Without this emission, the virtual output would exist at the wlroots level but Wayfire’s compositor logic would never create a wf::output_t for it, rendering it invisible to all window placement and rendering code.

Output Layout and Coordinate System

Section titled “Output Layout and Coordinate System”

The wayfire.ini configuration file defines a three-output layout with precise geometric positioning to ensure correct window placement and capture semantics. The interplay between these output configurations is subtle, a misconfiguration results in windows landing on invisible surfaces or the streaming pipeline capturing blank frames.

Output Resolution Position Status Purpose
the tactile surface 2240×1400 (2560, 0) Active Physical touchscreen, keyboard + touch fusion
HEADLESS-1 1280×720 (default) N/A Disabled (mode = off) Default wlroots headless ghost, suppressed
the display output 2560×1080 (0, 0) Active Virtual spatial canvas, captured for streaming

The critical design decision is forcing the display output to position (0,0). Wayfire auto-positions outputs left-to-right based on registration order. Because the headless backend always creates a default 1280×720 HEADLESS-1 output that momentarily claims the (0,0) origin before the plugin’s virtual output is registered, auto-positioning would place the display output at (1280, 0), inside the disabled phantom area. The explicit position = 0,0 override ensures the spatial coordinate model (which assumes the 2560×1080 canvas starts at global origin) places windows inside the region that portal_stream captures.

The the tactile surface panel is positioned at (2560, 0), to the right of the display output in the virtual layout. This placement is irrelevant for the streaming pipeline (which only captures the display output) but matters for Wayfire’s pointer behavior: without the position directive, Wayfire would attempt to auto-layout both outputs and potentially overlap them.

Window Placement: Forcing All Apps to the display output

Section titled “Window Placement: Forcing All Apps to the display output”

Two complementary mechanisms ensure every application window lands on the the display output rather than the tactile surface or the disabled HEADLESS-1.

Wayfire window-rules provide the first layer, a declarative rule forces any newly created view to start on the display output:

[window-rules]
rule1 = on created then start_on_output "the display output"

This fires for every view_created signal, redirecting the view to the display output before Wayfire’s own placement logic runs.

The portal-spatial plugin provides the second layer, it intercepts view_mapped signals and applies zone-specific geometry within the the display output canvas. When a window maps, the plugin calls into the Rust spatial library via FFI (portal_spatial_view_mapped) to determine which of three spatial zones the window should occupy. It then computes the zone’s X offset, clamps the window to zone dimensions (853×900 max), and centers it within the assigned zone.

Zone Model: Three-Zone Canvas at 2560×1080

Section titled “Zone Model: Three-Zone Canvas at 2560×1080”

The the display output canvas is divided into three equal-width zones mapped to the AR glasses’s binocular field of view. The 21:9 ultrawide aspect ratio matches the glasses’ ~31° horizontal optical FOV and reduces H.265 encoder bandwidth by approximately 33% compared to the previous 3840×1080 (32:9) layout.

┌──────────────┬──────────────┬──────────────┐
│              │              │              │
│ LeftPeriph   │   Center     │ RightPeriph  │
│  x∈[0,853)   │  x∈[853,1706)│ x∈[1706,2560]│
│  width 853   │   width 853  │  width 854   │
│  Zone ID: 1  │   Zone ID: 0 │  Zone ID: 2  │
│              │              │              │
└──────────────┴──────────────┴──────────────┘

The 2560-pixel width divides into three zones of 853.33 pixels each. Integer truncation yields 853 for the left and center zones, with the right peripheral zone absorbing the 1-pixel remainder at 854 pixels wide. The portal-spatial-config.h header documents this rounding explicitly to prevent it from being mistaken for a bug. Window height is clamped to 900px (below the 1080px canvas height) to leave room for future shell chrome and to reduce visual fatigue in the AR headset.

Zone ID X Range Width Default Scale Display Priority
Center 0 [853, 1706) 853 1.0 100
Left Peripheral 1 [0, 853) 853 0.8 50
Right Peripheral 2 [1706, 2560] 854 0.8 50
Overlay 3 , , 1.0 200

The ZonePolicy engine in the WM daemon assigns windows to zones using a two-tier priority system. Utility applications (terminals, file managers, text editors) prefer peripheral zones, filling left first then right. Non-utility applications (browsers, media players) prefer the center zone. When all zones in the preferred tier are occupied, the engine falls back to the opposite tier. This policy is implemented in pure Rust and communicates zone assignments to the spatial plugin via the FFI boundary.

Input Routing: Virtual Pointer Confinement to the display output

Section titled “Input Routing: Virtual Pointer Confinement to the display output”

The Portal OS Virtual Pointer, a uinput device created by the input daemon, is explicitly mapped to the display output in the compositor configuration. This mapping is essential for correct spatial interaction: pointer events from the touch fusion engine must resolve to coordinates within the 2560×1080 virtual canvas, not the 2240×1400 the tactile surface touchscreen geometry.

[input-device:Portal OS Virtual Pointer]
map_to_output = the display output

The input daemon (portal-input.service) runs as a Wayland layer-shell client on the tactile surface, receiving raw touch events from the capacitive touch sensor via direct evdev. It fuses touch coordinates into pointer/button events and injects them through /dev/uinput as relative movement deltas. The compositor then applies these deltas to the cursor on the display output (the mapped output), translating physical touches on the laptop screen into spatial pointer positions on the virtual canvas.

This decoupling means the tactile surface panel serves only as a touch input surface for the keyboard daemon, the compositor’s pointer cursor never appears on the tactile surface. The keyboard daemon renders its own UI directly on the tactile surface via the layer-shell OVERLAY layer, independently of Wayfire’s cursor rendering.

The Capture Pipeline: From the display output to AR Glasses

Section titled “The Capture Pipeline: From the display output to AR Glasses”

The streaming daemon (portal_stream) connects to Wayfire as a Wayland client and captures the the display output using the zwlr_export_dmabuf_manager_v1 protocol. Despite the plugin naming the virtual output HEADLESS-1, the systemd service passes HEADLESS-1 as the capture target argument, this works because the daemon selects outputs by substring match against the Wayland output name, and the virtual output registered under that name matches.

flowchart LR
    subgraph "Capture"
        DMABUF["zwlr_export_dmabuf_v1<br/>frame_frame → frame_object"]
        UBWC["UBWC Modifier<br/>0x0500000000000001"]
    end
    
    subgraph "Linearization"
        EGL["eglCreateImageKHR<br/>(modifier-aware import)"]
        FBO["Scratch FBO +<br/>glReadPixels"]
        LIN["Linear BGRx<br/>(malloc'd)"]
    end
    
    subgraph "Encode"
        APPSRC["appsrc<br/>(zero-copy GstBuffer)"]
        VC["videoconvert<br/>BGRx → NV12"]
        V4L2["v4l2h265enc<br/>8 Mbps VBR"]
        PAY["rtph265pay pt=96"]
    end
    
    subgraph "Transport"
        UDP["udpsink<br/>192.168.50.255:5000"]
        HMAC["HMAC-SHA256<br/>pad probe (optional)"]
    end
    
    DMABUF --> UBWC
    UBWC --> EGL
    EGL --> FBO
    FBO --> LIN
    LIN --> APPSRC
    APPSRC --> VC
    VC --> V4L2
    V4L2 --> PAY
    PAY --> HMAC
    HMAC --> UDP

The EGL UBWC linearizer is the most architecturally significant component of the capture pipeline. wlroots’ gles2 renderer on the Adreno X1-85 allocates DMA-bufs with the DRM_FORMAT_MOD_QCOM_COMPRESSED modifier (Qualcomm Universal Bandwidth Compression). GStreamer’s videoconvert reads raw dmabuf bytes via mmap, which produces garbage because UBWC is a tiled/compressed layout. The linearizer solves this by importing each dmabuf via EGL (which understands UBWC through EGL_EXT_image_dma_buf_import_modifiers), rendering it to a scratch framebuffer, and calling glReadPixels to extract linear pixel data. The resulting CPU-allocated buffer is then wrapped in a GStreamer buffer and pushed to the encoding pipeline.

Service Dependency Graph

Section titled “Service Dependency Graph”

The dual-display architecture is orchestrated by four systemd services with precise ordering constraints. Each service depends on the compositor’s Wayland socket (/run/portal/wayland-1) but operates independently after initial connection.

graph TD
    SEATD["seatd.service"] --> PORTAL["portal.service<br/>(Wayfire compositor)"]
    PORTAL -->|wayland-1 socket| INPUT["portal-input.service<br/>(keyboard daemon on the tactile surface)"]
    PORTAL -->|wayland-1 socket| STREAM["portal-stream.service<br/>(captures the display output)"]
    PORTAL -->|wayland-1 socket| WM["portal-wm.service<br/>(toplevel tracking + zones)"]
    
    INPUT -->|BindsTo| PORTAL
    STREAM -->|PartOf| PORTAL
    
    PORTAL --> EDP["the tactile surface<br/>2240×1400<br/>(DRM modeset)"]
    PORTAL --> GLASS["the display output<br/>2560×1080<br/>(headless virtual)"]
    STREAM --> NET["the wireless interface AP → Glasses"]
    
    style PORTAL fill:#e1f5fe
    style GLASS fill:#fff3e0
    style EDP fill:#e8f5e9

The portal.service is the root dependency, it starts Wayfire, creates both outputs, and establishes the Wayland socket. The portal-input.service binds to the compositor (if Wayfire dies, the keyboard daemon also terminates) and renders on the tactile surface. The portal-stream.service is a PartOf the portal service (it restarts when the compositor restarts) and captures the display output. The portal-wm.service connects independently and monitors toplevel windows via the ext_foreign_toplevel_list_v1 protocol, assigning them to spatial zones.

All services share /run/portal as the XDG_RUNTIME_DIR, a deliberate choice that avoids the systemd-logind session directory (/run/user/1000) which gets recreated and destroyed as login sessions start and stop, deleting Wayfire’s socket in the process.

Configuration Reference

Section titled “Configuration Reference”

The dual-display architecture is configured through three layers of configuration files, each addressing a different concern:

Configuration File Deployment Path Purpose
wayfire.ini /etc/portal/wayfire.ini Output layout, plugin loading, input mapping, keybindings
spatial-warp.toml /etc/portal/spatial-warp.toml Canvas geometry and zone boundaries (runtime source of truth)
portal-stream.env /etc/portal/portal-stream.env Streaming daemon network config (bind/target/interface)

The portal-spatial-config.h C++ header provides compile-time defaults that must stay in sync with the TOML. The header comment explicitly states that if the two ever diverge, the TOML wins because it is read at runtime by libportal_spatial_plugin.so. The WM daemon’s Rust config module mirrors the same defaults in Config::default() and loads the TOML from /etc/portal/spatial-warp.toml at startup, falling back to defaults if the file is missing.

Key Design Decisions and Trade-offs

Section titled “Key Design Decisions and Trade-offs”

Why a virtual output instead of a real HDMI output? The the hardware’s the tactile surface panel is the only physical display. The previous legacy hardware architecture used HDMI-A-2 as the glasses output, but the Portal OS hardware migration eliminated the HDMI path in favor of a headless virtual output. This approach avoids contention for DRM master on a physical connector and eliminates the need for a dummy display EDID.

Why 2560×1080 instead of 3840×1080? The 21:9 ultrawide resolution matches the AR glasses’s ~31° horizontal optical FOV while reducing H.265 encoder bandwidth by ~33% versus the previous 32:9 layout. The three-zone model divides cleanly into 853-pixel segments.

Why HEADLESS-1 naming instead of the display output? The plugin currently sets the output name to HEADLESS-1 in source code, but the wayfire.ini references the display output for window placement and input mapping. This apparent mismatch exists because the streaming daemon captures by the HEADLESS-1 name (matching the actual wlroots output name), while the Wayfire configuration sections target the display output, which is the logical alias used in deployment documentation and the deployment state tracker. The wayfire.ini comments reference both names, explaining that the default HEADLESS-1 output must be disabled so only the the virtual display output canvas remains.

Why EGL linearization instead of TU_DEBUG=noubwc? The compositor service already sets TU_DEBUG=noubwc (Turnip Vulkan flag), but this does not affect wlroots’ GBM allocation path. wlroots uses gbm_bo_create_with_modifiers, which explicitly requests modifiers, the driver returns UBWC as a valid option and wlroots selects it regardless of debug flags. EGL is the only API layer that understands how to import UBWC-compressed dmabufs correctly on this platform.

Next Steps

Section titled “Next Steps”

The dual-display architecture touches several adjacent subsystems that warrant deeper exploration:

Last updated: