Runway Labs tests Project Continuum with real-time video
Runway Labs unveiled Project Continuum, a prototype operating system that renders interfaces as real-time video. It remains an experimental lab project.
The conceptual prototype replaces standard windows and UI code with live video frames, remaining an experimental demo without public software or launch plans.
On October 1, 2026, Runway Labs introduced Project Continuum. The project is an experimental operating system prototype that replaces conventional desktop windows, menus, and user interface code with interactive video generated in real time by artificial intelligence. Runway unveiled the concept through a video demonstration and technical posts on its official X account.
How Project Continuum replaces windows with video
Conventional operating systems, such as Windows, macOS, or Linux, build desktop interfaces out of deterministic code. Buttons, text fields, and window frames are explicitly defined structures. When you click an icon or drag a window edge, the operating system recalculates coordinates and redraws those graphic elements.
Project Continuum takes a different path by predicting the entire screen frame by frame as video pixels. The underlying engine is Solaris, an interface world model that Runway announced on August 31, 2026, based on its Gen-4.5 architecture. Solaris takes mouse clicks and cursor movement as input data and predicts subsequent video frames. To keep latency manageable, Runway combines autoregressive generation with step distillation techniques.
Four concepts shown in the demonstration
Runway structured its presentation around four interaction modes:
- Contextual Portals: floating, soft-edged windows that appear during web browsing to illustrate simulations or concepts, which users can reposition beside the main page.
- Responsive Video Interfaces: video surfaces that adapt within three-dimensional space when resized. In one example, a straight staircase reshapes itself into a spiral staircase as the window narrows, rather than cropping the visual content.
- Visual Thinking: a supervision interface for autonomous coding agents, displaying a continuous video feed of the tools and execution steps the software runs.
- Interactive Worlds: tools that convert static photographs or brief video clips into three-dimensional environments navigable by people or digital agents.
The practical appeal and technical hurdles
Using video as an interface surface addresses an obvious challenge in supervised automation. Tracking autonomous coding agents through terminal output requires heavy mental effort. A direct visual stream eases that friction by showing what the software is doing step by step.
Despite the visual appeal, the design runs into basic hurdles that computing solved decades ago. Because the interface is rendered as video pixels, it lacks a native semantic tree. That breaks screen readers and leaves the system inaccessible to blind and low-vision users. Familiar expectations such as keyboard navigation, undo commands, and state persistence also become uncertain when every frame relies on probabilistic output that can suffer from visual hallucinations.
What remains unanswered
Runway noted that Continuum is strictly an exploratory lab prototype. The company has not released downloadable binaries, browser extensions, or a public release schedule. It has also withheld application programming interfaces (APIs), software development kits (SDKs), and integration guides for third-party developers.
The computing infrastructure required to run the system at scale also remains unknown. Generating uninterrupted video in real time demands significant computing power. Runway stated that Solaris targets latency under 500 milliseconds at 720p resolution, but there are no independent benchmarks testing Continuum under realistic workloads. The company has not disclosed network bandwidth requirements, local or cloud GPU specifications, or how much it costs per minute to keep inference running.
Sources
Press
Official