MartinDelophy

MartinDelophy

Timeline Studio - I’m building an open-source, local-first AI video editor that runs in the browser

Hi everyone,

I’ve been building Timeline Studio, an open-source video editor that runs directly in a modern browser. It combines a multi-track timeline with browser-local AI features, rather than treating AI as a separate “upload a file and wait for a result” service.

The basic idea is simple: for supported workflows, project media and inference should stay on the user’s device. The difficult part was making that idea behave like an editor rather than a collection of AI demos.

What it currently does

Timeline Studio has a CapCut-style multi-track workspace with a contiguous main visual track, picture-in-picture overlays, captions, stickers, voiceovers, source audio, and music.

Clips remain editable: they can be moved, split, trimmed, duplicated, transformed, masked, filtered, animated, speed-adjusted, and keyframed.

The browser-local AI workflows include:

  • Multilingual voiceovers
  • Whisper-based automatic captions
  • WebGPU AI music generation
  • Subject detection, portrait matting, and smart framing
  • Object and watermark repair and video denoising
  • Vocal separation
  • Talking-portrait generation

Generated or processed media goes back into the editable project instead of becoming an opaque final result. Projects can be saved as portable .timeline files, and the editor has a separate deterministic export path for producing video.

Why build this in the browser?

I wanted to explore whether a serious creative workflow could be local-first without requiring users to install a large desktop editor or upload every source clip to an editing backend.

The browser now provides many of the required pieces:

  • WebGPU and ONNX Runtime Web for selected model inference
  • Web Workers to keep model execution, decoding, and export away from the UI thread
  • WebCodecs and native media APIs for frame-accurate media processing
  • Cache Storage and service workers for revision-pinned model delivery
  • IndexedDB and portable project files for persistent local editing state

But using these APIs together exposed a different class of problems. Browser AI is not just about loading a model. It is also about memory lifetime, execution-provider differences, media timestamps, caching, progress reporting, cancellation, and keeping preview and export consistent.

Three engineering problems that took more work than expected

1. A video timeline must respect source timestamps

Generating a strip of video thumbnails sounds easy until a clip contains variable frame timing or a hard cut. Sampling the nearest decoded frame can reveal the next shot before the playhead reaches the real cut.

The editor keeps the ordinary filmstrip readable while replacing the cell beneath the playhead with the exact source presentation-timestamp frame.

Long videos publish a bounded seed set first, then refine visible cells before continuing through offscreen cells in the background. Scrubbing temporarily pauses that refinement so pointer movement remains responsive.

2. Preview and export are two rendering systems

A responsive editor preview can use native media playback, but deterministic export has to reconstruct the composition frame by frame.

Crop geometry, masks, captions, overlays, effects, speed curves, and audio timing must agree between both paths.

The project therefore treats the editable timeline as the source of truth. Export uses a separate offline renderer, with native browser encoding where possible and compatibility fallbacks when codec support differs.

3. Local-first model delivery still has real costs

Keeping inference local avoids uploading project media for supported operations, but it moves the cost to first-run downloads, device storage, GPU and CPU capability, and memory pressure.

Models are lazy-loaded, pinned to immutable revisions, cached by a shared service worker, and mirrored across Hugging Face and ModelScope.

The UI reports actual setup progress and keeps initialized workers alive for repeated runs. A local-first product still needs honest failure states; it should not pretend every device can run every model equally well.

Making the timeline usable mattered as much as the AI

A surprising amount of development time went into interaction details rather than inference:

  • Keeping the main visual track magnetically gapless
  • Supporting rolling boundaries and explicit ripple editing
  • Preserving wheel-gesture semantics while the timeline scrolls beneath the pointer
  • Keeping linked captions and voiceovers aligned without forcing them to remain linked forever
  • Preventing overlapping audio clips from unexpectedly moving to another lane
  • Making progressive thumbnail preparation coexist with real-time scrubbing
  • Keeping desktop and touch workflows usable in 11 interface languages

These details are less impressive in a model demo, but they determine whether the result feels like a tool people can actually edit with.

An agent can edit the project without reducing it to a rendered video

The repository also includes a Timeline Studio Skill and a versioned command runner for coding agents.

An agent can inspect a portable project, produce a semantic diff, apply supported edits transactionally, and write a new .timeline artifact instead of overwriting the input.

That direction is important to me: an AI agent should make reversible edits with stable clip IDs and explicit timestamps, while the human keeps an editable project and can inspect what changed.

Current limitations

This is still an evolving project, not a claim that the browser has replaced every desktop NLE.

  • The heaviest AI workflows require a modern Chromium browser and benefit significantly from WebGPU.
  • First use can involve large model downloads; later runs reuse the browser cache.
  • Performance and codec support vary by device and browser.
  • Some integrations use remote providers and are clearly presented as connectors rather than local inference.
  • Third-party model weights have their own licenses even though the original application code is MIT-licensed.

I’d value developer feedback

I’m especially interested in feedback from people working with WebGPU, WebCodecs, ONNX Runtime Web, browser media pipelines, or timeline editors.

If you try it, I’d love to know:

  1. Which local-first workflow is most useful to you: captions, voice, repair, music, or something else?
  2. Where should a browser editor draw the line between local inference and remote providers?
  3. Which device and browser combination did you use, and where did performance or compatibility break down?

Thanks for taking a look. Bug reports, focused contributions, and architecture discussions are all welcome.

Where Next?

Popular Frontend topics Top

wolf4earth
I’m currently in the process of rebuilding my personal website/blog from scratch. At the moment my website is using hugo, and while I thi...
New
AstonJ
Another WebAssembly Frontend Framework for Rust :nerd_face: Seed is a frontend Rust framework for creating fast and reliable web apps w...
New
New
First poster: bot
passiomatic/elm-designer. A code generator for Elm UI. Contribute to passiomatic/elm-designer development by creating an account on GitH...
New
vLev
As a 16y/o with enormous amounts of time during quarantine, I decided to learn neural networks. To further understand the concept, I deci...
New
First poster: bot
This thread was posted by one of our members via one of our news source trackers.
/js
New
First poster: bot
Solid stands on the shoulders of giants, particularly React and Knockout. If you’ve developed with React Hooks before, Solid should seem ...
New
NKTgLaw
The NKTg Law (Law of Variable Inertia) introduces a new way to treat inertia not only as a theoretical physics concept but as quantifiabl...
New
abhay-rana
Bit of a personal itch-scratch post — curious if anyone else runs into the same thing. Every time I kick off a new React project I end u...
New
MartinDelophy
Hi everyone, I’ve been building Timeline Studio, an open-source video editor that runs directly in a modern browser. It combines a multi...
New

Other popular topics Top

AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
AstonJ
Do the test and post your score :nerd_face: :keyboard: If possible, please add info such as the keyboard you’re using, the layout (Qw...
New
Exadra37
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
New
AstonJ
If you get Can't find emacs in your PATH when trying to install Doom Emacs on your Mac you… just… need to install Emacs first! :lol: bre...
New
PragmaticBookshelf
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
New
PragmaticBookshelf
Author Spotlight Mike Riley @mriley This month, we turn the spotlight on Mike Riley, author of Portable Python Projects. Mike’s book ...
New
sir.laksmana_wenk
I’m able to do the “artistic” part of game-development; character designing/modeling, music, environment modeling, etc. However, I don’t...
New
PragmaticBookshelf
Get the comprehensive, insider information you need for Rails 8 with the new edition of this award-winning classic. Sam Ruby @rubys ...
New
NewsBot
Node.js v22.14.0 has been released. Link: Release 2025-02-11, Version 22.14.0 'Jod' (LTS), @aduh95 · nodejs/node · GitHub
New
mindriot
Ok, well here are some thoughts and opinions on some of the ergonomic keyboards I have, I guess like mini review of each that I use enoug...
New