Make Videos Programmatically with JSON

The cleanest definition of programmatic video: a video is a document, rendering is a function, and the document is JSON.

Describe the timeline as structured data — which clips, on which tracks, what text, which effects, how long — hand it to a rendering engine, and collect an MP4. No framework lock-in, no browser choreography, no proprietary project files that only one application can open. Just data in, video out.

Meishe’s rendering pipeline works exactly this way, and its JSON examples are worth reading even if you end up choosing another engine — they show how completely a professional video can be expressed as plain data.

The Anatomy of a Video-as-JSON

Strip the specifics and every JSON video description has the same skeleton:

timeline
├── videoTracks[]        → clips with paths, in/out points, speed, flips
├── audioTracks[]        → music and voice, volumes
├── captionTracks[]      → text: styling, animation, timing
├── stickerTracks[]      → images, stickers, watermarks
├── timelineVideoFxTracks[] → filters and effects with intensity
└── duration, fps, resWidth/resHeight, videoSize

The parts that surprise teams new to JSON video:

Time is in microseconds. Not seconds, not frames — microseconds, because the engine resolves frame boundaries itself at the output fps. Precision belongs to the machine.

Effects are parameterized. An effect isn’t baked into a clip; it’s an object with a type, an intensity between 0 and 1, and a parameter list. “Warm filter, 60% strength, from second two to second five” is three JSON fields.

Everything is addressable. Because every element is a field, every element is automatable. The title, the clip order, the transition timing — nothing in the document requires a human at an editing surface.

Why JSON Beats Code-Described Video

Frameworks that describe video as code (React components and the like) are powerful for developers. But data as the source of truth has structural advantages for products:

Any stack can generate it. Python, Go, Node, a no-code zap, an LLM — anything that can assemble a JSON object can author a video. Your rendering pipeline stops caring what your backend is written in.

Templates are data, not codebases. A template is a JSON document with holes in it. Fill the holes from your database, post the result. Versioning templates is versioning documents — diffable, reviewable, roll-back-able.

Storage, search, and audit work. Millions of render requests as JSON documents are queryable: which templates ran, which assets appeared, what durations shipped. Try doing that over a folder of React components.

The same document opens in an editor. This is the Meishe-specific kicker: the JSON your pipeline renders is the project the interactive editor edits. Machine output and human refinement share one format — no export/import bridge, no lossy conversion.

A Real Payload, Abbreviated

From Meishe’s examples — one clip, one styled caption, enough structure to recognize:

{
  "timeline": {
    "duration": 49200000,
    "fps": 25,
    "resWidth": 960,
    "resHeight": 540,
    "videoSize": "16:9",
    "videoTracks": [{
      "videos": [{
        "path": "https://your-cdn.com/clip.mp4",
        "inPoint": 0,
        "outPoint": 49200000,
        "trimIn": 0,
        "trimOut": 49200000
      }]
    }],
    "captionTracks": [{
      "captions": [{
        "text": "Spring Sale",
        "inPoint": 5200000,
        "outPoint": 20000000
      }]
    }]
  }
}

Every field there is one you could generate from a spreadsheet row. That’s the entire trick: if your data has a title, a clip URL, and a duration, your data has a video.

Designing Your Own Video JSON Layer

If you’re building a product on JSON-driven video, three practices pay for themselves immediately:

Separate business data from timeline data. Your API should accept {"product": {...}, "template": "promo-v2"} — not raw timelines. Let a template-resolution step merge them. Designers change templates; your payload schema stays stable for years.

Validate before you queue. Missing media URLs, zero durations, unreachable paths — catch them at payload build time, not at render time. A render farm discovering a broken URL is the most expensive possible validation.

Treat renders as idempotent jobs. Same JSON in, same video out. Retry-safe, cache-friendly, and the foundation for per-template metrics when volume grows.

From JSON to Farm

Single videos validate the pipeline; scale proves it. Meishe’s batch rendering architecture — task scheduler, worker fleet, shared storage — exists precisely for the JSON-in/video-out loop at volume: nightly catalog runs, personalized campaigns, continuous content generation. The same payloads that render once in testing are the payloads the farm chews through in production.

Start with one timeline, one render, one honest look at the JSON examples. The moment a video becomes data, everything downstream — generation, versioning, scaling, automation — becomes an engineering problem you already know how to solve.