Video Cutscene Contract v1

Scope

aura.video is currently a visual playback surface, not a full cinematic media stack.

What is supported today:

• frame-sequence playback
• sprite-sheet playback
• native-only .mp4 playback on desktop hosts (in-process decode, no external tools required)
• supported video codecs in .mp4 containers: H.264, H.265, VP9, MPEG-4 Part 2
• play / pause / stop / seek
• texture-handle consumption from rendering surfaces
• playback state inspection through getInfo()

What is not supported as an engine-level guarantee today:

• browser/web codec-backed parity
• synchronized audio playback
• subtitle rendering
• timeline event callbacks from the native host
• checkpoint persistence semantics

That means the honest cutscene contract today is:

1. The engine owns video frame playback and texture updates.
2. Games own subtitles, checkpoints, skip rules, and cue/event logic in userland.
3. Audio sync remains out of scope even with native .mp4 playback.

Native .mp4 specifics are frozen separately in:

• docs/native-mp4-video-contract-v1.md

Userland Cutscene Controller

AuraJS now ships a small JS helper at @auraindustry/aurajs/cutscene for the current contract.

It wraps an aura.video-compatible API and gives you:

• ordered cue dispatch
• subtitle cue callbacks
• checkpoint cue callbacks
• checkpoint export / restore snapshots
• play / pause / resume / stop / seek / skip helpers
• deterministic re-fire behavior after seeking backwards

It also exposes thin helper wrappers for the current 3D media-presentation lane:

• createSpatialAudioEmitter(aura.audio, path, options?)
• createVideoBillboardSurface(aura.video, source, options?)
• createMediaPresentationController({ video: aura.video }, source, options?)

The authored native media-presentation boundary is frozen separately in:

• docs/native-media-presentation-contract-v1.md

Example

import { createVideoCutsceneController } from '@auraindustry/aurajs/cutscene';

const handle = aura.video.load('assets/intro.mp4', {
  type: 'mp4',
  looping: true,
});

const cutscene = createVideoCutsceneController(aura.video, handle, {
  skipTarget: 6.0,
  cues: [
    { time: 0.5, type: 'subtitle', text: 'Wake up.' },
    { time: 2.0, type: 'checkpoint', checkpoint: 'intro-midpoint' },
    { time: 4.0, type: 'event', id: 'door-open' },
  ],
  onSubtitle(cue) {
    ui.subtitle = cue.text;
  },
  onCheckpoint(cue) {
    gameState.lastCheckpoint = cue.checkpoint;
  },
  onCue(cue) {
    if (cue.id === 'door-open') playDoorSfx();
  },
});

cutscene.play();

aura.update = () => {
  cutscene.update();
};

Interpretation

This is intentionally not a Unity-style cinematic stack.

The current engine contract is good enough for:

• silent or stylized cutscenes
• native desktop .mp4 screens or short visual cutscenes
• billboard or in-world video screens
• scene-authored spatial emitters and screen-plus-audio presentation helpers
• lightweight narrative sequences driven by JS cues

It is not yet the right thing to call:

• full FMV support
• audio-locked cinematics
• subtitle/localization pipeline
• editor-authored timeline/cinematic tooling