Skip to content

Export: MP4 and MOV video of the deck, with our own H.264 encoder - #608

Open
NikitkaRa wants to merge 1 commit into
storytold:mainfrom
NikitkaRa:feat/video-export
Open

NikitkaRa wants to merge 1 commit into
storytold:mainfrom
NikitkaRa:feat/video-export

Conversation

@NikitkaRa

Copy link
Copy Markdown
Contributor

No issue. This is the "No video or animated GIF export" item from the after-beta list in docs/gaps.md. Animated GIF is #171/#177; this PR does the MP4/MOV half and doesn't touch GIF.

Summary

  • file.export with format: mp4|mov (or a .mp4/.m4v/.mov path): plays each slide's transition, then holds the slide for its "advance after" time, otherwise secondsPerSlide (default 5 s). Takes height (default 1080, from 480p to 4K; width follows the slide aspect, both even), fps (1–60, default 30) and includeHidden. The result reports bytes, frames, size and seconds.
  • New deckcraft_h264::Encoder (crates/h264/src/encoder.rs), written from H.264 itself, as the inverse of our decoder:
    • Constrained Baseline, CAVLC, 4:2:0, one slice per picture, deblocking off.
    • Every coded macroblock is Intra 16x16 (V, H or DC, picked by SAD) with chroma DC prediction.
    • In P pictures, macroblocks whose source is unchanged are skipped, so a held slide costs a few bytes a frame.
    • The SPS carries BT.709 VUI and timing, and the level comes from the size and frame rate.
  • crates/engine/src/video.rs:
    • Builds the timeline (capped at 2 h at 30 fps), renders frames through render_slide/render_transition and muxes them with deckcraft_isobmff::Mp4Writer (faststart, MP4 or QuickTime brand).
    • A key frame starts every slide and every transition, and at least every 2 s.
    • The file is written through a temporary sibling.
  • No external tools or new third-party crates: the engine now depends directly on our own deckcraft-h264 and deckcraft-isobmff (both lower layers; the layers check passes).
  • Docs: the MP4/MOV row in file-format-parity.md is marked yes, and the gaps.md entry now names what's left (in-slide animations, audio, GIF in Export: animated GIF of the deck (Export… dialog, CLI, MCP) (#171) #177).

Overlap: #177 also edits file.export and format_for_path in crates/engine/src/cmd/file.rs (for GIF), so expect a small textual conflict in whichever lands second.

Test plan

  • h264 decodes_with_our_decoder_at_good_quality: 200×120 (cropped), 4 pictures. The decoder's output matches the encoder's reconstruction bit for bit, PSNR > 38 dB, and unchanged pictures are < 16 bytes.
  • h264 big_levels_and_flat_pictures_round_trip (QP 10, noisy blocks, level escapes) and rejects_bad_sizes_and_writes_levels_for_big_pictures
  • engine timeline_holds_slides_and_plays_transitions, timeline_is_capped_and_sizes_are_even, frames_are_exactly_the_frame_size, encodes_a_playable_mp4 (demux, key frames at slide and transition starts, decode every sample)
  • By hand: deckcraft-cli run --sample --cmd 'transition.set=…' --cmd 'file.export={"path":"deck.mp4","height":720,"fps":25,"secondsPerSlide":1.5}' gives a 21 s 1280×720 file. ffprobe reports h264 Constrained Baseline yuv420p 25 fps, 528 frames, and ffmpeg -f null decodes it without errors. Frames pulled at 0.4, 1.2, 3.0 and 7.5 s show the push from black, the title slide, a push between slides and a later slide, checked by eye.
  • cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm)

Parity checklist: no checklist row for video export (gaps.md says "checklist row missing"), so file-format-parity.md is updated instead. No UI changes: the command is reachable from the CLI, MCP and command palette, and an Export dialog page can follow once #177's dialog lands.

anishguduru pushed a commit to anishguduru/deckcraft that referenced this pull request Oct 16, 2026
storytold#451 renamed conventions_resources_and_modern_results to
conventions_resources before merging, but the misc.mcp-resources evidence
kept the old name, so `cargo xtask parity` failed on main.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
anishguduru pushed a commit to anishguduru/deckcraft that referenced this pull request Oct 16, 2026
* M14: clarify Ukrainian certificate and accessibility wording

Distinguish the certificate holder from the document topic, translate
fixed certificate types, and clarify fingerprints, keyboard access,
table sections and equal field sizes. Validate the complete catalog
with VESUM 6.8.6 and review technical-word exceptions in context.

Fixes storytold#503
Follow-up to storytold#509

Co-authored-by: Codex <noreply@openai.com>

* M14: restore CI after merged test changes

Keep one byte-identical copy of each appearance regression test instead
of five definitions, preserving all three test bodies and assertions.
Correct MCP resource parity evidence to the existing passing test name.

Co-authored-by: Codex <noreply@openai.com>

* ROADMAP: drop the CI follow-up sentence (main fixed the duplicated tests and MCP evidence in storytold#605/storytold#608)

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
file.export writes .mp4/.mov: each slide's transition, then the slide
held for its "advance after" time (else secondsPerSlide), at a chosen
height (480p to 4K) and frame rate, hidden slides left out unless asked.

Frames come from the slide and transition renderers and are encoded by
a new Constrained Baseline H.264 encoder in deckcraft-h264 (CAVLC,
Intra 16x16 with V/H/DC prediction, unchanged macroblocks skipped,
deblocking off), checked bit-exact against our decoder, and muxed with
deckcraft-isobmff (faststart). A key frame starts every slide and
transition and at least every 2 s. No external tool is needed.

Co-authored-by: Cursor <cursoragent@cursor.com>

@FredApps FredApps left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReviewerA — source-read COMMENT (not an approval)

I read the head of the diff: export of the deck as MP4 and MOV video with "our own H.264 encoder". The lock gains deckcraft-h264 and deckcraft-isobmff as new workspace crates, wired into the engine, and file.export accepts mp4 and mov. Writing an H.264 encoder in Rust avoids patent-encumbered libraries as dependencies but not the patents themselves: H.264 is covered by the MPEG LA pool for distribution of encoded video, and for a free tool that ships an encoder the maintainers need to decide, which is a legal question and not a code one. The container writer (ISO BMFF) is straightforward. Encoding every slide of a long deck is slow, so progress and cancellation are needed, and the output size must be bounded. A test that decodes a result with a reference decoder is the proof. I have not run it, and no exact-head CI result exists for this head. Nothing was run.

I ran nothing and have no exact-head CI result for this PR, so this is not an approval. Where a diff was long I read its code hunks rather than every generated or documentation line.

Status: waiting_author.

@aiongg aiongg added the missing feature The reference application has it and we do not, or only part of it label Nov 4, 2026

@aiongg aiongg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-generated review (Opus). Not a maintainer decision.

Small fixes needed (not certain):

Adds MP4/MOV export through file.export with a built-in H.264 encoder (Intra 16x16 plus skipped unchanged macroblocks) and the existing MP4 writer. The bitstream syntax and reconstruction match the decoder, and the timeline and key-frame placement are clear.

  • apps/deckcraft-cli/src/main.rs:415: .mp4/.mov are now in WRITABLE_EXTENSIONS, so convert in.pptx out.mp4 passes the format check and goes to file.saveAs, which writes native deck bytes named out.mp4. Route mp4/mov to file.export here.
  • crates/engine/src/cmd/file.rs:261: save_bytes has no mp4/mov arm and falls through to the native format, so file.save/saveAs and run --save x.mp4 write a deck file under a video name. Return an error for these formats.
  • crates/engine/src/video.rs:65: render_transition blends from slides[index-1] (crates/render/src/transition.rs:73), so when that slide is hidden and left out of the video, its content still appears in the transition frames.
  • crates/engine/src/cmd/file.rs:557: height is clamped to 2–4320, not the 480–2160 range described. Below 16 the encoder errors, and above 4K the stream exceeds level 5.1.
  • crates/h264/src/encoder.rs:level_idc: anything past level 5.0 is labelled 5.1. That is wrong for 4K at 60 fps (needs 5.2) and for frames over 36,864 macroblocks (needs level 6).
  • crates/engine/src/video.rs:146: the doc comment on encode mentions path and qp parameters the function does not take.
  • crates/mcp/src/tools.rs:122: the MCP export tool's format enum does not list mp4/mov.
  • The export runs synchronously with no progress or cancellation, and the tests decode only with the in-repo decoder.

Claims checked:

  • verified: file.export writes video for format mp4|mov or a .mp4/.m4v/.mov path (crates/engine/src/cmd/file.rs:371-374 maps the extensions; file.rs:521 dispatches to video_export)
  • contradicted: Each slide's transition plays, then the slide is held for its advance-after time, else secondsPerSlide (default 5 s); hidden slides are left out unless includeHidden (crates/engine/src/video.rs:58-80 builds the timeline as described, but render_transition always blends from slides[index-1] (crates/…
  • contradicted: height defaults to 1080 and ranges from 480p to 4K; width follows the aspect, both even (crates/engine/src/cmd/file.rs:557 clamps height to 2..4320; crates/engine/src/video.rs:40-45 makes both sides even; crates/h264/src/encoder.rs Encoder::new rejects sides under 16)
  • verified: fps 1–60, default 30; the result reports bytes, frames, size and seconds (crates/engine/src/cmd/file.rs:558 clamps fps to 1..60; file.rs:565 returns path, bytes, frames, width, height, fps, seconds)
  • unverified: Encoder is Constrained Baseline, CAVLC, 4:2:0, one slice, deblocking off, Intra 16x16 V/H/DC by SAD with chroma DC, and unchanged macroblocks are skipped in P pictures (crates/h264/src/encoder.rs sps() writes profile 66 with constraint 0xC0; pps() sets entropy_coding_mode 0; encode_rgba writes disa…
  • contradicted: The SPS carries BT.709 VUI and timing, and the level comes from size and frame rate (crates/h264/src/encoder.rs sps() writes the BT.709 colour description and timing correctly, but level_idc falls back to _ => 51, which is wrong for 4K60 (needs 5.2) and for frames over 36,864 MBs (needs 6.x))
  • verified: The timeline is capped at 2 h at 30 fps (crates/engine/src/video.rs:11 MAX_FRAMES = 26060*30; timeline() stops at the cap (video.rs:67-76); at 60 fps the cap is 1 h)
  • unverified: A key frame starts every slide and every transition, and at least every 2 s (crates/engine/src/video.rs encode(): continues is true only for consecutive Transition steps of the same slide; start || since_key >= key_every with key_every = fps*2)
  • verified: The file is written through a temporary sibling (crates/engine/src/cmd/file.rs:563 calls write_file, which goes to crates/engine/src/io.rs:21-25 (unique temporary sibling, then rename))
  • verified: No external tools or new third-party crates; the engine only adds its own lower-layer crates (crates/engine/Cargo.toml:22-23 adds deckcraft-h264 and deckcraft-isobmff, which already exist as workspace crates (Cargo.toml:32,36) and are already used by crates/media/Cargo.toml:12,16)
  • contradicted: Only file.export and format_for_path change for the new formats (crates/engine/src/cmd/file.rs:340 adds mp4/m4v/mov to WRITABLE_EXTENSIONS, which apps/deckcraft-cli/src/main.rs:410-418 and 466-482 use to allow paths; mp4 then goes to file.saveAs, and save_bytes (file.rs:261-268) writes native deck…
  • verified: Docs: the MP4/MOV parity row is marked yes and the gaps entry names what is left (docs/file-format-parity.md:32 row is yes (its tests column still says none); docs/gaps.md:141-148 lists animations, audio and the dialog as left)

Recommendation: merge after the fixes above; they may be pushed to this branch.

Reviewed at 71c4178.

@aiongg aiongg self-assigned this Nov 5, 2026
@aiongg

aiongg commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Automated message. Not a maintainer decision.

This branch conflicts with main. Could you merge or rebase main into it? The conflict needs a person to resolve.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing feature The reference application has it and we do not, or only part of it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants