| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
feat(dsl): standardize the asset manifest and converge the export paths (#1007 part 3) (#1117) * feat(dsl): standardize the document asset manifest Add asset-manifest.ts to @openmaic/dsl: the canonical AssetManifestEntry shape (ref, kind, and byteSize/mimeType/duration/voice/prompt metadata where available) plus enumerateAssetManifest, the pure document-to-manifest enumeration. An entry's ref is the reference exactly as the document holds it -- the manifest is the id-based reference enumeration with metadata, not a content hash and not a resolution result. The traversal walks the stage whiteboard, each scene's canvas/whiteboards/speech actions, and the stage video-manifest keys in document order, with logical-owner reference counts that match the accounting duplication-safe replacement uses. This settles the media-ref + asset-manifest schema question (#779 open question 4) on the side the asset-pool RFC already implied: the schema is a function of the id semantics decided there. The type lives in the dsl rather than a new @openmaic/exporter package because the enumeration is pure over document types the dsl already owns (Stage/Scene/Slide/Action), so a separate package would add a published artifact and release-workflow surface without adding a capability; the storage contract comment now points at the module. Refs #1007 * refactor(export): drive the classroom ZIP from the asset manifest collectMediaFiles used to scan the whole mediaFiles table for the stage, so any row the document no longer references -- an orphan left by an edit or a superseded regeneration -- rode along into the archive. Both ZIP collectors now take their reference sets from the standardized asset manifest (buildStageAssetManifest wraps the dsl enumeration with the compatibility rows' metadata): only referenced assets are archived, and a referenced asset whose bytes exist only in the pool is still collected via a synthesized record. Byte resolution is unchanged: pool first through resolveStoredBytes / resolveAudioBlob, with the compatibility row kept as the legacy byte fallback and as the metadata source. mediaIndex is now a serialized view of the manifest, and the missing-audio report derives from the manifest's audio entries instead of a second action walk. The audioRef mapping and the legacy audioUrl fetch path (collectLegacyAudioForExport) are untouched. Refs #1007 * refactor(video-export): take the timeline's reference sets from the manifest createVideoTimelineDeps scanned the whole mediaFiles table for the stage and derived its audio id set from its own action walk -- a third, independent answer to "which media does this course use?". Both record loads now key off the standardized asset manifest: media rows are read per manifest ref by compound key instead of by table scan, and the audio id set is the manifest's audio entries. Orphan rows were never reachable through the scene-scoped elementId-to-mediaRef bridge; now they are not even read. The bridge itself is untouched: element ids recur across scenes, so the elementId-to-mediaRef mapping stays scoped per scene, and the legacy audioUrl fallback keeps its own action walk because a URL is not a manifest ref. AssetPlan remains the video IR's view of the same references. Refs #1007 * refactor(export): resolve PPTX media through the shared resolver only Each PPTX element branch carried its own resolution chain: a task-state renderable-URL lookup first, then -- gated on the legacy placeholder predicate -- a stored-bytes override, with the poster block repeating the pattern. One helper now owns resolution for backgrounds, images, video / audio sources, and posters: opaque refs (allocated ids and legacy placeholders alike, no placeholder-pattern gate) resolve pool-first through resolveStoredBytes and embed as data URLs, concrete addresses resolve through the media state machine and keep the caller's fetch path. exportMediaResolution and the resolveStoredMediaBlob wrapper fold into the helper; resolvePptxMediaBinding stays as the state-machine entry the resolution-surface test matrix drives. Refs #1007 * refactor(export): retire the export-side Dexie byte fallbacks Export call sites no longer read bytes off compatibility rows directly. The ZIP collectors and the video timeline's audio load resolve bytes only through the shared resolvers (resolveStoredBytes / resolveAudioBlob), which answer pool-first and keep the compatibility row as their internal legacy fallback level; the row reads that remain at the call sites supply metadata (format/duration/voice/mime/size/prompt) only. The rows themselves stay for legacy and regeneration readers -- what goes is the export paths' own fallback logic. One observable tightening: a failed media row (error set, empty placeholder blob) no longer ships a 0-byte file into the classroom ZIP, and an evicted row no longer ships its empty local blob; referenced-but- byteless assets are simply absent from the archive, as they already were when no row existed. Refs #1007 * test(media): cover the enriched stage asset manifest builder Pins the join between the pure dsl enumeration and the compatibility rows: metadata attaches by ref, rows no document reference names never appear, and a referenced asset with no row keeps a metadata-free entry. Refs #1007 * fix(video-export): widen the deps stage input for the manifest enumeration enumerateAssetManifest reads the stage's whiteboard and videoManifest, so createVideoTimelineDeps declares them on its input instead of the bare id; callers pass only the id today and the optional fields stay absent. Also applies the repo prettier formatting to the files this branch touched. Refs #1007 * fix(dsl): enumerate slide audio elements in the asset manifest Slide audio elements carry their own src, and the manifest skipped them, so a manifest-driven collector could never archive their bytes. The audio slot maps to kind 'audio' alongside narration ids. Refs #1007 * fix(media): harden ref-keyed lookups against prototype-named asset refs AssetRef is an unconstrained string alias, so a media reference can legitimately be "__proto__", "constructor", or any other Object.prototype member. Plain objects keyed by such refs silently drop assignments or answer lookups with the prototype object, which rewrite paths then accept as a mapped id. Convert the remaining ref-keyed lookup tables introduced by the export convergence to prototype-safe structures: the classroom import media/poster alias maps and the legacy-conversion video-manifest reconstruction now use Map / null-prototype containers with explicit membership checks, and every consumed value is validated as a string before it is written into a src / mediaRef / audioId slot. The shared media-task lookup receives the same treatment: one centralized own-property-checked lookupMediaTask now serves the stored-bytes resolver, the PPTX embeddable-src path, the video collection path, and the element/background task resolution, so a prototype-named placeholderRef can no longer hide a re-keyed task from the fallback chain. Adversarial tests drive "__proto__" and "constructor" refs through the import round trip, the PPTX fallback path, the legacy conversion commit path, and the media-task fallback end to end, including a buildPptxBlob regression with a task re-keyed to an allocated id while retaining a prototype-named placeholderRef. * fix(export): use safe archive asset paths * refactor(dsl): centralize slide media slot roles * refactor(export): derive consumer refs from manifest * fix(export): sanitize classroom archive extensions * fix(video-export): preserve narration speech order * fix(export): enforce kind-coherent archive media * fix(export): define media coherence boundary * fix(export): carry task-owned poster binding for PPTX export A video element with no explicit poster falls back to its media task's generated poster URL, but resolveVideoMediaForElement left posterTask undefined for that case, so the PPTX manifest guard saw a foreign URL with no task-ownership exemption and dropped the video element instead of using the established runtime poster fallback. Carry the poster task binding whenever the task poster is the effective poster: the task-owned URL then satisfies the guard's objectUrl exemption end to end. A concrete explicit element poster still stays element-owned and never borrows the binding, and the guard's foreign-ref rejection is preserved (and exported as a directly testable predicate). Coverage: an element with no poster plus a task-provided poster embeds the task poster as the PPTX cover (red at the pre-fix head, green now), and a genuinely unrelated URL with no task ownership is still rejected by the guard. * fix(export): preserve legacy narration source refs in the media index The explicit sourceRef contract was partial: primary audio and generated media entries carried it, but legacy URL narration serialized no source ref. The legacy URL itself is the natural source ref — it is known at fetch time — so wire it through the collected blob into the mediaIndex entry. Import already registers serialized sourceRefs as aliases, so the URL now round-trips as an explicit mapping instead of being reconstructed only from the action's audioRef. Poster siblings are deliberately NOT given their own mediaIndex entry: a sibling poster (media/asset-<n>.poster.<ext>) is a legacy byte copy written from the video record and is not an independently referenced document asset — when the poster is a real document asset it already has its own indexed entry with a sourceRef, and import reconstructs the sibling by path derivation from its parent video entry, reusing the poster's own indexed allocation when one exists. The PR description is narrowed to match; corrected paragraph: "Archive names never interpolate refs — sequential safe paths (media/asset-<n>.<ext>, audio/audio-<n>.<ext>) with the original ref preserved through an explicit sourceRef mapping on every independently indexed media entry: generated media assets, poster assets, primary narration, and legacy URL narration (the legacy URL itself is the entry's sourceRef). Extensions are allowlisted per kind. The one exception is the legacy sibling poster byte copy (media/asset-<n>.poster.<ext>, written next to its video when the video record still carries the pre-pool poster bytes): it is not an independently referenced document asset, so it has no mediaIndex entry or sourceRef of its own — its identity is derivable from its parent video entry (same index), and import reconstructs it by sibling-path derivation from that video entry, reusing the poster's own indexed allocation when one exists." --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> | 20 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
feat(i18n): add German localization (#1128) Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
fix(video-export): constrain GenUI iframe visibility (#1125) * fix(video-export): constrain interactive iframe visibility * fix(render-service): allow screenshot fallback in standard profile * fix(render-service): lower standard memory floor to 8 GiB * refactor(video-export): address review findings | 25 天前 | |
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): deterministic Quiz/PBL cover cards (#985) (#995) * feat(video-export): add deterministic Quiz/PBL cover cards Replace unsupported placeholders with authored-metadata cover cards that stay independent of learner progress, and pin dense-layout evidence for 720p/480p. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(video-export): add configurable localized cover CTA Replace the inert Start Quiz affordance with a destination-configurable CTA (NEXT_PUBLIC_VIDEO_EXPORT_CTA_DESTINATION), including Unicode/bidi display guards. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): require Hyperframes video-export gates Lock CI/Docker contracts so cover materialization and Hyperframes sample lint stay required for export changes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): move HF_E2E_DIR onto e2e steps GitHub rejects runner.* in job-level env, which aborted the CI workflow with zero jobs. Keep the sample dir on the materialize/lint steps only. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(video-export): tidy CI invoke and panel-box export Run the Hyperframes materialize step through pnpm exec vitest like the cover-card guardrail above it; the e2e job already pins Node 22. Stop re-exporting PBL_PANEL_DESIGN_BOX from the public barrel — only the layout browser test needs it, and it can import the emitter module directly. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 1 个月前 | |
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): VideoTimeline IR contract + pure compile pipeline (#864) (#913) * feat(video-export): VideoTimeline IR contract + pure compile pipeline (#864) First architecture slice of #854: make a VideoTimeline IR the system contract for classroom-video export, produced by pure compile passes with no FFmpeg / Chrome / DOM (live state enters only through injected TimingProbe / AssetSource). - ir.ts: zod-sourced VideoTimeline IR — envelope + per-scene base/narration/ effects/videos/markers; effects reference an animation descriptor id (spotlight.v1) rather than inlining params; diagnostics first-class. - deps.ts: synchronous TimingProbe / AssetSource DI boundary + CompilerScene. - geometry.ts: pure elementId -> percentage-geometry resolver. - passes/: normalize, probe, timeline (folds lib/choreography's resolveActionTimeline), geometry, assets (dedup + zip naming), emit. - compile.ts: compileVideoTimeline orchestrator; quiz/interactive/pbl scenes represented as markers + diagnostics, never silently dropped. - eslint.config.mjs: machine-enforced purity boundary on lib/video-export/** (allows @openmaic/dsl, zod, ../choreography, relatives; bans @/ + render backends). Tests: 41 golden/stub-driven specs (no browser/FFmpeg); choreography regression still passes. tsc + eslint clean; purity boundary verified to block @/ and react imports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address cross-review findings on #864 - Minor: play_video durationSource is now 'capped' (not 'stored') when a resolved duration exceeds MAX_VIDEO_WAIT_MS, so a consumer doesn't read the safety cap as the clip's true length. - Nit: distinguish a play_video with no associated media asset (null) from one whose referenced bytes are unavailable (present:false) — separate diagnostic messages, mirroring the audio missing-vs-skipped distinction. - Nit: reword the lib/video-export eslint allowlist messages to match the intended rule (relative paths within the module are permitted; the @/ and render-backend bans are the purity teeth), instead of claiming no parent-escape. - Tests: pin the previously-uncovered branches — video durationSource (stored / capped / over-cap / zero-policy), AssetPlanner unique() collision suffixing, and cross-elementId video dedup. 107 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): make asset presence authoritative per assetId (#864) Resolves the last cross-review finding (latent nit #5): if an AssetSource returned inconsistent `present` for the same assetId, a dedup entry could claim a different presence than its owner. AssetPlanner now records the first (owner) entry per assetId and every later reference — and the caller's stamped segment — inherits the owner's path AND presence, so the plan is internally consistent regardless of per-reference metadata. Presence is a property of the asset id, not of an individual reference. Test: a later present:true ref to an id first seen as absent inherits absent (no assetRef), matching the owner. 48 video-export specs pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address @wyuc CHANGES_REQUESTED review on #864 Make the IR sufficient as the standalone input to the P1d emitter (#865). P1: - VideoSegment now carries placement (geometry in 0-100 space + rotate degrees), resolved by the geometry pass, so the emitter positions the clip from the IR alone — no scene-DSL reread. Degrades (geometry null, rotate 0) with an unresolved-element diagnostic when the target element is missing. - Unavailable play_video is skipped, not turned into a 5-minute blank: video availability is resolved up front from the AssetSource and fed to the timeline so an unavailable clip gets a 0ms dwell (durationSource 'skipped'), leaving later actions unshifted — matching #854's skip+diagnostic behavior. - EffectSegment now carries effective `params`: descriptor defaults merged with authored overrides (spotlight dimOpacity -> dimness, laser color), so an IR-only emitter reproduces authored behavior instead of only defaults. P2: - Referenced-but-missing video is represented structurally (assetId + present on the segment AND an AssetPlanEntry), so downstream code distinguishes it from "no association" without parsing diagnostics. - Asset extensions are sanitized to a bare, traversal-free token before entering the zip path (a hostile format like '../../escape' can no longer steer paths). P3: - Tighten the lib/video-export eslint import allowlist: only ./…, single-level ../… within the module, and the explicit ../../choreography are allowed; a ../../ escape into the rest of the app is now rejected. Tests: +7 specs pinning each fix (placement, skip-without-shift, effect params, structural missing-video, extension sanitization). 115 pass; tsc/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address @wyuc re-review on #864 (2 remaining findings) P1 — video availability no longer conflated by action id. The DSL does not enforce stage-wide action-id uniqueness, so an id-keyed availability set let two scenes sharing a play_video id conflate: an absent-media clip could keep a 300000ms cap dwell while assets stamped it 'skipped', shifting the next scene by 5 minutes. resolveAvailableVideos now keys the set by the action OBJECT identity (resolveActionTimeline passes the same normalized action objects back by reference), so lookup is exact regardless of id collisions. P2 — purity lint no longer lets module-root files escape. A single '../…' means different things by depth: inside passes/ it stays in-module, but from a module-root file (compile.ts) '../action/engine' escapes. The boundary is now split into two disjoint file scopes — root ('lib/video-export/*') allowing only './…' + '../choreography', and passes ('lib/video-export/passes/**') allowing './…', a single in-module '../…', and '../../choreography'. Verified: root '../action/engine' and passes '../../action' both rejected; all real imports and '../choreography' pass. Tests: +1 regression pinning the duplicate-id case (absent clip skipped to 0ms, scene 2 starts at 0, not 300000). 116 pass; tsc/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 1 个月前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): capture self-contained static interactive HTML (#1086) * feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps | 28 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
fix(video-export): make Cyrillic and Arabic Quiz fonts deterministic (#1114) * feat(video-export): make Quiz script fonts deterministic * test(video-export): verify Arabic shaping visually * fix(video-export): cover Arabic extension characters --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 24 天前 | |
feat(video-export): fidelity polish — spotlight geometry, video clips, formulas, subtitles (#952) * fix(video-export): bridge play_video element id → media ref Generated-media records are keyed by the element's media ref (gen_vid_…), but a play_video action targets the slide element by its .id. The app-side compiler deps looked media/durations up by the raw element id, so every generated video missed → present:false → no <video> emitted, no bytes collected → the clip vanished from the export (only its static first frame, baked into the base PNG, remained). Build an elementId → mediaRef bridge across all slide scenes in createVideoTimelineDeps (mirroring the live engine's resolveMediaPlaceholderId) and resolve both media() and videoDurationMs() through it. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): re-measure KaTeX fit-scale after fonts settle KaTeX sizes large delimiters (\left\{, \begin{cases}) from its KaTeX_Size faces, which load asynchronously. KatexContent measured its shrink-to-fit scale once on mount, against the fallback font, and never recomputed — so a cold slide snapshot baked a stale scale and the big brace desynced from the piecewise body ("大括号后面的分段函数与大括号错位"). - Re-run the fit-scale measurement on document.fonts.ready and on each `loadingdone`, keeping the shrink-only (cap-at-1) behavior. - slideToPng waits two extra frames before capture so the font-triggered React re-measure commits into the DOM html2canvas reads. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): place spotlight/laser/video on the rendered content box The pure geometry helper resolves an element's authored outer box against a fixed 1000×562.5 base, but the live overlays — and the exported frame PNG — measure the element's `.element-content` box, which for horizontal text is auto-height plus 10px content padding. So a spotlight/laser sat offset from where the text actually renders ("spotlight 与对应元素的位置有偏差", #867 item 5). Add an optional GeometryProbe to the compiler's DI boundary: when supplied, the geometry pass prefers each element's measured content-box geometry (video keeps its rotation from the authored element), degrading to the pure authored-box calc on a miss so the compiler stays deterministic in unit contexts. App side: measureSlideElementGeometry (new, in @openmaic/renderer/snapshot) mounts the slide off-screen and reads the same `.element-content`-against- container box the live SpotlightOverlay uses; createVideoTimelineDeps pre-measures every spotlight/laser/play_video target per scene and serves the probe as a synchronous table lookup. Tests: geometry.test.ts quantifies the outer-box vs content-box delta and proves probe-wins / fallback / rotation-preservation; timeline-deps.test.ts covers the pre-measure wiring. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): decode a first-frame poster for posterless videos A play_video clip only shows during its play window; the rest of the time the static base-frame PNG backs that spot. But html2canvas can't draw a <video>, so the snapshot swaps it for its poster <img> — and generated videos usually carry no poster, leaving the video area blank whenever the clip isn't playing. When a video record has no stored poster bytes, decode the video blob's first frame (seek slightly off 0, draw to a canvas) and set element.poster before the base snapshot, so that spot shows the first frame — the "paused on frame one" look — instead of blank. Falls back to no poster on decode failure / timeout / CORS taint, never failing the export. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video-export): split subtitles, make burn-in optional, centered modal Burned-in captions used one cue per whole narration paragraph, so a long cue became a 4–6 line block covering the lower half of the slide, and there was no way to turn it off. - Split cues (pure, compiler layer): lib/video-export/split-cue.ts splits each cue on sentence → clause → hard-wrap to ≤~40 CJK-equiv units, distributing the parent window by character weight (last piece pinned to the parent end, sub-1.2s slivers merged). Wired into the timeline pass, so the burned-in overlay and the SRT/VTT sidecar share one split track and can't drift. - Burn-in optional (default off, #867 item 2): emitHyperframes gains burnInSubtitles; off → clean video + sidecar SRT/VTT only. Caption layout hard- clamped to 2 lines so an outlier can't grow tall again. - Download subtitles without rendering: compileSubtitles() runs only the compiler (no assets/zip/service); new useDownloadSubtitles saves an .srt. - Centered modal: video-export-dialog.tsx replaces the cramped dropdown section, grouping Output / Subtitles / Actions / Progress with a stable lifecycle for multi-minute renders. header-controls opens it from an "Export Video" entry; old video-export-menu.tsx removed. i18n keys added across all 8 locales. Refs #867. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: prettier format video-export fidelity changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: drop unused playVideo helper in timeline-deps test The media-bridge tests assert via deps.assets.media(...) directly, so the playVideo helper (and its PlayVideoAction import) were dead. Flagged by the code-quality bot on #952. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): resolve cross-review findings + lower subtitle band - emit-hyperframes: drop duplicate `display:-webkit-box` on subtitle cues so they start truly `display:none` (a second declaration overrode the `none`, showing every cue stacked at t=0 when burn-in was enabled). - geometry: don't feed the measured content-box AABB to a rotated video clip — it already encloses the rotation, so re-applying `rotate` doubled it. Rotated elements fall back to the authored box (one, un-doubled source of truth). - build-export-zip: extract shared `compileStageIr` so buildExportZip and compileSubtitles can't drift on the timing/assets/geometry wiring. - split-cue: join merged CJK cue pieces without a spurious ASCII space. - subtitles: lower the caption band (bottomRatio 0.055 → 0.01) and hoist the band's magic numbers into a named SUBTITLE constants block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): stop the snapshot onclone override from reshaping KaTeX The export snapshot injects a neutral-kerning / geometricPrecision style over `.slide-renderer-prose *` to keep CJK prose+table glyphs from mis-advancing under html2canvas-pro. KaTeX formulas (output:'html') also live inside `.slide-renderer-prose`, so the override was reshaping their math glyph advances — a burned-in formula rendered subtly differently from the live canvas, which applies no such override. Empirical check (Playwright + system Chrome, cases+fraction formula, native paint vs html2canvas): the override raised the formula's mean per-pixel delta vs the live paint from 120.1 to 139.2 (~16%) and moved ~75% of inked pixels. Restoring native font-kerning/text-rendering for `.katex` subtrees brings the formula's delta back to 120.1 (bit-identical to the no-override capture) while prose and table text keep the override. The residual 120.1 is the deeper html2canvas rasterization gap (vlist/frac-line/delimiter), a separate track. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): render slides via native paint, bundle KaTeX font embed Root-fix the exported-vs-live formula distortion (and the filter/mask/video gaps) by rasterizing slides with native Chrome paint instead of html2canvas's reimplementation. - slideToPng now captures via html-to-image (foreignObject → the same Chrome engine that paints the live classroom rasterizes the same DOM). KaTeX formulas, CSS filter, soft-edge masks, and mixed CJK/Latin text come out as the classroom shows them. html2canvas-pro stays as a fallback (with the filter/mask bakes + KaTeX text-rendering reset) for when native paint can't run — e.g. a cross-origin image taints the canvas. - A foreignObject SVG can't reach the document font registry, so fonts must be inlined. KaTeX math faces are prepended from the bundled woff2 (KATEX_FONT_EMBED_CSS, generated by scripts/generate-katex-fonts.mjs), NOT left to runtime getFontEmbedCSS — which reads cssRules and silently drops a cross-origin KaTeX stylesheet, collapsing a large brace to a fallback glyph (the observed "formula render failure" on e.g. a \begin{cases} with \text{}). A per-face guard verifies every KaTeX face the formula references is embedded, falling back to html2canvas otherwise. Verified in a Playwright spike (native vs html2canvas vs html-to-image): formula inked-delta 143→53; filter/mask/video match native; and with getFontEmbedCSS forced empty, the bundled embed alone still renders the failing formula correctly. Renderer package tests: 226 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): prettier-format the generated KaTeX font-embed module The generator emitted an unformatted TS file (double quotes, long unwrapped CSS literal) that failed CI `prettier --check`. Run the generated output through the repo's Prettier config before writing, so every rebuild produces CI-clean, stable output. Regenerated the committed file to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lint): rename reserved `module` var in katex-fonts generator; ignore .scratch - generate-katex-fonts.mjs assigned to a variable named `module`, tripping @next/next/no-assign-module-variable (the one hard eslint error). Renamed to `fileContents`; generated output is byte-identical (deterministic). - Add `.scratch/**` to the eslint globalIgnores (alongside .claude/.worktrees) so local throwaway verification scaffolding doesn't pollute `pnpm lint`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address PR #952 review — scene-scoped media bridge, safer cue splitting Resolves the two blocking correctness issues and three follow-ups from the cross-review on #952. - Scope the elementId→mediaRef bridge by scene (was deck-wide/last-writer-wins): slide-local ids like `video_001` now resolve within their own scene. `assets.media` uses its scene arg; `timing.videoDurationMs` resolves via action object identity. Adds a cross-scene collision test. - Stop treating every ASCII period as a sentence boundary: a lone `.` splits only when not between word chars (3.14, v1.2) and not an abbreviation dot (e.g.), collapsing repeated-punctuation runs. Adds decimal/version/abbrev/ellipsis tests. - Share the module-level in-flight guard across ZIP export and subtitle download (extracted to export-in-flight.ts); dialog `busy` folds in `downloading`. - Skip the off-screen geometry render for the subtitles-only path (skipGeometry); duration probes are kept so sidecar cue timing stays in sync with the video. - Align measure.ts font settling with slideToPng: force-load each used face before fonts.ready so measured content-boxes don't bake fallback advance widths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(video-export): prettier-format split-cue test Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(video-export): enable beginFrame capture via PRODUCER_BROWSER_GPU_MODE=hardware The producer defaults to `software` GPU mode, which silently force-enables the CPU-bound `Page.captureScreenshot` fallback: SwiftShader pins every core on rasterization and caps frame capture at ~10 fps. Selecting `hardware` keeps Chromium's `HeadlessExperimental.beginFrame` capture active instead. Measured on a 900-frame (30s) clip, no real GPU on the host: - capture ~10 -> ~19 fps, total 100s -> 62s (~38% faster) - CPU peak ~810% -> ~98%; peak mem 2.2 GiB -> 0.9 GiB - static frames pixel-identical; animation frames differ only in sub-pixel edge antialiasing (same-path re-render is deterministic/inf PSNR) No real GPU is required — the win is the faster capture API, not hardware rasterization; beginFrame falls back gracefully if the host lacks a GPU. Also eases the RENDER_JOB_DEADLINE_MS timeout risk on long videos. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(video-export): sentence splitting via Intl.Segmenter + abbreviation merge Addresses the PR #952 follow-up: separate sentence-boundary detection from cue shaping and stop growing a custom punctuation heuristic. - splitSentences now delegates boundary detection to Intl.Segmenter (locale- agnostic ICU), the multilingual/CJK baseline. It already preserves 3.14, v1.2, e.g., U.S., ellipses, and CJK 。！？… without any hand-rolled period scan. - ICU still ends a sentence after a titlecase abbreviation before a capitalized word (Dr. Smith, Fig. 3) — the edge case the reviewer flagged. A small abbreviation-aware post-merge re-joins those, driven by an ABBREVIATIONS set. - Semicolons are no longer sentence ends (they join clauses); moved ；; into the clause-level SECONDARY splitter so over-budget sentences still break there. - Cue shaping (budget / clause / hard-wrap / weighted timing) is unchanged, so detection and shaping are now distinct stages. Not using `sbd` as suggested: lib/video-export/*.ts sits behind the machine- enforced eslint purity boundary (imports limited to @openmaic/dsl, zod, ../choreography, siblings), so a new npm dep is rejected. Intl.Segmenter is a pure Node >=20.9 builtin, matching the reviewer's "small abbreviation-aware post-merge layer" alternative. Tests: add Dr. Smith / Fig. 3 / U.S. coverage; the rhetorical ellipsis now stays one under-budget cue (was split); add a semicolon over-budget case. 22/22 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(video-export): prettier-format split-cue sentence refactor Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(es-MX): add 7 subtitle/video export keys to align with en-US main added the Spanish (Mexico) locale (#942) after this branch's subtitle keys landed, so es-MX was the only locale missing export.subtitles* and export.videoBurnSubtitles* — which failed the i18n key-alignment CI check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): restrict title post-merge so it never deletes real boundaries Addresses the blocking re-review on the Intl.Segmenter change: the abbreviation post-merge unconditionally rejoined the next ICU segment whenever the previous one ended in a listed abbreviation, which erased genuine sentence boundaries — `It was made by Acme Inc. Next point.` collapsed into one cue, and even an explicit newline (`Acme Inc.\nNext topic`) was rejoined. The merge now fires only for the case ICU actually mis-splits — a title (Mr/Mrs/Ms/Dr/Prof/Sr/Jr) before a capitalized proper noun — and only when: - the previous segment does not end at a newline (hard boundary), and - the next segment starts with a capital letter. Sentence-trailing abbreviations (Inc./Ltd./etc./vs./No.) are dropped from the set entirely: they legitimately end sentences, and ICU already keeps a title before a number (No. 5, Fig. 3) or a lowercase continuation (etc. and…) whole without any merge, so nothing there needs rejoining. Regression tests: Inc./etc. boundary preserved, newline-after-title preserved, lowercase continuation kept whole, plus the retained Dr. Smith / Fig. 3 / U.S. cases. 26 split-cue tests pass; full video-export suite 123 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | 1 个月前 | |
feat(video-export): L1 Hyperframes emitter + browser collection + export ZIP (#865) (#931) * feat(video-export): L1 Hyperframes emitter + browser collection + export ZIP (#865) Consume the VideoTimeline IR (#864/#913) and produce a self-contained Hyperframes composition project that `npx hyperframes render` turns into an MP4 (render execution itself is #866). - lib/video-export/emit-hyperframes: pure IR → Hyperframes project emitter (one flat composition, one paused GSAP timeline on window.__timelines, spotlight/laser overlays from lib/choreography descriptors). Stays under the purity boundary; emits HTML/JS strings only. - lib/video-export/subtitles: SRT/VTT serialization of the IR subtitle track. - lib/video-export-app: impure app glue (DI adapters over Dexie, slide-snapshot collection lifted from #849, streaming ZIP packaging, useExportVideo hook). - Vendor GSAP locally (public/vendor/gsap.min.js) — determinism red-line. - Export menu entry with resolution select + i18n across all 8 locales. Verified end-to-end against the real hyperframes CLI: lint passes 0 errors on a real classroom export, and a rendered slice produces a valid H.264+AAC MP4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video-export): gate Export Video menu behind NEXT_PUBLIC_ENABLE_VIDEO_EXPORT flag The video export UI is experimental until the render pipeline (#866) lands, so hide the "Export Video" affordance behind an off-by-default feature flag. - Add isVideoExportEnabled() to lib/config/feature-flags (NEXT_PUBLIC_, so it inlines at build time for the client-side export menu). - Gate the Export Video block in header-controls on the flag. - Cover the new flag in tests/config/feature-flags.test.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): ossKey fallback for evicted blobs + emitter dedup Address code-review findings on #931. - collect: resolveBytes() prefers the local Dexie blob and falls back to the record's CDN ossKey (audio) / ossKey+posterOssKey (media/poster) so a live-mode classroom whose local blobs were LRU-evicted still exports a self-contained ZIP. timeline-deps presence checks widened to `blob.size > 0 || !!ossKey` so the compiler keeps these entries instead of marking them absent; video-duration probe stays local-only. - emit-hyperframes: extract one shared escapeHtml + sec into format.ts, dropping the two divergent escapers (attr vs escapeHtml) and the two sec() definitions with different rounding, plus the middle-man escapeAttr. Output is byte-identical (snapshot unchanged). - Add tests/video-export/collect.test.ts (ossKey fallback, local-first, fetch 404/throw, video+poster). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): restore evicted generated media into base-frame snapshots Address review blocker (P1) on #931: the ossKey fallback covered standalone audio/video/image/poster asset entries but not the slide base-frame path. resolveGeneratedMedia only accepted a non-empty local blob, so a live-mode record with an evicted blob + valid ossKey had its image src cleared (and video/poster not restored) before slideToPng snapshotted the slide — the frame PNG lost the generated media even though the standalone asset was fetched. - resolveGeneratedMedia now resolves bytes via resolveBytes (local blob first, then ossKey / posterOssKey) before creating the snapshot object URLs. - Add frame-collection tests: evicted image restored via ossKey, evicted video+poster restored, and image with no ossKey still cleared. The renderer snapshot is mocked so the frame path runs in plain Node. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 1 个月前 | |
feat(dsl): standardize the asset manifest and converge the export paths (#1007 part 3) (#1117) * feat(dsl): standardize the document asset manifest Add asset-manifest.ts to @openmaic/dsl: the canonical AssetManifestEntry shape (ref, kind, and byteSize/mimeType/duration/voice/prompt metadata where available) plus enumerateAssetManifest, the pure document-to-manifest enumeration. An entry's ref is the reference exactly as the document holds it -- the manifest is the id-based reference enumeration with metadata, not a content hash and not a resolution result. The traversal walks the stage whiteboard, each scene's canvas/whiteboards/speech actions, and the stage video-manifest keys in document order, with logical-owner reference counts that match the accounting duplication-safe replacement uses. This settles the media-ref + asset-manifest schema question (#779 open question 4) on the side the asset-pool RFC already implied: the schema is a function of the id semantics decided there. The type lives in the dsl rather than a new @openmaic/exporter package because the enumeration is pure over document types the dsl already owns (Stage/Scene/Slide/Action), so a separate package would add a published artifact and release-workflow surface without adding a capability; the storage contract comment now points at the module. Refs #1007 * refactor(export): drive the classroom ZIP from the asset manifest collectMediaFiles used to scan the whole mediaFiles table for the stage, so any row the document no longer references -- an orphan left by an edit or a superseded regeneration -- rode along into the archive. Both ZIP collectors now take their reference sets from the standardized asset manifest (buildStageAssetManifest wraps the dsl enumeration with the compatibility rows' metadata): only referenced assets are archived, and a referenced asset whose bytes exist only in the pool is still collected via a synthesized record. Byte resolution is unchanged: pool first through resolveStoredBytes / resolveAudioBlob, with the compatibility row kept as the legacy byte fallback and as the metadata source. mediaIndex is now a serialized view of the manifest, and the missing-audio report derives from the manifest's audio entries instead of a second action walk. The audioRef mapping and the legacy audioUrl fetch path (collectLegacyAudioForExport) are untouched. Refs #1007 * refactor(video-export): take the timeline's reference sets from the manifest createVideoTimelineDeps scanned the whole mediaFiles table for the stage and derived its audio id set from its own action walk -- a third, independent answer to "which media does this course use?". Both record loads now key off the standardized asset manifest: media rows are read per manifest ref by compound key instead of by table scan, and the audio id set is the manifest's audio entries. Orphan rows were never reachable through the scene-scoped elementId-to-mediaRef bridge; now they are not even read. The bridge itself is untouched: element ids recur across scenes, so the elementId-to-mediaRef mapping stays scoped per scene, and the legacy audioUrl fallback keeps its own action walk because a URL is not a manifest ref. AssetPlan remains the video IR's view of the same references. Refs #1007 * refactor(export): resolve PPTX media through the shared resolver only Each PPTX element branch carried its own resolution chain: a task-state renderable-URL lookup first, then -- gated on the legacy placeholder predicate -- a stored-bytes override, with the poster block repeating the pattern. One helper now owns resolution for backgrounds, images, video / audio sources, and posters: opaque refs (allocated ids and legacy placeholders alike, no placeholder-pattern gate) resolve pool-first through resolveStoredBytes and embed as data URLs, concrete addresses resolve through the media state machine and keep the caller's fetch path. exportMediaResolution and the resolveStoredMediaBlob wrapper fold into the helper; resolvePptxMediaBinding stays as the state-machine entry the resolution-surface test matrix drives. Refs #1007 * refactor(export): retire the export-side Dexie byte fallbacks Export call sites no longer read bytes off compatibility rows directly. The ZIP collectors and the video timeline's audio load resolve bytes only through the shared resolvers (resolveStoredBytes / resolveAudioBlob), which answer pool-first and keep the compatibility row as their internal legacy fallback level; the row reads that remain at the call sites supply metadata (format/duration/voice/mime/size/prompt) only. The rows themselves stay for legacy and regeneration readers -- what goes is the export paths' own fallback logic. One observable tightening: a failed media row (error set, empty placeholder blob) no longer ships a 0-byte file into the classroom ZIP, and an evicted row no longer ships its empty local blob; referenced-but- byteless assets are simply absent from the archive, as they already were when no row existed. Refs #1007 * test(media): cover the enriched stage asset manifest builder Pins the join between the pure dsl enumeration and the compatibility rows: metadata attaches by ref, rows no document reference names never appear, and a referenced asset with no row keeps a metadata-free entry. Refs #1007 * fix(video-export): widen the deps stage input for the manifest enumeration enumerateAssetManifest reads the stage's whiteboard and videoManifest, so createVideoTimelineDeps declares them on its input instead of the bare id; callers pass only the id today and the optional fields stay absent. Also applies the repo prettier formatting to the files this branch touched. Refs #1007 * fix(dsl): enumerate slide audio elements in the asset manifest Slide audio elements carry their own src, and the manifest skipped them, so a manifest-driven collector could never archive their bytes. The audio slot maps to kind 'audio' alongside narration ids. Refs #1007 * fix(media): harden ref-keyed lookups against prototype-named asset refs AssetRef is an unconstrained string alias, so a media reference can legitimately be "__proto__", "constructor", or any other Object.prototype member. Plain objects keyed by such refs silently drop assignments or answer lookups with the prototype object, which rewrite paths then accept as a mapped id. Convert the remaining ref-keyed lookup tables introduced by the export convergence to prototype-safe structures: the classroom import media/poster alias maps and the legacy-conversion video-manifest reconstruction now use Map / null-prototype containers with explicit membership checks, and every consumed value is validated as a string before it is written into a src / mediaRef / audioId slot. The shared media-task lookup receives the same treatment: one centralized own-property-checked lookupMediaTask now serves the stored-bytes resolver, the PPTX embeddable-src path, the video collection path, and the element/background task resolution, so a prototype-named placeholderRef can no longer hide a re-keyed task from the fallback chain. Adversarial tests drive "__proto__" and "constructor" refs through the import round trip, the PPTX fallback path, the legacy conversion commit path, and the media-task fallback end to end, including a buildPptxBlob regression with a task re-keyed to an allocated id while retaining a prototype-named placeholderRef. * fix(export): use safe archive asset paths * refactor(dsl): centralize slide media slot roles * refactor(export): derive consumer refs from manifest * fix(export): sanitize classroom archive extensions * fix(video-export): preserve narration speech order * fix(export): enforce kind-coherent archive media * fix(export): define media coherence boundary * fix(export): carry task-owned poster binding for PPTX export A video element with no explicit poster falls back to its media task's generated poster URL, but resolveVideoMediaForElement left posterTask undefined for that case, so the PPTX manifest guard saw a foreign URL with no task-ownership exemption and dropped the video element instead of using the established runtime poster fallback. Carry the poster task binding whenever the task poster is the effective poster: the task-owned URL then satisfies the guard's objectUrl exemption end to end. A concrete explicit element poster still stays element-owned and never borrows the binding, and the guard's foreign-ref rejection is preserved (and exported as a directly testable predicate). Coverage: an element with no poster plus a task-provided poster embeds the task poster as the PPTX cover (red at the pre-fix head, green now), and a genuinely unrelated URL with no task ownership is still rejected by the guard. * fix(export): preserve legacy narration source refs in the media index The explicit sourceRef contract was partial: primary audio and generated media entries carried it, but legacy URL narration serialized no source ref. The legacy URL itself is the natural source ref — it is known at fetch time — so wire it through the collected blob into the mediaIndex entry. Import already registers serialized sourceRefs as aliases, so the URL now round-trips as an explicit mapping instead of being reconstructed only from the action's audioRef. Poster siblings are deliberately NOT given their own mediaIndex entry: a sibling poster (media/asset-<n>.poster.<ext>) is a legacy byte copy written from the video record and is not an independently referenced document asset — when the poster is a real document asset it already has its own indexed entry with a sourceRef, and import reconstructs the sibling by path derivation from its parent video entry, reusing the poster's own indexed allocation when one exists. The PR description is narrowed to match; corrected paragraph: "Archive names never interpolate refs — sequential safe paths (media/asset-<n>.<ext>, audio/audio-<n>.<ext>) with the original ref preserved through an explicit sourceRef mapping on every independently indexed media entry: generated media assets, poster assets, primary narration, and legacy URL narration (the legacy URL itself is the entry's sourceRef). Extensions are allowlisted per kind. The one exception is the legacy sibling poster byte copy (media/asset-<n>.poster.<ext>, written next to its video when the video record still carries the pre-pool poster bytes): it is not an independently referenced document asset, so it has no mediaIndex entry or sourceRef of its own — its identity is derivable from its parent video entry (same index), and import reconstructs it by sibling-path derivation from that video entry, reusing the poster's own indexed allocation when one exists." --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> | 20 天前 | |
feat(video-export): VideoTimeline IR contract + pure compile pipeline (#864) (#913) * feat(video-export): VideoTimeline IR contract + pure compile pipeline (#864) First architecture slice of #854: make a VideoTimeline IR the system contract for classroom-video export, produced by pure compile passes with no FFmpeg / Chrome / DOM (live state enters only through injected TimingProbe / AssetSource). - ir.ts: zod-sourced VideoTimeline IR — envelope + per-scene base/narration/ effects/videos/markers; effects reference an animation descriptor id (spotlight.v1) rather than inlining params; diagnostics first-class. - deps.ts: synchronous TimingProbe / AssetSource DI boundary + CompilerScene. - geometry.ts: pure elementId -> percentage-geometry resolver. - passes/: normalize, probe, timeline (folds lib/choreography's resolveActionTimeline), geometry, assets (dedup + zip naming), emit. - compile.ts: compileVideoTimeline orchestrator; quiz/interactive/pbl scenes represented as markers + diagnostics, never silently dropped. - eslint.config.mjs: machine-enforced purity boundary on lib/video-export/** (allows @openmaic/dsl, zod, ../choreography, relatives; bans @/ + render backends). Tests: 41 golden/stub-driven specs (no browser/FFmpeg); choreography regression still passes. tsc + eslint clean; purity boundary verified to block @/ and react imports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address cross-review findings on #864 - Minor: play_video durationSource is now 'capped' (not 'stored') when a resolved duration exceeds MAX_VIDEO_WAIT_MS, so a consumer doesn't read the safety cap as the clip's true length. - Nit: distinguish a play_video with no associated media asset (null) from one whose referenced bytes are unavailable (present:false) — separate diagnostic messages, mirroring the audio missing-vs-skipped distinction. - Nit: reword the lib/video-export eslint allowlist messages to match the intended rule (relative paths within the module are permitted; the @/ and render-backend bans are the purity teeth), instead of claiming no parent-escape. - Tests: pin the previously-uncovered branches — video durationSource (stored / capped / over-cap / zero-policy), AssetPlanner unique() collision suffixing, and cross-elementId video dedup. 107 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): make asset presence authoritative per assetId (#864) Resolves the last cross-review finding (latent nit #5): if an AssetSource returned inconsistent `present` for the same assetId, a dedup entry could claim a different presence than its owner. AssetPlanner now records the first (owner) entry per assetId and every later reference — and the caller's stamped segment — inherits the owner's path AND presence, so the plan is internally consistent regardless of per-reference metadata. Presence is a property of the asset id, not of an individual reference. Test: a later present:true ref to an id first seen as absent inherits absent (no assetRef), matching the owner. 48 video-export specs pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address @wyuc CHANGES_REQUESTED review on #864 Make the IR sufficient as the standalone input to the P1d emitter (#865). P1: - VideoSegment now carries placement (geometry in 0-100 space + rotate degrees), resolved by the geometry pass, so the emitter positions the clip from the IR alone — no scene-DSL reread. Degrades (geometry null, rotate 0) with an unresolved-element diagnostic when the target element is missing. - Unavailable play_video is skipped, not turned into a 5-minute blank: video availability is resolved up front from the AssetSource and fed to the timeline so an unavailable clip gets a 0ms dwell (durationSource 'skipped'), leaving later actions unshifted — matching #854's skip+diagnostic behavior. - EffectSegment now carries effective `params`: descriptor defaults merged with authored overrides (spotlight dimOpacity -> dimness, laser color), so an IR-only emitter reproduces authored behavior instead of only defaults. P2: - Referenced-but-missing video is represented structurally (assetId + present on the segment AND an AssetPlanEntry), so downstream code distinguishes it from "no association" without parsing diagnostics. - Asset extensions are sanitized to a bare, traversal-free token before entering the zip path (a hostile format like '../../escape' can no longer steer paths). P3: - Tighten the lib/video-export eslint import allowlist: only ./…, single-level ../… within the module, and the explicit ../../choreography are allowed; a ../../ escape into the rest of the app is now rejected. Tests: +7 specs pinning each fix (placement, skip-without-shift, effect params, structural missing-video, extension sanitization). 115 pass; tsc/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video-export): address @wyuc re-review on #864 (2 remaining findings) P1 — video availability no longer conflated by action id. The DSL does not enforce stage-wide action-id uniqueness, so an id-keyed availability set let two scenes sharing a play_video id conflate: an absent-media clip could keep a 300000ms cap dwell while assets stamped it 'skipped', shifting the next scene by 5 minutes. resolveAvailableVideos now keys the set by the action OBJECT identity (resolveActionTimeline passes the same normalized action objects back by reference), so lookup is exact regardless of id collisions. P2 — purity lint no longer lets module-root files escape. A single '../…' means different things by depth: inside passes/ it stays in-module, but from a module-root file (compile.ts) '../action/engine' escapes. The boundary is now split into two disjoint file scopes — root ('lib/video-export/*') allowing only './…' + '../choreography', and passes ('lib/video-export/passes/**') allowing './…', a single in-module '../…', and '../../choreography'. Verified: root '../action/engine' and passes '../../action' both rejected; all real imports and '../choreography' pass. Tests: +1 regression pinning the duplicate-id case (absent clip skipped to 0ms, scene 2 starts at 0, not 300000). 116 pass; tsc/eslint/prettier clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> | 1 个月前 | |
feat(video-export): add deterministic Quiz question-list scrolling (#1102) * feat(video-export): add deterministic Quiz question-list scrolling * fix(video-export): bound Quiz layout measurement | 28 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 28 天前 | ||
| 20 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 24 天前 | ||
| 24 天前 | ||
| 24 天前 | ||
| 25 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 1 个月前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 1 个月前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 24 天前 | ||
| 24 天前 | ||
| 28 天前 | ||
| 28 天前 | ||
| 24 天前 | ||
| 24 天前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 20 天前 | ||
| 1 个月前 | ||
| 28 天前 |