| Files | Last commit | Last update |
|---|---|---|
docs: eval 23b at 1.25.0 — karaoke-style word + faststart-fix coverage (#280) Continues eval 23 (closed partial at 1.19.3, evals/results/iteration-23-partial.json) with real-execution prompts for the two features shipped since: caption.py --karaoke-style word (#276) and the +faststart fixes (#275/#277/#279). - evals/agent_prompts_24.json: two new prompts, kw1 (karaoke-style word, phrased after the #276 reporter's own complaint that the highlight "just changed colour") and ff1 (a YouTube template delivery through the loudness-normalize path, probing for a faststart regression). - Both run for real (actual ffmpeg encodes, not simulated) and graded with the repo's own evals/grade_runs_24.py: routing 100%, report format 2/2, disclosure 2/2, real execution 2/2. kw1's generated ASS carries \fscx112\fscy112 on the active word (genuinely bigger, not just recoloured); ff1's output has moov before mdat after the loudnorm re-encode and check.py's loudness/true-peak rows PASS. No regression found. - docs/roadmap.md: corrects the stale "released version today" line (1.24.1 -> 1.25.0, which is only a chore: faststart-path bump, #279), adds table rows for 1.24.1/1.25.0, and a narrative section for eval 23b. - The Opus-focused-grader and 55-prompt trigger-judge passes are NOT run here: this sandbox has no independent/non-Claude model access, the same blocker iteration-23-partial already documented (see also docs/design-decisions.md's "The evaluation is Claude-only"). Documented as an open gap, not faked. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 1 month ago | |
docs(evals): iteration 24 at 2.0.0 — the four 2.0 changes, 5/5 real runs (#284) Five Sonnet agents ran the 2.0 surface for real (overwrite refusal x2, --quality for a "CRF" request, BT.2020 SDR vs HDR, HDR->SDR), graded from each run's command log and md5 checks of the pre-existing files, not from the agents' summaries. 5/5 pass; h1 answered "not HDR" correctly but called a BT.2020 SDR file safe for a BT.709-only client, so references/gotchas.md now says color.py --to-sdr comes first (export.py only retags). Two follow-ups for 2.1.0 recorded in the results file. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 1 month ago | |
FFmpeg 8 capability detection; audio extraction, audio join, sample-accurate trims, typed dynamics (0.9.1) (#20) doctor / contract: filter rows recognised by their io-spec token, so the two-character flag column of FFmpeg 8+ no longer hides every filter; unreadable listings yield `unknown`, never `missing`; fixtures captured from FFmpeg 6.1.1, 8.1.2 and 9.0.1. audio.py: audio extraction, --audio-stream, typed --compress/--limit/--gate. join.py: audio-only inputs joined as audio. cut.py: codec from the output extension, sample-accurate --accurate, precision reported. CI on pull_request with listing artifacts. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs | 2 months ago | |
Audio-only inputs: SKILL.md section, agent/trigger evals, grader checks, test (#16) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
Machine-readable execution contract: contract --json, doctor, ToolSpecs, contract tests (#18) - scripts/_contract.py: contract_version 1.0, one ToolSpec per public script (ffmpeg-skill/<name>) with input schema generated from the argparse parser, output schema, role, required/optional ffmpeg capabilities, dry-run semantics (measured), mutates_input=false, verification and visual-check policy - bin/install.js: contract / doctor subcommands; package.json shipped with the skill for the version - mcp/server.py: batch tool added so tools/list == contract tools - _common: --json results carry status; failures under --json print {status: failed, error} - caption/report: --dry-run writes nothing - tests/test_contract.py, evals/contract/, docs/contract.md, release-check extended; version 0.9.0 Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 10 at 1.10.0, README catches up with 1.8-1.10, PR checklist names the README (#210) * docs: eval iteration 10 at 1.10.0 (108 runs, focused independent grader, trigger set, corpus report) and README row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: README catches up with 1.8-1.10 (--codec/--quality, time grammar, hdr_signal, export --normalize, deprecated list, NO_OVERWRITE) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: PR checklist and process note: a feat PR updates the README too Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
fix(evals): f05-unsupported taught the wrong answer -- reverse.py exists (#83) evals/agent_prompts_exec.json's f05-unsupported claimed "no reverse tool in the skill" and scored refusing a reverse request as correct -- scripts/reverse.py has existed the whole time and SKILL.md's own Request->script table routes "reverse this clip" to it. The eval also endorsed "a hand-written ffmpeg reverse... if run and verified" as an acceptable fallback, directly contradicting SKILL.md's and agent_prompts_24.json's explicit "never fall back to a raw ffmpeg invocation, even when asked directly" policy. - Replaced f05-unsupported with a genuinely unsupported case (a custom -filter_complex effects chain no script exposes), reusing the same scenario agent_prompts_24.json already uses consistently, so the two eval files no longer disagree with each other or with SKILL.md. - Added x06-reverse as a normal success case for the real reverse.py path, verified against a real fixture (reverse.py exits 0, JSON reports a valid probed output). - Left evals/results/exec-1.json's historical record unedited (it's a record of a run, not a living spec) but added a "superseded" note explaining f05's old grading was wrong and pointing at this fix, rather than silently rewriting a past result. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
0.8.1: skill craft — restructured SKILL.md, agent-run evals, fixes found in transcripts (#11) * SKILL.md restructure: trigger-focused description, workflow + decisions + report format; script reference and device notes moved to references/ - description states when to trigger, not just what exists - body 402 -> 169 lines; per-script CLI reference in references/scripts.md, real-device notes in references/devices.md (installed and packaged) - new sections: what to ask vs assume, report format, things that look right but are wrong - check.py: untagged 8-bit video passes the colour check (players assume BT.709); agents were adding a pointless --retag step - look.py: --at frames are stamped with the requested time, not the post-seek 0 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * SKILL.md: do not normalise near-silent ambience to speech targets Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 0.8.1: skill-craft release — agent-run evals, check step reworded, eval artifacts in evals/ Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 10 at 1.10.0, README catches up with 1.8-1.10, PR checklist names the README (#210) * docs: eval iteration 10 at 1.10.0 (108 runs, focused independent grader, trigger set, corpus report) and README row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: README catches up with 1.8-1.10 (--codec/--quality, time grammar, hdr_signal, export --normalize, deprecated list, NO_OVERWRITE) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: PR checklist and process note: a feat PR updates the README too Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 10 at 1.10.0, README catches up with 1.8-1.10, PR checklist names the README (#210) * docs: eval iteration 10 at 1.10.0 (108 runs, focused independent grader, trigger set, corpus report) and README row Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: README catches up with 1.8-1.10 (--codec/--quality, time grammar, hdr_signal, export --normalize, deprecated list, NO_OVERWRITE) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: PR checklist and process note: a feat PR updates the README too Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 11 at 1.11.0 (36 prompts, tokens per run measured) (#214) * docs: eval iteration 11 at 1.11.0 (36 prompts, tokens per run measured) Quality held at iteration-10 levels: routing 36/36, honesty 36/36, language 36/36, format 36/36, visual check 8/8, trigger 22/22, independent grader 4.71 mean. Tokens did not fall: 72.2k mean per run against 68.7k at 1.10.0, because the new SKILL.md intro and step 0 sent agents to scripts.md (36/36), gotchas.md (31/36) and doctor (31/36) before every job. Per-prompt table and the transcript counts are in iteration-11.json; the 1.11.1 fix PR rewords SKILL.md and iteration 12 re-measures. Also widens the regex grader's refusal-honesty keywords (two false negatives: "has no translation feature", "no script in this skill exposes") and adds the trigger-set result and the independent grades. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: trigger-set result for iteration 11 (22/22) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 12 at 1.11.1 (36 prompts); correct iteration 11's transcript counts (#216) Quality held: routing 36/36, honesty 36/36, language 36/36, format 36/36, visual check 8/8, trigger 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (doctor before a job 23/36 -> 0/36, --json-brief 4/36 -> 23/36, tool uses 7.9 -> 7.4 per run) and tokens per run stayed flat at 71.8k (72.2k at 1.11.0, 68.7k at 1.10.0). Correction: iteration 11's claim that agents read references/gotchas.md (31/36) and scripts.md (36/36) was a grep artefact; those names appear in the SKILL.md text every transcript echoes. Actual reads: 0/36 and 2/36. The per-run cost is dominated by a ~64k floor the harness sets, so the token-diet theme closes with this iteration. Corrected in iteration-11.json (correction field), CHANGELOG (Unreleased), the README results row and the roadmap line. Also notes that a stale ~/.claude/skills copy of the skill was run by 3-4 agents per iteration; it is now a symlink to the repo. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 12 at 1.11.1 (36 prompts); correct iteration 11's transcript counts (#216) Quality held: routing 36/36, honesty 36/36, language 36/36, format 36/36, visual check 8/8, trigger 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (doctor before a job 23/36 -> 0/36, --json-brief 4/36 -> 23/36, tool uses 7.9 -> 7.4 per run) and tokens per run stayed flat at 71.8k (72.2k at 1.11.0, 68.7k at 1.10.0). Correction: iteration 11's claim that agents read references/gotchas.md (31/36) and scripts.md (36/36) was a grep artefact; those names appear in the SKILL.md text every transcript echoes. Actual reads: 0/36 and 2/36. The per-run cost is dominated by a ~64k floor the harness sets, so the token-diet theme closes with this iteration. Corrected in iteration-11.json (correction field), CHANGELOG (Unreleased), the README results row and the roadmap line. Also notes that a stale ~/.claude/skills copy of the skill was run by 3-4 agents per iteration; it is now a symlink to the repo. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 12 at 1.11.1 (36 prompts); correct iteration 11's transcript counts (#216) Quality held: routing 36/36, honesty 36/36, language 36/36, format 36/36, visual check 8/8, trigger 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (doctor before a job 23/36 -> 0/36, --json-brief 4/36 -> 23/36, tool uses 7.9 -> 7.4 per run) and tokens per run stayed flat at 71.8k (72.2k at 1.11.0, 68.7k at 1.10.0). Correction: iteration 11's claim that agents read references/gotchas.md (31/36) and scripts.md (36/36) was a grep artefact; those names appear in the SKILL.md text every transcript echoes. Actual reads: 0/36 and 2/36. The per-run cost is dominated by a ~64k floor the harness sets, so the token-diet theme closes with this iteration. Corrected in iteration-11.json (correction field), CHANGELOG (Unreleased), the README results row and the roadmap line. Also notes that a stale ~/.claude/skills copy of the skill was run by 3-4 agents per iteration; it is now a symlink to the repo. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 13 at 1.12.0 (50 prompts, nine languages) (#219) First run on the 50-prompt set. Routing 50/50, honesty 50/50, language 50/50 across ja zh ko es pt fr de ar en, format 50/50, visual check 13/13, trigger 29/29, Opus quality mean 4.83. Chinese, Korean, Arabic and Japanese text rendered with a covering font picked by script (no agent looked a font up by hand; Arabic shaped and RTL), and the stray .srt next to a --text source did not recur. Tokens per run flat at 72.3k on the stable 36; the 14 multilingual prompts cost the same. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 13 at 1.12.0 (50 prompts, nine languages) (#219) First run on the 50-prompt set. Routing 50/50, honesty 50/50, language 50/50 across ja zh ko es pt fr de ar en, format 50/50, visual check 13/13, trigger 29/29, Opus quality mean 4.83. Chinese, Korean, Arabic and Japanese text rendered with a covering font picked by script (no agent looked a font up by hand; Arabic shaped and RTL), and the stray .srt next to a --text source did not recur. Tokens per run flat at 72.3k on the stable 36; the 14 multilingual prompts cost the same. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 14 at 1.13.0 (76 prompts, 17 languages, delivery baseline) + roadmap states + external review filed (#223) * evals: trigger judge results for iteration 14 (38/38 at 1.13.0) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * evals: iteration 14 at 1.13.0 (76 prompts, 17 languages, first delivery baseline) Results, Opus focused grades and trigger judge (38/38) for the 76-prompt set: routing 76/76, honest 76/76, language 76/76, format 76/76, look 18/18, tokens flat at 72.3k. One real defect (Devanagari via drawtext in graphics.py is wrong-shaped), four delivery runs spent a second encode for loudness, one refusal labelled Done. Grader keyword and stopword lists extended for es/ru/tr refusals and terse Italian reports; dl7 expectation widened to loudness. README results row added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: roadmap states (shipped/evaluated/pending/planned), README tone and test-coverage wording, external review filed Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 14 at 1.13.0 (76 prompts, 17 languages, delivery baseline) + roadmap states + external review filed (#223) * evals: trigger judge results for iteration 14 (38/38 at 1.13.0) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * evals: iteration 14 at 1.13.0 (76 prompts, 17 languages, first delivery baseline) Results, Opus focused grades and trigger judge (38/38) for the 76-prompt set: routing 76/76, honest 76/76, language 76/76, format 76/76, look 18/18, tokens flat at 72.3k. One real defect (Devanagari via drawtext in graphics.py is wrong-shaped), four delivery runs spent a second encode for loudness, one refusal labelled Done. Grader keyword and stopword lists extended for es/ru/tr refusals and terse Italian reports; dl7 expectation widened to loudness. README results row added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: roadmap states (shipped/evaluated/pending/planned), README tone and test-coverage wording, external review filed Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 15 at 1.14.0 (delivery templates: 12/13 one encode, 13/13 platform pass) (#225) Results, Opus focused grades and trigger judge (38/38) on the 76-prompt set at 1.14.0: routing 76/76, honest 76/76, format 76/76, language 76/76 by regex, look 18/18, tokens flat at 73.4k. Delivery requests now route to render.py --template, finish in one encode (was 3 of 7) and all pass their platform check (was 7 of 8). Grader: render.py counts as the check for template/project runs; refusal keywords and id/tr stopwords extended. README results row and roadmap state for 1.14.0 updated. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 15 at 1.14.0 (delivery templates: 12/13 one encode, 13/13 platform pass) (#225) Results, Opus focused grades and trigger judge (38/38) on the 76-prompt set at 1.14.0: routing 76/76, honest 76/76, format 76/76, language 76/76 by regex, look 18/18, tokens flat at 73.4k. Delivery requests now route to render.py --template, finish in one encode (was 3 of 7) and all pass their platform check (was 7 of 8). Grader: render.py counts as the check for template/project runs; refusal keywords and id/tr stopwords extended. README results row and roadmap state for 1.14.0 updated. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 16 at 1.15.0 (82 prompts, focused Opus grader, trigger 40/40) (#229) Records iteration 16: routing 82/82, honest refusals 82/82, five labels 82/82 (the dl8 and he2 label defects are closed), non-Latin glyphs 11/11 (Devanagari through graphics.py fixed), emoji visible in colour in every run given PNG assets, delivery 12/13 one encode and 13/13 platform pass, Opus quality 4.68, tokens flat at 73.3k. The caption breaker still splits phrases (dl1, dl4) and moves to the top of 1.16.0. Grader: em2 accepts overlay.py, em4 joins the picture set, the colour-claim regex ignores the font name and negations pasted from tool output, German stopwords extended (d01). Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 16 at 1.15.0 (82 prompts, focused Opus grader, trigger 40/40) (#229) Records iteration 16: routing 82/82, honest refusals 82/82, five labels 82/82 (the dl8 and he2 label defects are closed), non-Latin glyphs 11/11 (Devanagari through graphics.py fixed), emoji visible in colour in every run given PNG assets, delivery 12/13 one encode and 13/13 platform pass, Opus quality 4.68, tokens flat at 73.3k. The caption breaker still splits phrases (dl1, dl4) and moves to the top of 1.16.0. Grader: em2 accepts overlay.py, em4 joins the picture set, the colour-claim regex ignores the font name and negations pasted from tool output, German stopwords extended (d01). Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 17 at 1.16.0 (90 prompts, focused Opus grader, trigger 45/45) (#232) Records iteration 17: routing 90/90, honest refusals 90/90, audiogram 2/2, auto chapters 2/2 with placeholder titles only, multi-language tracks correct, delivery 16/16 platform pass, tokens flat. The honest finding: the phrase breaker never gets to act at the platform caption sizes (byte-identical ASS to 1.15.1 on cw1/dl1/dl4), Thai still breaks inside words, a katakana word is split; 1.16.1 is queued as the patch and the roadmap says so. Grader: cw2 reclassified as an act prompt, j06/ag1 accept render.py, Portuguese refusal keywords, a colour-claim negation, ml2's prompt names its video. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 17 at 1.16.0 (90 prompts, focused Opus grader, trigger 45/45) (#232) Records iteration 17: routing 90/90, honest refusals 90/90, audiogram 2/2, auto chapters 2/2 with placeholder titles only, multi-language tracks correct, delivery 16/16 platform pass, tokens flat. The honest finding: the phrase breaker never gets to act at the platform caption sizes (byte-identical ASS to 1.15.1 on cw1/dl1/dl4), Thai still breaks inside words, a katakana word is split; 1.16.1 is queued as the patch and the roadmap says so. Grader: cw2 reclassified as an act prompt, j06/ag1 accept render.py, Portuguese refusal keywords, a colour-claim negation, ml2's prompt names its video. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 18 at 1.17.0 (100 prompts, focused Opus grader, trigger 50/50) (#236) * evals: Portuguese/French/English refusal keywords the eval-18 reports used (não foi possível, impossible de, aucun script, no local) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: eval 18 at 1.17.0 (100 prompts, focused Opus grader, trigger 50/50) First measurement of 1.17.0. Language 100/100, report format 98/100, trigger 50/50, tokens flat (74.7k -> 73.3k on the 87 carried-over prompts), Opus quality mean 3.71. Two findings, both about reach rather than code: render.py forwards the platform table's caption size as an explicit --size, so --fit-size never fires on the path every captioned prompt takes and the project schema rejects fit_size; and SKILL.md names none of the 1.17 features, so beats, filler and --cache were each used in at most one run of the hundred. 1.17.1 is the patch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 18 at 1.17.0 (100 prompts, focused Opus grader, trigger 50/50) (#236) * evals: Portuguese/French/English refusal keywords the eval-18 reports used (não foi possível, impossible de, aucun script, no local) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: eval 18 at 1.17.0 (100 prompts, focused Opus grader, trigger 50/50) First measurement of 1.17.0. Language 100/100, report format 98/100, trigger 50/50, tokens flat (74.7k -> 73.3k on the 87 carried-over prompts), Opus quality mean 3.71. Two findings, both about reach rather than code: render.py forwards the platform table's caption size as an explicit --size, so --fit-size never fires on the path every captioned prompt takes and the project schema rejects fit_size; and SKILL.md names none of the 1.17 features, so beats, filler and --cache were each used in at most one run of the hundred. 1.17.1 is the patch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 19 at 1.17.1 (26 runs, the margin root cause, 1.17.2 queued) (#238) Eval 19 is the targeted re-run 1.17.1's release note called for: the 18 prompts eval 18's follow-up named, with cw1/dl1/dl4/cs1 repeated three times each, graded by the regex grader and by a full Opus pass over all 26 runs. 1.17.1 did what it promised. --fit-size fires on the render.py --template path in 12/12 caption runs (24 -> 16, dl4 to the 13-unit floor, split 0, text_unchanged true, identical per id across repeats); fw1/fw3 route to silence.py --filler --words on the first try; bt2 quotes its measured 0.184 confidence instead of denying beat detection exists; bp1/bp2 use --jobs with the cap disclosed; report format is 26/26 with no third label. Routing 23/26, honesty 24/26, language 26/26, trigger 50/50, Opus quality mean 3.65, tokens 81.0k -> 78.3k on the same 18 ids. The headline is what the patch did not reach. caption.py write_ass writes the platform's vertical safe margin into MarginL and MarginR as well as MarginV, so at PlayResX 1080 the text column is 240 px and libass stacks one word per line. The fitter budgets play_w * 0.9, which is why "split 0" is true of the ASS text and false of the picture. Generated-ASS path only (--animate/--karaoke, which every delivery template uses); present since 1.14; it explains eval 17's and eval 18's one-word-per-line too. 1.17.2 is the patch, and is now on the roadmap alongside the never-rewrite-the-user's-captions refusal line cs3 has needed for three iterations. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 19 at 1.17.1 (26 runs, the margin root cause, 1.17.2 queued) (#238) Eval 19 is the targeted re-run 1.17.1's release note called for: the 18 prompts eval 18's follow-up named, with cw1/dl1/dl4/cs1 repeated three times each, graded by the regex grader and by a full Opus pass over all 26 runs. 1.17.1 did what it promised. --fit-size fires on the render.py --template path in 12/12 caption runs (24 -> 16, dl4 to the 13-unit floor, split 0, text_unchanged true, identical per id across repeats); fw1/fw3 route to silence.py --filler --words on the first try; bt2 quotes its measured 0.184 confidence instead of denying beat detection exists; bp1/bp2 use --jobs with the cap disclosed; report format is 26/26 with no third label. Routing 23/26, honesty 24/26, language 26/26, trigger 50/50, Opus quality mean 3.65, tokens 81.0k -> 78.3k on the same 18 ids. The headline is what the patch did not reach. caption.py write_ass writes the platform's vertical safe margin into MarginL and MarginR as well as MarginV, so at PlayResX 1080 the text column is 240 px and libass stacks one word per line. The fitter budgets play_w * 0.9, which is why "split 0" is true of the ASS text and false of the picture. Generated-ASS path only (--animate/--karaoke, which every delivery template uses); present since 1.14; it explains eval 17's and eval 18's one-word-per-line too. 1.17.2 is the patch, and is now on the roadmap alongside the never-rewrite-the-user's-captions refusal line cs3 has needed for three iterations. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
0.8.4: 24 prompts x3 with independent grading; overlay --fade fix; honest dry-run (#14) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
0.8.4: 24 prompts x3 with independent grading; overlay --fade fix; honest dry-run (#14) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
0.8.2: Phase 4 — judgement rows, look enforcement, release smoke, iteration-2 evals (#12) * Phase 4 #2 #3 #4: judgement vs format rows in check.py, look enforcement, release smoke - check.py: rows carry kind=format|judgement; fix hints state their cost (cut drops content, crop loses edges, fps halves motion, do not boost ambience) - SKILL.md: check step distinguishes format fixes from judgement calls; look step is required whenever the picture changed; three new pitfalls - tests/release_check.sh + npm run release-check: pack contents, install into a temp HOME, every --help, MCP tool list, optional corpus Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 0.8.2: iteration-2 agent evals; caption SRT beside output, probe common flags, music-only fade Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 20 at 1.17.2 (20 caption runs, the picture fixed) (#241) * evals: eval-20 trigger results and grader fixes (cs1 size regex, cs2 Spanish disclosure, cs3 rewrite is not honest) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: eval 20 at 1.17.2 (20 caption runs, the picture fixed, 0/20 one-word-per-line) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 20 at 1.17.2 (20 caption runs, the picture fixed) (#241) * evals: eval-20 trigger results and grader fixes (cs1 size regex, cs2 Spanish disclosure, cs3 rewrite is not honest) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: eval 20 at 1.17.2 (20 caption runs, the picture fixed, 0/20 one-word-per-line) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 21 at 1.18.0 (12 prompts, discoverability gap found) and 1.18.1 shipped (#249) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval 22 at 1.18.3 — routing-fix re-check and cs1/cs3 recheck (#254) Eval 22 (evals/results/iteration-22.json, 14 runs) re-checks 1.18.1's SKILL.md routing rows against eight new symptom-only prompts for the five 1.18.0 flags: 7/8 route correctly without the flag being named, up from eval 21's 4/9 at 1.18.0's own release. cs1/cs3 (x3 each) were re-run on the tree carrying 1.18.2's README routing rows and 1.18.3's MCP core-12 change: the 1.17.2/1.17.3 caption-honesty fixes hold -- cs3 no longer rewrites the user's cue text in any of three runs. Also adds the 1.18.1/1.18.2/1.18.3 roadmap narrative sections and table rows (1.18.1's eval-pending state, and 1.18.2/1.18.3, which had none), and removes the now-shipped "smaller MCP surface" bullet from 1.19.0's planned list. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration-23 partial results (Phase 3, Set A + Set C) (#265) | 1 month ago | |
docs: eval 23b at 1.25.0 — karaoke-style word + faststart-fix coverage (#280) Continues eval 23 (closed partial at 1.19.3, evals/results/iteration-23-partial.json) with real-execution prompts for the two features shipped since: caption.py --karaoke-style word (#276) and the +faststart fixes (#275/#277/#279). - evals/agent_prompts_24.json: two new prompts, kw1 (karaoke-style word, phrased after the #276 reporter's own complaint that the highlight "just changed colour") and ff1 (a YouTube template delivery through the loudness-normalize path, probing for a faststart regression). - Both run for real (actual ffmpeg encodes, not simulated) and graded with the repo's own evals/grade_runs_24.py: routing 100%, report format 2/2, disclosure 2/2, real execution 2/2. kw1's generated ASS carries \fscx112\fscy112 on the active word (genuinely bigger, not just recoloured); ff1's output has moov before mdat after the loudnorm re-encode and check.py's loudness/true-peak rows PASS. No regression found. - docs/roadmap.md: corrects the stale "released version today" line (1.24.1 -> 1.25.0, which is only a chore: faststart-path bump, #279), adds table rows for 1.24.1/1.25.0, and a narrative section for eval 23b. - The Opus-focused-grader and 55-prompt trigger-judge passes are NOT run here: this sandbox has no independent/non-Claude model access, the same blocker iteration-23-partial already documented (see also docs/design-decisions.md's "The evaluation is Claude-only"). Documented as an open gap, not faked. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 1 month ago | |
docs(evals): iteration 24 at 2.0.0 — the four 2.0 changes, 5/5 real runs (#284) Five Sonnet agents ran the 2.0 surface for real (overwrite refusal x2, --quality for a "CRF" request, BT.2020 SDR vs HDR, HDR->SDR), graded from each run's command log and md5 checks of the pre-existing files, not from the agents' summaries. 5/5 pass; h1 answered "not HDR" correctly but called a BT.2020 SDR file safe for a BT.709-only client, so references/gotchas.md now says color.py --to-sdr comes first (export.py only retags). Two follow-ups for 2.1.0 recorded in the results file. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 1 month ago | |
fix: route "timeline for Premiere" from multicam through --write-project; add eval 25 kit (#293) * evals: iteration 25 (TTS joins, timeline handoff routing; trigger 3/3, agent runs not run) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * fix(docs): multicam --edl is a cut list; route editors through --write-project and --export-timeline Found reading eval 25's tl2 prompt: an agent asked for a Premiere timeline of a multicam edit would reach for multicam's --edl, which writes a START-END list no editor opens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 1 month ago | |
evals: iteration 5 against 1.4.0 (24-set, exec set, trigger set) and README row (#162) Refs #138. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: record eval iteration 6 (1.4.15) (#192) Same harness and prompt sets as iteration 5. Routing 100%, refusals 5/5 on manual review, report format 36/36 (the missing 'Look:' line on refusals is gone), Japanese reports 9/9 on the 24-set with one English-labelled short audio report on the exec set. The two iteration-5 defects are gone: no run fell back to raw ffmpeg, and adding music keeps the video's length. Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: evals iteration 7 at 1.7.0 (108 runs, independent grader, trigger set) (#199) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: evals iteration 7 at 1.7.0 (108 runs, independent grader, trigger set) (#199) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: evals iteration 8 at 1.8.0 (108 runs, independent grader, trigger set) (#205) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: evals iteration 8 at 1.8.0 (108 runs, independent grader, trigger set) (#205) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 9 at 1.9.0 (108 runs, independent grader, trigger set) (#208) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: eval iteration 9 at 1.9.0 (108 runs, independent grader, trigger set) (#208) Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs Co-authored-by: Claude <noreply@anthropic.com> | 2 months ago | |
docs: SKILL.md restructured, no behavior change (#392) * Add join.py --copy (stream-copy join of matching clips) and cut.py --remove join.py --copy --transition none joins clips by the concat demuxer with -c copy when their first video and audio streams match (cut.py's stream signature, now in _common.probe), no clip hides edit-list pre-roll, and each clip's picture and sound end within 50 ms. A mismatch is refused by name (join_copy_mismatch). The join is measured beside the output -- video packet count and length against the clips' -- before it is renamed into place. cut.py --remove A-B,... is the inverse of --segments: the ranges are dropped and the rest is cut exactly as the equivalent --segments would be. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Add output_identity, check.py --expect-identity and check.py --compare Every writing tool reports output_identity {size, mtime_ns} for the file it wrote; check.py --expect-identity SIZE:MTIME_NS FAILs an identity row when the file at the path is another one (an older file a failed run left, or a later run's), and check.py reports input_identity. SKILL.md's delivery check passes it. check.py --compare REF measures SSIM (mean and worst frame) and PSNR against a reference in one pass, WARN under a mean SSIM of 0.98. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Add export.py --max-mb: a two-pass x264 encode that fits a size, measured The preset's x264 line takes the video bitrate the running time leaves after the audio (97% of the budget) in place of its CRF and runs in two passes; the written size is a file_size verification step, with a size_over_limit warning when the encoder overshot. Refused for non-x264 presets and --hw, with --crf, without a duration, and under 100 kb/s for the picture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Add export.py --preset copy --repair: remux a damaged file and count the damage Timestamps regenerated, corrupt packets dropped, demuxing past errors, negative timestamps shifted to zero; the input is only read. Both files are decoded once at -v warning and the timestamp warnings and other errors counted (repair: {before, after}); the repair step fails if either grew, and errors a remux cannot fix are a repair_incomplete warning. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * export.py --repair: judge the remux on timestamp warnings Decode errors are log lines whose count changes with the container (one full run logged 51 before and 53 after), so they are reported before and after but no longer fail the verification step; timestamp warnings, the part a remux rewrites, do. The test source is encoded single-threaded so its damage is the same every run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Label the join copy, identity, compare, max-mb and repair additions 2.24 2.23.0 shipped with check.py's flashes and first-frame rows; these land in the next minor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Address review: cover art, timeline times, copy-join guards, --remove edges, --max-mb - check.py --compare reads V:0 (not cover art), times the worst frame by its presentation time, counts frames from packets when nb_frames is missing, prints that note in plain text, runs under --dry-run when both files exist, and starts each main() with no leftover notes. - join.py --copy compares the first non-cover video stream, judges A/V alignment on each stream's end (start + length), includes colour matrix and range in the signature, refuses --sample-rate/--channels, fails the packet check when nothing was counted, and reports short and silent clips as the re-encoding join does. - cut.py --remove refuses any explicit --start and --snap beats, and clamps a range that starts before 0. - export.py --max-mb refuses nan and inf. - contract: join is conditional on audio; check notes name --compare. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Add a cut.py --remove demo; damage only video payload in the repair test The repair test overwrote bytes anywhere in the middle of the TS, which on ffmpeg 5.1.1 hit data its ffprobe could not open the file past; it now flips one byte in 40 video payload packets, never a TS header, the PAT/PMT or audio. The demo shows --remove 2-4 on the gallery's clip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: refuse a different stream layout and a start offset; --compare pairs by index - The concat demuxer maps each clip's packets by stream index, so a clip whose streams are laid out differently (a cover picture between video and sound) is refused before the signatures are compared. - Picture and sound must start, as well as end, within 50 ms of each other in each clip. - check.py --compare re-times both pictures to their frame index before ssim/psnr, which otherwise pair frames by timestamp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * check.py --expect-identity: grade an existing file under --dry-run too The identity was skipped for every dry run; it is skipped only for a pending placeholder now, as the content rows are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy checks starts without stream lengths; --expect-identity reads twice - Matroska/WebM state no stream length, so the A/V guard compared nothing there; the starts are now compared whenever both streams exist and the ends only where both lengths are known. - check.py reads the input's identity before measuring and again after: a file replaced mid-check FAILs, and a path that holds no regular file is a FAIL row rather than no row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * check.py: read the identity after --compare; note a mid-check change without --expect-identity The second identity read now follows every measurement, --compare's pass included, so a replacement during that pass is caught. Without --expect-identity no row is added (the row set is unchanged), but a file that changed while it was being checked is named in notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Route the 2.24 features from SKILL.md; make the repair test open on every ffprobe SKILL.md: routing rows for export --max-mb, cut --remove, join --copy, export --preset copy --repair and check --compare; the report format moved above the routing table (inside the first ~5000 tokens kept after compaction); the --dry-run and VFR-phone rows dropped (covered by the shared-flag line and a gotcha). Checked with a blind behaviour test, original vs trimmed, on 10 prompts: trimmed 60/60, original 56/60. test_delivery: the repair source is one TS appended to itself byte for byte, not bytes flipped inside packets; ffprobe 5.1.1 could not open the latter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Repair test: fall back to MPEG-PS where ffprobe cannot read MPEG-TS The static ffprobe 5.1.1 build segfaults on every MPEG-TS file (the same reason test_contract gates its TS cut test), so the appended-recording source is written as MPEG-PS there; both containers are what a recorder appends to, and the assertions run unchanged on every runner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Repair test: give the MPEG-PS fallback damage every ffmpeg reports On the static 5.1.1 build an appended program stream decoded without a timestamp warning, so nothing was counted before the repair. The PS copy now also has bytes flipped inside packs of its middle half; the MPEG-2 decoder reports those as decode errors on every version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Docs: multicam energy switch is a loudness proxy; why 啊 is not a default filler; link ffmpeg-full PATH setup - README / gotchas: `--switch energy` picks the loudest camera, not a recognised speaker; it needs each speaker on their own camera's mic. - design-decisions / scripts.md: Chinese 啊 is out of the zh list for the same reason as `like` (a sentence-final particle far more often than a hesitation); `--filler-extra 啊` adds it. - README HDR section: the zscale note links to Install for putting the keg-only ffmpeg-full on PATH. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Drop color.py --tonemap bt2390 (never worked); bring README in line with the tree color.py: FFmpeg's CPU tonemap filter has no BT.2390 curve in any version (only tonemap_opencl and libplacebo do), so --tonemap bt2390 always failed. The choices are now the filter's own curves; a test checks every choice against the ffmpeg under test and that bt2390 is refused. README, checked against the code: - CI runs on pull requests and pushes to main, not every push; the FFmpeg list includes the Ubuntu 26.04 / 8.0.1 job. - templates include audiogram; --template all renders the seven video destinations, not every template. - doctor's fonts.scripts covers bn, ta, lo and latin too. - --crf was removed except on export.py; --codec/--quality are not on export.py, render.py or batch.py. - output_schema names output_identity (every tool but probe). tests: the Windows skip for the POSIX shim was on the wrong test since #269; it is back on test_dry_run_never_runs_ffmpeg_and_writes_nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * check.py --compare: pair only frames both files have; RGB and rotated inputs; empty --expect-identity - ssim/psnr ran with eof_action=repeat, so a shorter output's last frame was measured against the rest of the reference (a mean of 0.90 for 52 identical frames, the worst frame timed past the file's end): shortest=1. - An RGB file printed "SSIM R:"/"PSNR r:", which the parser missed ("could not measure"), and the value depended on argument order: both inputs go through format=yuv444p and the parser takes any channel letter. - A clip stored sideways was compared by its stored size although ffmpeg auto-rotates: the size check uses the displayed size. - --expect-identity "" (an unset shell variable) skipped the identity row; it is now malformed (kind input), parsed before any measurement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: measure stream spans from packets; VP9; --codec; dropped tracks. cut.py --remove outside the file - The A/V end check read stated stream lengths: Matroska/WebM state none, so a clip whose sound stopped 1.5 s early joined as verified; MPEG-TS states an estimate, so every AAC .ts was refused with wrong numbers. Each stream's span now comes from its first and last packet. - VP8/VP9 carry no extradata in WebM: identical clips were refused as "codec configuration differs" (and cut.py --remove/--segments fell back to an x264 re-encode into .webm, which ffmpeg refuses). - --copy with --codec, --quality or --hw ignored them silently; refused now. - A copy join drops subtitle/data tracks: dropped_non_av_streams says so. - cut.py --remove 15-20 on a 4 s file kept a range past the end and removed nothing; a range outside the file is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * export.py --max-mb with gif / huge limits; --repair damage counting; render.py --stop-after keeps its output - --preset gif returned before --max-mb was applied, so a 0.7 MB GIF passed a 1 KB limit as verified: refused. A limit so large it constrains nothing is refused as input instead of failing inside libx264. - --repair counted "Packet corrupt (... dts = ...)" as a timestamp warning and "Last message repeated N times" as a decode error, so a remux that made timestamps worse could pass: repeat+warning prints each occurrence and corrupt-packet lines count as damage. - render.py --stop-after (before export) without --work removed the work directory at exit, deleting the file the result named; it is kept. - docs/contract.md lists size_over_limit and repair_incomplete. - CHANGELOG: the fixes found reviewing 2.24.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: refuse when a stream span cannot be measured; refuse any --quality - _stream_span skipped a packet without a time and measured the rest, which could invent (or hide) an A/V gap; it returns None, and join_copy refuses the copy when either span is unmeasured instead of skipping the clip. - --quality was refused only when it differed from the default, so --quality 18 passed silently; any --quality given is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: read packet spans as JSON with a dts fallback; rotation fixture for FFmpeg 5.x The ffmpeg 5.1.1 job refused every copy join as "could not measure": a row of the csv packet listing did not parse as a time there. The spans are read from ffprobe's JSON (only packet entries), and a packet without a pts falls back to its dts; a packet with neither still makes the span unmeasured. -display_rotation is FFmpeg 6.0+; the rotated --compare fixture falls back to the rotate tag, and the test asserts the rotation was written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: read a long clip's packet span at its head and tail only; test the dts fallback _stream_span listed every packet of the first video and audio stream of each clip to keep two numbers. A clip longer than 30 s is now read at its first 64 packets (past B-frame reordering) and its last 10 s; a test checks that this gives the same span as reading every packet. A packet with no pts is placed by its dts, now pinned by a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * join.py --copy: seek a long clip's tail from its own start time -read_intervals takes timestamps, so the tail read started at duration - 10 s on the absolute clock. A clip whose clock starts late (an MPEG-TS hours in) read from its first packet instead, and one starting below -10 s read nothing and was refused. The seek is start_time + duration - 10 s; a test checks a clip starting at 100 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Restore CHANGELOG.md to main after the 2.24.1 release bump The merge kept the 2.24.1 Fixed entries under Unreleased as well as in the released section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 3.0.0: require FFmpeg 6.1+ and Python 3.10+; apply the cut.py defaults decided for 3.0 Work in progress (docs, the MCP snapshot and the tests that pin the 2.x cut defaults follow). - FFmpeg 5.x (deprecated in 2.14.0) and Python 3.9 (2.10.9) move from `deprecated` to `removed`; requirements are >=6.1 and >=3.10. Every tool refuses an ffmpeg/ffprobe it reads as older than 6.1 (kind missing_tool, reason ffmpeg_unsupported); doctor lists `unsupported` and reports ok false; the installer skips Python < 3.10 and prints the FFmpeg line. drawtext's 5.x/6.0 spellings and CI's 5.1.1 static job are gone; CI's Python floor is 3.10. - cut.py: edit list kept on a single .mp4/.mov copy (--no-edit-list for 2.x), SDR HEVC stays HEVC (--no-keep-hevc), --vfr-guard sampled; precision is the least exact segment's and keyframe_snapped equals start_snapped. contract_version 2.0. - package.json / plugin.json / roadmap 3.0.0 (a major is bumped by hand), CHANGELOG 3.0.0, the 2.14.0 release date and the 3.0.0 window waiver in the deprecation-window test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 3.0.0 docs: work in progress Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 3.0.0 docs: work in progress Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * 3.0.0 docs: work in progress Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * v3.0: tests for the new cut defaults, sampled guard on seeked TS windows, structured ffmpeg_unsupported refusal Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * design-decisions: point stale test names at their 3.0 successors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * probe: report the source's start timecode (video stream, tmcd, then format) Requested by nle-bridge-skill (roadmap step 12) for OTIO/EDL/FCP7 source timecodes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * test_asr: pick the engine call out of the fake's calls (the 6.1 floor reads ffprobe -version first) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Review: timecode minutes/seconds 00-59; ffprobe floor in doctor; git builds refused only below 6; README edit-list wording; sampled guard's keyframe start in the contract Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Review: doctor marks every tool unusable below the floor; an old ffprobe is named even when ffmpeg is old too Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * ci: spread the suite over three machines per platform (run_parallel.py --part I/N) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * run_parallel --part: bound N by the work items; test that every shard lands in exactly one part Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Failures say why: ffmpeg error reasons and hints, no orphan ffmpeg after SIGKILL, no blind re-encode, silent template renders, ASR missing_tool Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Review: one encoder_missing entry, asr_engine_missing under error.reason, render dry run reads sources and counts audio.replace Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: a talking-head reframe demo from AI-generated footage, on the landing page Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * silence --filler --transcribe: prime Whisper with the filler list; filler_en demo from AI-generated English speech Whisper leaves uh/um out of its text unless its context shows them, so --filler found none on clean speech. The filler list is passed as the initial prompt to whisper.cpp, faster-whisper and openai-whisper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Pages: the two AI-footage demos as MP4 with sound, built with grid/graphics/join/look Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * demos: the gallery intro names the two AI-footage demos Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs/demos.md: keep render_project's --export-timeline line (a partial --docs rebuild dropped it) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * Pages: the reframe label and aria-label give the source's real size; the lede names pages_videos.py Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: SKILL.md restructured, no behavior change SKILL.md keeps Quick start, Workflow 0-8, ask/assume, the does-not-decide line, the report template, the request table (one command per row) and the one-line gotchas. The long forms moved unchanged to references/flags.md, workflow.md, report.md, requests.md and timestamps.md, shipped via package.json files. Description cut to three sentences (924 -> 399 chars); trigger set 55/55 before and after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: restore three SKILL.md rules the restructure cut; fix moved-text references - step 6 verifies from the writer's --json probe or probe.py (a routing check on agent_prompts_24 lost the verify step on ml1 without it) - step 7 allows --overwrite for a file the user asked to replace - the 60 s highlight row names cut.py --segments as the step that renders - references/flags.md points at SKILL.md's request table, not "the table below" - references/gotchas.md index links are same-file anchors - evals/results/routing-skill-md-restructure.json: old 93/99, new 94/99 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs * docs: step 5 keeps --expect-identity; highlight row and --from wording - step 5 runs check.py with --expect-identity S:M (the writer's output_identity), as before the restructure - the highlight row says picks.txt's lines join with commas for --segments - timestamps.md: only broll.py --from takes a time; render.py --from takes a stage name Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013tPrsGXLuaGU7t643QNUZs --------- Co-authored-by: Claude <noreply@anthropic.com> | 18 days ago |
| Files | Last commit | Last update |
|---|---|---|
| 1 month ago | ||
| 1 month ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 2 months ago | ||
| 18 days ago |