| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Agentic Workflow Support (#1357) * Initial Agentic Workflow support setup * initial setup of sub-directory files * Optimizing rools * Adjustments to model support based on an initial end-to-end test * Review changes * Additional review pass * second end-to-end test changes * brevity pass * additional adjustments * additional detail files * Additional documentation cleanup | 3 个月前 | |
feat(backward_lens): add vocabulary readout of GPT-2 MLP gradient factors (#1723) * feat(backward_lens): add gradient factorization contracts - Reconstruct linear weight gradients from forward inputs and output VJPs. - Support GPT-2 Conv1D and torch.nn.Linear weight layouts. - Add detached CPU contracts, validation, error metrics, and signed vocabulary rankings. - Cover numerical, validation, overflow, ownership, and ranking edge cases. * feat(backward_lens): capture GPT-2 MLP gradient factors - Validate raw GPT-2 Bridge models, prompts, targets, layers, and original Conv1D weights. - Capture both MLP projections with one forward pass and one autograd call. - Reconstruct exact weight gradients and return detached CPU-owned factors. - Preserve model state, RNG, gradients, and existing hooks with comprehensive integration coverage. * feat(backward_lens): project GPT-2 MLP gradient factors into vocabulary space - Add public BackwardLens.analyze(prompt, target_token, layers, normalized=False) returning detached, CPU-owned result dataclasses. - Project residual-width factors through fresh ln_final and unembed, with an optional Normalized Logit Lens for low-norm factors. - Expose factor norms, zero-norm masks, signed top/bottom vocabulary rankings, token decoding, and raw-gradient target ranks. - Export the public result contracts and cover the API with model-free unit tests and GPT-2 integration tests. * docs(backward_lens): document GPT-2 gradient factor analysis - Derive the FF1 and FF2 gradient factorizations, vocabulary projections, and tensor-shape contracts. - Document the public API, raw Bridge and single-token restrictions, state-safety guarantees, and error behavior. - Explain raw versus normalized projections, zero-factor handling, gradient-versus-SGD signs, interpretation limits, and troubleshooting. - Add the guide to the documentation index and cite Katz et al. without copying external code or assets. * docs(backward_lens): add executed GPT-2 gradient factor demo - Add an executed GPT-2-small walkthrough of MLP gradient factorization and vocabulary projection. - Visualize reconstruction error, numerical rank, layer-position token directions, VJP norms, and target ranks. - Compare raw and normalized projections, expose gradient-update sign semantics, mark near-zero signals, and sort cumulative reconstruction by contribution. - Save reproducible outputs and register all ten notebook cells in CI and the Makefile notebook test target. * fix(backward_lens): stabilize demo notebook validation - Suppress the narrow Typeguard instrumentation warning emitted during fresh TransformerLens imports. - Display target ranks as stable vocabulary percentiles while retaining exact ranks in the analysis data. - Regenerate all notebook outputs and verify all ten nbval cells pass. * fix(backward_lens): stabilize rank percentile output * fix(backward_lens): support beartype test instrumentation * fix(backward_lens): support beartype coverage instrumentation * fix(backward_lens): ignore setup-cell notebook output * feat(backward_lens): bound vocabulary readout retention * fix(backward_lens): harden capture preconditions * refactor(backward_lens): align tensor validation contracts * test(backward_lens): strengthen gradient behavior coverage * docs(backward_lens): clarify factor and token semantics * fixing export ordering * fix(backward_lens): restore weight via in-place copy instead of functional_call - swap torch.func.functional_call reparametrization for weight.copy_(updated_weight) under no_grad - restore original weight via try/finally so cleanup happens even if assertion fails - functional_call left layer 11 c_proj weight overridden in the bridge's original_model, breaking the three public_* fixtures later in this module (coverage-tier failure; Compatibility jobs don't run this tier) --------- Co-authored-by: jlarson4 <jonahalarson@comcast.net> | 17 天前 | |
added news link (#620) * added news link * moved or * fixed format * added note about docs branches to pr template | 2 年前 | |
Agentic Workflow Support (#1357) * Initial Agentic Workflow support setup * initial setup of sub-directory files * Optimizing rools * Adjustments to model support based on an initial end-to-end test * Review changes * Additional review pass * second end-to-end test changes * brevity pass * additional adjustments * additional detail files * Additional documentation cleanup | 3 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 3 个月前 | ||
| 17 天前 | ||
| 2 年前 | ||
| 3 个月前 |