文件	最后提交记录	最后更新时间
source	feat(jacobian_lens): add dynamic J-space coordinate-patch hooks (#1749) * feat(jacobian_lens_coordinate_patch): add per-position patch loop core - Add solve_coordinate_patch_positions: model-free loop applying solve_coordinate_patch independently to every (batch, position) pair in a [batch, num_positions, d_model] chunk - Support an optional caller-owned decomposition_cache keyed (layer, batch_idx, position); a hit skips get_sparse_decomposition, a miss solves once and stores - Fail fast (no try/except) when a source is inactive at any pair, so no partial write reaches the activation tensor - Validate 3-D activations and matching position_labels length - Add unit tests for offline parity, batch independence, fail-fast, cache miss/hit behavior, and label-length validation * test(tools): extract shared J-lens toy-bridge fixtures into conftest - move D_MODEL/N_LAYERS/D_VOCAB/SEQ_LEN/SKIP_FIRST/CORPUS, _ToyBlock, _CausalSumBlock, _ToyTokenizer, _ToyBridge, _NotABridge, _lens, and the toy_model fixture from test_jacobian_lens.py into conftest.py - import the shared symbols back into test_jacobian_lens.py and remove the now-unused contextmanager and HookPoint imports - centralize the shared test setup so additional hook tests can reuse the same fixtures without duplicating roughly 90 lines of test code * feat(jacobian_lens): expose coordinate_patch_hooks on the Bridge - Add JacobianLens.coordinate_patch_hooks, a forward-hook variant of the offline coordinate_patch primitive, following the swap_hooks builder pattern - Solve one J-space coordinate patch per (batch, position) pair at each layer via solve_coordinate_patch_positions, with an optional caller-owned decomposition_cache keyed (layer, batch_idx, position) - Require positions explicitly and reject identical source/target tokens; fail fast on any inactive source rather than partially patching a batch - Warn once per call naming the layer x position count that performs a live vocabulary-scale solve on every cache miss - Add a dedicated test file covering shape parity, warning-once, cache hit/miss, oracle parity with offline coordinate_patch, and uncaught error and warning propagation through the hook * feat(jacobian_lens): export and document coordinate_patch_hooks Export solve_coordinate_patch_positions from tools.analysis and document the distinction between offline and dynamic/hooked coordinate patching. Replace the stale dynamic-patching claim in jacobian_lens_fitting.md and add documentation covering required positions, caller-owned decomposition_cache, per-pair fail-fast behavior, and the once-per-call cost warning. Add a cached GPT-2 integration test verifying alpha=0 is an exact no-op and untouched positions remain bit-identical. * fix(jacobian_lens): drop leaked sparse_probing exports from analysis __init__ Commit 02ffc277 accidentally added sparse_probing imports and __all__ entries to transformer_lens/tools/analysis/__init__.py without committing the sparse_probing module itself. On CI (which only checks out tracked files) importing transformer_lens.tools.analysis raised ModuleNotFoundError, breaking package import and failing every job that imports transformer_lens (unit, docstring, compatibility, benchmark, coverage, notebooks). Remove the out-of-scope sparse_probing exports; that work belongs to its own PR. * chore(jacobian_lens): drop unrelated sparse-probing and cursor changes from PR scope - Restore .cursor/rules/transformerlens.mdc deleted on this branch (unrelated to coordinate-patch hooks) - Remove sparse_probing import + test from tests/mps/test_mps_basic.py that broke tests/mps collection with ModuleNotFoundError (accidentally picked up from another branch) - Remove content/sparse_probing toctree line from docs/source/index.md referencing a nonexistent page * fix(jacobian_lens): straight-through gradient in coordinate-patch position loop - Scatter `x + patch.delta` instead of the detached `patch.patched` in solve_coordinate_patch_positions, keeping x's graph; bitwise identical forward for float32 activations. - Add test_solve_coordinate_patch_positions_preserves_gradient_to_patched_position: asserts finite nonzero grad at the patched position (exactly 0 before) and bitwise-equal forward output. * test(jacobian_lens): pin batch component of decomposition_cache key - Add test_solve_coordinate_patch_positions_populates_batch_component_of_cache_key: two distinct batch rows share one cache at the same (layer, position); assert both (layer, 0, position) and (layer, 1, position) populate with different active supports. - Closes the gap where the batch-agnostic [1, 1, d_model] fixtures let a collapsed (layer, 0, position) key pass the whole unit tier; the new test fails when the key drops batch_idx. * docs+test(jacobian_lens): document top-k support precondition and cover plural-layer install - Strengthen coordinate_patch_hooks Raises note: source must be in every patched position's top-k active support after earlier band hooks edit the residual, not merely active on a clean pass; substitute/swap remove the source coordinate, so stacking layers/positions makes it progressively harder. - Add the first plural (two-layer) end-to-end hook test, pinning the per-closure layer=layer/dictionary=dictionary binding via distinct per-layer dictionaries and per-layer cache keys. * docs(jacobian_lens): correct dynamic-hook generate() claim and cache-reuse caveat - Drop the model.generate(...) claim from the dynamic coordinate-patch hooks section: on GPT-2 with the published lens, generate re-decomposes each freshly generated token whose top-k support generally lacks the source concept, so fail-fast aborts generation. Keep run_with_hooks(...). - Document that the (layer, batch_idx, position) cache key uses the chunk-local position index, so decomposition_cache is valid only across identical-chunking passes; warn against reuse across decode steps (use_past_kv_cache prefill vs per-step shapes collide) or differently-shaped prompts.	6 天前
Makefile	Sphinx documentation Solving issue #132 (#174) * Add Sphinx Documentation	3 年前
README.md	Docs/fix links uv setup (#1435) * tidy: Remove reference to deleted additional comments (#1410) Simple resolution for #1409 * Fix docs links and clarify UV setup * Address review feedback --------- Co-authored-by: Dan Raviv <dan.raviv@gmail.com>	2 个月前
make.bat	Sphinx documentation Solving issue #132 (#174) * Add Sphinx Documentation	3 年前
make_docs.py	docs(jacobian_lens): add sparse decomposition demo (#1738) * docs: add Jacobian Lens decomposition demo * docs(jacobian_lens): synchronize demo captions with saved outputs * fix(demos): read RUN_FULL_ANALYSIS from environment variable Notebook hard-coded RUN_FULL_ANALYSIS = True, so the documented quick tier was only reachable by hand-editing the notebook. Read it from the RUN_FULL_ANALYSIS env var (default "true") so CI or callers can select the quick tier without touching the file.	13 天前

Transformer-Lens Docs

This repo contains the website for TransformerLens. This site is currently in Beta and we are in the process of adding/editing information.

The documentation uses Sphinx. However, the documentation is written in regular md, NOT rst.

Build the Documentation

For the standard contributor setup, install the default dependency groups:

uv sync

For a docs-focused environment without the other default groups, install only the docs group:

uv sync --no-default-groups --group docs

Then for hot-reloading, run this (note the model properties table won't hot reload, but everything else will):

uv run docs-hot-reload

Alternatively to build once, run:

uv run build-docs