{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/neelnanda-io/TransformerLens/blob/main/demos/Interactive_Neuroscope.ipynb\">\n",
" <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
"</a>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Interactive Neuroscope\n",
"\n",
"*This is an interactive accompaniment to [neuroscope.io](https://neuroscope.io) and to the [studying learned language features post](https://www.alignmentforum.org/posts/Qup9gorqpd9qKAEav/200-cop-in-mi-studying-learned-features-in-language-models) in [200 Concrete Open Problems in Mechanistic Interpretability](https://neelnanda.io/concrete-open-problems)*\n",
"\n",
"There's a surprisingly rich ecosystem of easy ways to create interactive graphics, especially for ML systems. If you're trying to do mechanistic interpretability, the ability to do web dev and to both visualize data and interact with it seems high value! \n",
"\n",
"This is a demo of how you can combine TransformerBridge and [Gradio](https://gradio.app/) to create an interactive Neuroscope - a visualization of a neuron's activations on text that will dynamically update as you edit the text. I don't particularly claim that this code is any *good*, but the goal is to illustrate what quickly hacking together a custom visualisation (while knowing fuck all about web dev, like me) can look like! (And as such, I try to explain the basic web dev concepts I use)\n",
"\n",
"Note that you'll need to run the code yourself to get the interactive interface, so the cell at the bottom will be blank at first!\n",
"\n",
"To emphasise - the point of this notebook is to be a rough proof of concept that just about works, *not* to be the well executed ideal of interactively studying neurons! You are highly encouraged to write your own (and ideally, to [make a pull request](https://github.com/neelnanda-io/TransformerLens/pulls) with improvements!)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# NBVAL_IGNORE_OUTPUT\n",
"# Janky code to do different setup when run in a Colab notebook vs VSCode\n",
"import os\n",
"\n",
"DEVELOPMENT_MODE = True\n",
"IN_GITHUB = os.getenv(\"GITHUB_ACTIONS\") == \"true\"\n",
"try:\n",
" import google.colab\n",
"\n",
" IN_COLAB = True\n",
" print(\"Running as a Colab notebook\")\n",
"except:\n",
" IN_COLAB = False\n",
" print(\"Running as a Jupyter notebook - intended for development only!\")\n",
" from IPython import get_ipython\n",
"\n",
" ipython = get_ipython()\n",
" # Code to automatically update the HookedTransformer code as its edited without restarting the kernel\n",
" ipython.run_line_magic(\"load_ext\", \"autoreload\")\n",
" ipython.run_line_magic(\"autoreload\", \"2\")\n",
"\n",
"if IN_COLAB or IN_GITHUB:\n",
" %pip install transformer_lens\n",
" %pip install gradio\n",
" %pip install datasets==2.19.1"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"import gradio as gr\n",
"from transformer_lens.model_bridge import TransformerBridge\n",
"from transformer_lens.utilities import to_numpy\n",
"from IPython.display import HTML"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extracting Model Activations\n",
"\n",
"We first write some code using TransformerBridge's cache to extract the neuron activations on a given layer and neuron, for a given text"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n",
"The following generation flags are not valid and may be ignored: ['output_attentions']. Set `TRANSFORMERS_VERBOSITY=info` for more details.\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:592: FutureWarning: Hook 'W_V' is deprecated and will be removed in a future version. Use 'v.weight' instead.\n",
" w_v = block.attn.W_V\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:579: FutureWarning: Hook 'W_Q' is deprecated and will be removed in a future version. Use 'q.weight' instead.\n",
" w_q = block.attn.W_Q\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:566: FutureWarning: Hook 'W_K' is deprecated and will be removed in a future version. Use 'k.weight' instead.\n",
" w_k = block.attn.W_K\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:605: FutureWarning: Hook 'W_O' is deprecated and will be removed in a future version. Use 'o.weight' instead.\n",
" w_o = block.attn.W_O\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:616: FutureWarning: Hook 'W_in' is deprecated and will be removed in a future version. Use 'in.weight' instead.\n",
" return torch.stack([block.mlp.W_in for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:632: FutureWarning: Hook 'W_out' is deprecated and will be removed in a future version. Use 'out.weight' instead.\n",
" return torch.stack([block.mlp.W_out for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:637: FutureWarning: Hook 'b_K' is deprecated and will be removed in a future version. Use 'k.bias' instead.\n",
" return torch.stack([block.attn.b_K for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:652: FutureWarning: Hook 'b_O' is deprecated and will be removed in a future version. Use 'o.bias' instead.\n",
" return torch.stack([block.attn.b_O for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:642: FutureWarning: Hook 'b_Q' is deprecated and will be removed in a future version. Use 'q.bias' instead.\n",
" return torch.stack([block.attn.b_Q for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:647: FutureWarning: Hook 'b_V' is deprecated and will be removed in a future version. Use 'v.bias' instead.\n",
" return torch.stack([block.attn.b_V for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:657: FutureWarning: Hook 'b_in' is deprecated and will be removed in a future version. Use 'in.bias' instead.\n",
" return torch.stack([block.mlp.b_in for block in self.blocks], dim=0)\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:662: FutureWarning: Hook 'b_out' is deprecated and will be removed in a future version. Use 'out.bias' instead.\n",
" return torch.stack([block.mlp.b_out for block in self.blocks], dim=0)\n",
"`loss_type=None` was set in the config but it is unrecognised.Using the default loss: `ForCausalLMLoss`.\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_resid_pre' is deprecated and will be removed in a future version. Use 'hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_resid_mid' is deprecated and will be removed in a future version. Use 'attn.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_resid_post' is deprecated and will be removed in a future version. Use 'hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_attn_in' is deprecated and will be removed in a future version. Use 'attn.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_attn_out' is deprecated and will be removed in a future version. Use 'attn.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_q_input' is deprecated and will be removed in a future version. Use 'attn.q.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_k_input' is deprecated and will be removed in a future version. Use 'attn.k.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_v_input' is deprecated and will be removed in a future version. Use 'attn.v.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_mlp_in' is deprecated and will be removed in a future version. Use 'mlp.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_mlp_out' is deprecated and will be removed in a future version. Use 'mlp.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_normalized' is deprecated and will be removed in a future version. Use 'hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_scale' is deprecated and will be removed in a future version. Use 'hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_result' is deprecated and will be removed in a future version. Use 'hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_q' is deprecated and will be removed in a future version. Use 'q.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_k' is deprecated and will be removed in a future version. Use 'k.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_v' is deprecated and will be removed in a future version. Use 'v.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_z' is deprecated and will be removed in a future version. Use 'hook_hidden_states' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_pre' is deprecated and will be removed in a future version. Use 'in.hook_out' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n",
"/Users/bryce/Projects/Lingwave/TransformerLens/transformer_lens/model_bridge/bridge.py:156: FutureWarning: Hook 'hook_post' is deprecated and will be removed in a future version. Use 'out.hook_in' instead.\n",
" component_hooks = mod.get_hooks() # type: ignore\n"
]
}
],
"source": [
"# NBVAL_IGNORE_OUTPUT\n",
"model_name = \"gpt2\"\n",
"model = TransformerBridge.boot_transformers(model_name)\n",
"model.enable_compatibility_mode()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def get_neuron_acts(text, layer, neuron_index):\n",
" # Hacky way to get out state from a single hook - we have a single element list and edit that list within the hook.\n",
" cache = {}\n",
"\n",
" def caching_hook(act, hook):\n",
" cache[\"activation\"] = act[0, :, neuron_index]\n",
"\n",
" model.run_with_hooks(\n",
" text, fwd_hooks=[(f\"blocks.{layer}.mlp.hook_post\", caching_hook)]\n",
" )\n",
" return to_numpy(cache[\"activation\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can run this function and verify that it gives vaguely sensible outputs"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['<|endoftext|>', 'The', ' following', ' is', ' a', ' list', ' of', ' powers', ' of', ' 10', ':', ' 1', ',', ' 10', ',', ' 100', ',', ' 1000', ',', ' 10000', ',', ' 100', '000', ',', ' 100', '0000', ',', ' 100', '00000']\n"
]
}
],
"source": [
"default_layer = 9\n",
"default_neuron_index = 652\n",
"default_text = \"The following is a list of powers of 10: 1, 10, 100, 1000, 10000, 100000, 1000000, 10000000\"\n",
"print(model.to_str_tokens(default_text))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[-0.08643512 -0.14071988 -0.1039816 -0.12390755 -0.04058974 -0.11064906\n",
" -0.05189841 -0.1127614 -0.0690546 -0.11189383 -0.030592 -0.10336886\n",
" -0.04322351 1.5935613 -0.14205799 2.511661 -0.1331642 2.5196698\n",
" -0.11360849 3.076527 -0.11637434 0.5393868 2.3499725 -0.14952122\n",
" -0.16476354 1.944909 -0.13690136 -0.08802476 2.184888 ]\n"
]
}
],
"source": [
"# NBVAL_IGNORE_OUTPUT\n",
"print(get_neuron_acts(default_text, default_layer, default_neuron_index))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualizing Model Activations\n",
"\n",
"We now write some code to visualize the neuron activations on some text - we're going to hack something together which just does some string processing to make an HTML string, with each token element colored according to the intensity neuron activation. We normalize the neuron activations so they all lie in [0, 1]. You can do much better, but this is a useful proof of concept of what \"just hack stuff together\" can look like!\n",
"\n",
"I'll be keeping neuron 562 in layer 9 as a running example, as it seems to activate strongly on powers of 10.\n",
"\n",
"Note that this visualization is very sensitive to `max_val` and `min_val`! You can tune those to whatever seems reasonable for the distribution of neuron activations you care about - I generally default to `min_val=0` and `max_val` as the max activation across the dataset."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# This is some CSS (tells us what style )to give each token a thin gray border, to make it easy to see token separation\n",
"style_string = \"\"\"<style> \n",
" span.token {\n",
" border: 1px solid rgb(123, 123, 123)\n",
" } \n",
" </style>\"\"\"\n",
"\n",
"\n",
"def calculate_color(val, max_val, min_val):\n",
" # Hacky code that takes in a value val in range [min_val, max_val], normalizes it to [0, 1] and returns a color which interpolates between slightly off-white and red (0 = white, 1 = red)\n",
" # We return a string of the form \"rgb(240, 240, 240)\" which is a color CSS knows\n",
" normalized_val = (val - min_val) / max_val\n",
" return f\"rgb(240, {240*(1-normalized_val)}, {240*(1-normalized_val)})\"\n",
"\n",
"\n",
"def basic_neuron_vis(text, layer, neuron_index, max_val=None, min_val=None):\n",
" \"\"\"\n",
" text: The text to visualize\n",
" layer: The layer index\n",
" neuron_index: The neuron index\n",
" max_val: The top end of our activation range, defaults to the maximum activation\n",
" min_val: The top end of our activation range, defaults to the minimum activation\n",
"\n",
" Returns a string of HTML that displays the text with each token colored according to its activation\n",
"\n",
" Note: It's useful to be able to input a fixed max_val and min_val, because otherwise the colors will change as you edit the text, which is annoying.\n",
" \"\"\"\n",
" if layer is None:\n",
" return \"Please select a Layer\"\n",
" if neuron_index is None:\n",
" return \"Please select a Neuron\"\n",
" acts = get_neuron_acts(text, layer, neuron_index)\n",
" act_max = acts.max()\n",
" act_min = acts.min()\n",
" # Defaults to the max and min of the activations\n",
" if max_val is None:\n",
" max_val = act_max\n",
" if min_val is None:\n",
" min_val = act_min\n",
" # We want to make a list of HTML strings to concatenate into our final HTML string\n",
" # We first add the style to make each token element have a nice border\n",
" htmls = [style_string]\n",
" # We then add some text to tell us what layer and neuron we're looking at - we're just dealing with strings and can use f-strings as normal\n",
" # h4 means \"small heading\"\n",
" htmls.append(f\"<h4>Layer: <b>{layer}</b>. Neuron Index: <b>{neuron_index}</b></h4>\")\n",
" # We then add a line telling us the limits of our range\n",
" htmls.append(\n",
" f\"<h4>Max Range: <b>{max_val:.4f}</b>. Min Range: <b>{min_val:.4f}</b></h4>\"\n",
" )\n",
" # If we added a custom range, print a line telling us the range of our activations too.\n",
" if act_max != max_val or act_min != min_val:\n",
" htmls.append(\n",
" f\"<h4>Custom Range Set. Max Act: <b>{act_max:.4f}</b>. Min Act: <b>{act_min:.4f}</b></h4>\"\n",
" )\n",
" # Convert the text to a list of tokens\n",
" str_tokens = model.to_str_tokens(text)\n",
" for tok, act in zip(str_tokens, acts):\n",
" # A span is an HTML element that lets us style a part of a string (and remains on the same line by default)\n",
" # We set the background color of the span to be the color we calculated from the activation\n",
" # We set the contents of the span to be the token\n",
" htmls.append(\n",
" f\"<span class='token' style='background-color:{calculate_color(act, max_val, min_val)}' >{tok}</span>\"\n",
" )\n",
"\n",
" return \"\".join(htmls)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Displayed HTML\n"
]
},
{
"data": {
"text/html": [
"<style> \n",
" span.token {\n",
" border: 1px solid rgb(123, 123, 123)\n",
" } \n",
" </style><h4>Layer: <b>9</b>. Neuron Index: <b>652</b></h4><h4>Max Range: <b>4.0000</b>. Min Range: <b>0.0000</b></h4><h4>Custom Range Set. Max Act: <b>3.0765</b>. Min Act: <b>-0.1648</b></h4><span class='token' style='background-color:rgb(240, 245.1861074566841, 245.1861074566841)' ><|endoftext|></span><span class='token' style='background-color:rgb(240, 248.44319254159927, 248.44319254159927)' >The</span><span class='token' style='background-color:rgb(240, 246.23889595270157, 246.23889595270157)' > following</span><span class='token' style='background-color:rgb(240, 247.43445307016373, 247.43445307016373)' > is</span><span class='token' style='background-color:rgb(240, 242.43538431823254, 242.43538431823254)' > a</span><span class='token' style='background-color:rgb(240, 246.63894340395927, 246.63894340395927)' > list</span><span class='token' style='background-color:rgb(240, 243.11390474438667, 243.11390474438667)' > of</span><span class='token' style='background-color:rgb(240, 246.76568403840065, 246.76568403840065)' > powers</span><span class='token' style='background-color:rgb(240, 244.14327576756477, 244.14327576756477)' > of</span><span class='token' style='background-color:rgb(240, 246.7136299610138, 246.7136299610138)' > 10</span><span class='token' style='background-color:rgb(240, 241.83551989495754, 241.83551989495754)' >:</span><span class='token' style='background-color:rgb(240, 246.2021318078041, 246.2021318078041)' > 1</span><span class='token' style='background-color:rgb(240, 242.59341068565845, 242.59341068565845)' >,</span><span class='token' style='background-color:rgb(240, 144.38632249832153, 144.38632249832153)' > 10</span><span class='token' style='background-color:rgb(240, 248.52347910404205, 248.52347910404205)' >,</span><span class='token' style='background-color:rgb(240, 89.30033683776855, 89.30033683776855)' > 100</span><span class='token' style='background-color:rgb(240, 247.98985183238983, 247.98985183238983)' >,</span><span class='token' style='background-color:rgb(240, 88.81981372833252, 88.81981372833252)' > 1000</span><span class='token' style='background-color:rgb(240, 246.81650966405869, 246.81650966405869)' >,</span><span class='token' style='background-color:rgb(240, 55.40837287902832, 55.40837287902832)' > 10000</span><span class='token' style='background-color:rgb(240, 246.98246017098427, 246.98246017098427)' >,</span><span class='token' style='background-color:rgb(240, 207.63679146766663, 207.63679146766663)' > 100</span><span class='token' style='background-color:rgb(240, 99.0016508102417, 99.0016508102417)' >000</span><span class='token' style='background-color:rgb(240, 248.9712730050087, 248.9712730050087)' >,</span><span class='token' style='background-color:rgb(240, 249.88581240177155, 249.88581240177155)' > 100</span><span class='token' style='background-color:rgb(240, 123.30546140670776, 123.30546140670776)' >0000</span><span class='token' style='background-color:rgb(240, 248.21408182382584, 248.21408182382584)' >,</span><span class='token' style='background-color:rgb(240, 245.28148546814919, 245.28148546814919)' > 100</span><span class='token' style='background-color:rgb(240, 108.9067268371582, 108.9067268371582)' >00000</span>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"HTML String - it's just raw HTML code!\n",
"<style> \n",
" span.token {\n",
" border: 1px solid rgb(123, 123, 123)\n",
" } \n",
" </style><h4>Layer: <b>9</b>. Neuron Index: <b>652</b></h4><h4>Max Range: <b>4.0000</b>. Min Range: <b>0.0000</b></h4><h4>Custom Range Set. Max Act: <b>3.0765</b>. Min Act: <b>-0.1648</b></h4><span class='token' style='background-color:rgb(240, 245.1861074566841, 245.1861074566841)' ><|endoftext|></span><span class='token' style='background-color:rgb(240, 248.44319254159927, 248.44319254159927)' >The</span><span class='token' style='background-color:rgb(240, 246.23889595270157, 246.23889595270157)' > following</span><span class='token' style='background-color:rgb(240, 247.43445307016373, 247.43445307016373)' > is</span><span class='token' style='background-color:rgb(240, 242.43538431823254, 242.43538431823254)' > a</span><span class='token' style='background-color:rgb(240, 246.63894340395927, 246.63894340395927)' > list</span><span class='token' style='background-color:rgb(240, 243.11390474438667, 243.11390474438667)' > of</span><span class='token' style='background-color:rgb(240, 246.76568403840065, 246.76568403840065)' > powers</span><span class='token' style='background-color:rgb(240, 244.14327576756477, 244.14327576756477)' > of</span><span class='token' style='background-color:rgb(240, 246.7136299610138, 246.7136299610138)' > 10</span><span class='token' style='background-color:rgb(240, 241.83551989495754, 241.83551989495754)' >:</span><span class='token' style='background-color:rgb(240, 246.2021318078041, 246.2021318078041)' > 1</span><span class='token' style='background-color:rgb(240, 242.59341068565845, 242.59341068565845)' >,</span><span class='token' style='background-color:rgb(240, 144.38632249832153, 144.38632249832153)' > 10</span><span class='token' style='background-color:rgb(240, 248.52347910404205, 248.52347910404205)' >,</span><span class='token' style='background-color:rgb(240, 89.30033683776855, 89.30033683776855)' > 100</span><span class='token' style='background-color:rgb(240, 247.98985183238983, 247.98985183238983)' >,</span><span class='token' style='background-color:rgb(240, 88.81981372833252, 88.81981372833252)' > 1000</span><span class='token' style='background-color:rgb(240, 246.81650966405869, 246.81650966405869)' >,</span><span class='token' style='background-color:rgb(240, 55.40837287902832, 55.40837287902832)' > 10000</span><span class='token' style='background-color:rgb(240, 246.98246017098427, 246.98246017098427)' >,</span><span class='token' style='background-color:rgb(240, 207.63679146766663, 207.63679146766663)' > 100</span><span class='token' style='background-color:rgb(240, 99.0016508102417, 99.0016508102417)' >000</span><span class='token' style='background-color:rgb(240, 248.9712730050087, 248.9712730050087)' >,</span><span class='token' style='background-color:rgb(240, 249.88581240177155, 249.88581240177155)' > 100</span><span class='token' style='background-color:rgb(240, 123.30546140670776, 123.30546140670776)' >0000</span><span class='token' style='background-color:rgb(240, 248.21408182382584, 248.21408182382584)' >,</span><span class='token' style='background-color:rgb(240, 245.28148546814919, 245.28148546814919)' > 100</span><span class='token' style='background-color:rgb(240, 108.9067268371582, 108.9067268371582)' >00000</span>\n"
]
}
],
"source": [
"# NBVAL_IGNORE_OUTPUT\n",
"# The function outputs a string of HTML\n",
"default_max_val = 4.0\n",
"default_min_val = 0.0\n",
"default_html_string = basic_neuron_vis(\n",
" default_text,\n",
" default_layer,\n",
" default_neuron_index,\n",
" max_val=default_max_val,\n",
" min_val=default_min_val,\n",
")\n",
"\n",
"# IPython lets us display HTML\n",
"print(\"Displayed HTML\")\n",
"display(HTML(default_html_string))\n",
"\n",
"# We can also print the string directly\n",
"print(\"HTML String - it's just raw HTML code!\")\n",
"print(default_html_string)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create Interactive UI\n",
"\n",
"We now put all these together to create an interactive visualization in Gradio! \n",
"\n",
"The internal format is that there's a bunch of elements - Textboxes, Numbers, etc which the user can interact with and which return strings and numbers. And we can also define output elements that just display things - in this case, one which takes in an arbitrary HTML string. We call `input.change(update_function, inputs, output)` - this says \"if that input element changes, run the update function on the value of each of the elements in `inputs` and set the value of `output` to the output of the function\". As a bonus, this gives us live interactivity!\n",
"\n",
"This is also more complex than a typical Gradio intro example - I wanted to use custom HTML to display the nice colours, which made things much messier! Normally you could just make `out` into another Textbox and pass it a string."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"# The `with gr.Blocks() as demo:` syntax just creates a variable called demo containing all these components\n",
"with gr.Blocks() as demo:\n",
" gr.HTML(value=f\"Hacky Interactive Neuroscope for {model_name}\")\n",
" # The input elements\n",
" with gr.Row():\n",
" with gr.Column():\n",
" text = gr.Textbox(label=\"Text\", value=default_text)\n",
" # Precision=0 makes it an int, otherwise it's a float\n",
" # Value sets the initial default value\n",
" layer = gr.Number(label=\"Layer\", value=default_layer, precision=0)\n",
" neuron_index = gr.Number(\n",
" label=\"Neuron Index\", value=default_neuron_index, precision=0\n",
" )\n",
" # If empty, these two map to None\n",
" max_val = gr.Number(label=\"Max Value\", value=default_max_val)\n",
" min_val = gr.Number(label=\"Min Value\", value=default_min_val)\n",
" inputs = [text, layer, neuron_index, max_val, min_val]\n",
" with gr.Column():\n",
" # The output element\n",
" out = gr.HTML(label=\"Neuron Acts\", value=default_html_string)\n",
" for inp in inputs:\n",
" inp.change(basic_neuron_vis, inputs, out)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can now launch our demo element, and we're done! The setting share=True even gives you a public link to the demo (though it just redirects to the backend run by this notebook, and will go away once you turn the notebook off!) Sharing makes it much slower, and can be turned off if you aren't in a colab.\n",
"\n",
"**Exercise:** Explore where this neuron does and does not activate. Is it just powers of ten? Just comma separated numbers? Numbers in any particular sequence?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# NBVAL_SKIP\n",
"demo.launch(share=True, height=1000)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}