Inside BSF: How Goodfire unlocked a new level of AI interpretability and control
Current interpretability tools fracture continuous concepts into isolated points. Goodfire's new approach preserves the full shape of AI reasoning.
Deep neural networks have traditionally been black boxes. We rely on them for complex tasks, yet their internal decision-making processes remain opaque. The AI community has made significant progress in cracking open the mysteries of these models using mechanistic interpretability methods like Sparse Autoencoders (SAEs).
But a new technique introduced by researchers at Goodfire AI takes these current achievements one step further.
Dubbed Block-Sparse Featurizers (BSF), the framework introduces a new method to interpret model activations. Instead of decomposing model thoughts into isolated one-dimensional directions like SAEs, BSF decomposes them into multidimensional subspaces. This geometric shift provides a fine-grained explanation of model internals and can pave the way for better steerability and control in some AI applications.
The geometry of AI thought: Manifolds vs. 1D
SAEs are the current gold standard of mechanistic interpretability. They are trained by taking dense, unreadable neural network activations and projecting them into a much wider, overcomplete layer. During training, a sparsity penalty forces the SAE to reconstruct the original data using only a tiny handful of active 1D directions at a time. The goal is to isolate individual concepts into human-readable features.
However, the researchers point out, “recent work points to more structure in these models than a single direction allows.”
The problem is that complex concepts in neural networks, especially in vision models, naturally reside on continuous, low-dimensional geometric spaces known as manifolds. These manifolds form dense, connected clouds of data representing smooth transitions in the real world, like the changing angle of a face or the shifting gradient of a shadow.
To picture a manifold, consider how a vision model processes a curved line. If you slowly rotate that curve 360 degrees, the model’s activations smoothly shift along a continuous, circular path: the manifold.
Because SAEs are trained to enforce sparsity on individual 1D lines, they force these continuous clouds of meaning into rigid, isolated points. This leads to feature splitting, where a unified concept is fractured into disjointed parts. Instead of capturing the full rotation of the curve as one connected concept, the SAE assigns a separate, isolated line for a 10-degree curve, another for a 20-degree curve, and so on.
The limitation of SAEs enforcing linear assumptions on nonlinear models is a well-known issue across recent interpretability literature. However, seeing how this mathematical limitation fractures real-world concepts provided a striking realization for the research team.
“One very cool ‘aha moment’ at Goodfire came while working on our paper on how SAEs capture neural geometry, where we saw really clean examples of how SAE features ‘shatter’ manifold features like temperature, years, and colors,” Michael Byun, Member of Technical Staff at Goodfire, said in comments provided to TechTalks. “While it wasn’t unexpected, it made visceral how the 1D features were giving us a fractured view into much richer structures inside the model!”
Thinking in blocks with BSF
Goodfire’s BSF method takes a different approach. Instead of enforcing sparsity on individual 1D features, it assumes that a small number of multidimensional blocks are active at any given time.
A standard 1D feature acts like an isolated dial that captures one highly specific variant of a concept. A block represents a multidimensional subspace that encompasses a continuous range of variations for a concept within one cohesive structure.
BSFs use an autoencoder architecture similar to SAEs but with a different mathematical incentive. In standard SAEs, the model is penalized for every single direction it activates. BSFs apply this penalty at the block level.
“Unlike SAEs, which assume that a small (sparse) number of 1D features are firing at any given time, BSFs assume that a small (sparse) number of blocks are firing at any given time. That is, we enforce sparsity at the block level,” the researchers write.
Once the network activates a block, it can freely use all the dimensions within that block. This trains the model to cluster correlated, continuously varying concepts together. By enforcing sparsity at this block level, the featurizer captures the entire region of a concept.
Returning to the rotating curve example, a BSF does not fracture the rotation into separate 10-degree buckets. Instead, it assigns a single multidimensional block to the entire geometric space of the curve. As the line rotates in the image, the BSF does not jump between different features. The activation simply shifts coordinates within that one active block, smoothly tracking the continuous physical movement.
The concept of block-sparsity is a well-studied mathematical principle dating back to 2006 dictionary learning research. Scaling it to work on massive, modern vision models required significant engineering breakthroughs to remove restrictive legacy assumptions that previously held the technique back.
“On the research and methodological side, we ‘fixed’ things by removing unnecessary architecture assumptions, such as the positivity constraint on activations, and figuring out a good protocol for identifying which block or subspace should be used for reconstruction,” Byun said.
Goodfire proved the general principle of block-sparsity by successfully testing three distinct variants: Vanilla BSF, Grassmannian BSF, and Group Lasso BSF.
The stable rank sweet spot
Trading 1D features for multidimensional blocks might seem like an addition of complexity that defeats the human-readable goal of interpretability.
To address this, Goodfire conducted a stable-rank study to measure the actual intrinsic dimensionality of these newly discovered features. The researchers found that the recovered concepts are typically only two- to four-dimensional.
This represents a sweet spot. The features are rich enough to capture the true geometry of the model’s thought process, yet simple enough for human researchers to visualize and understand.
“Once the featurizer is trained, we can study the stable rank of each block – the effective number of dimensions it uses,” the researchers write. “In practice, though, we find that a block rarely exceeds a stable rank of 4. This is perhaps unsurprising: visual concepts are 2D projections of a three-dimensional scene.”
Put to the test: MDL and DINOv3 findings
To prove BSFs map the true shape of the data, Goodfire evaluated the method using Minimum Description Length (MDL). MDL is a principle from information theory stating that the best model for a dataset is the one that compresses it the most.
Because BSFs match the natural shape of the data, they achieve a lower MDL than 1D SAEs, proving they describe the model’s internal workings more efficiently.
“Quantitatively, we find that BSFs yield a more faithful and efficient account of the model: they better explain the original model activations with a lower minimum description length,” the researchers write.
When applied to the DINOv3 vision model, BSF uncovered novel manifolds that abstract away from physical objects, isolating concepts like shadows and lighting. A prime example is how BSF maps an arch. A standard SAE attempts to compress the dense data cloud representing an arch, resulting in a disjointed view that captures isolated slivers without physical context.
BSF keeps the entire cloud intact, resulting in a region with interpretable internal structure. Moving through this block continuously traces the physical parts of an arch in the real world, from the bottom pillar, smoothly up through the middle, to the top of the curve.
“A standard SAE squashes each cloud down to a single direction... A BSF keeps the whole cloud instead, treating that entire region as the concept,” the authors write.
Cleanly separating an object’s spatial identity from other variables has immediate practical value. It allows developers to debug critical computer vision systems, such as autonomous driving, ensuring the AI does not hallucinate physical barriers from weird shadows.
In physical systems like robotics, BSFs have uncovered exact spatial and topographic representations, mapping a robotic arm perfectly in activation space. While this visibility does not automatically make current models safe, it makes safety inspection and alignment realistic.
“It probably shouldn’t change your opinion of how safe or reliable models are currently — that’s ultimately a question of model behavior,” Byun said. “What it should change is your understanding of how tractable it is to understand and control that behavior. If we can deeply understand the mechanisms a model uses to understand the world and generate its outputs... then we could be more confident in the safety and reliability of the models we build.”
Steering generative AI
Interpretability serves as an engineering tool for actively steering AI behavior. BSFs allow for highly interpretable, fine-grained control over image generation in diffusion models.
In Stable Diffusion XL, Goodfire found a single multidimensional subspace that captures the concept of a pretzel. If this were a standard 1D feature, manipulating it would act like a simple volume knob, making the image look more like a pretzel in a generic way. Because BSF represents this as a multidimensional block, it acts more like a 2D or 3D joystick.
“A single subspace found by a BSF in SDXL captures the ‘pretzel’ concept; not as one fixed shape but as a whole region of possible pretzels. Walking through this region point by point generates a different pretzel each time. They are twisted, braided, and knotted differently,” the researchers write.
Activation steering expands the technology’s practical implications beyond image generation, offering a highly targeted alternative to coarse fine-tuning in non-text domains.
“The current dominant approach to post-training models — training on task-specific data — is quite coarse and often doesn’t result in the behavior you want,” Byun says. “Effective steering-based interventions mean we can understand, debug, and post-train models much better — including in domains where prompting is difficult or impossible, like DNA models or robotics models.”
No golden hammers: Caveats and the future
Goodfire acknowledges that BSF is not a universal solution for all AI architectures.
The physical world is fundamentally continuous and spatial, characterized by smooth lighting gradients and object rotations. This is why multi-dimensional blocks capture visual data accurately. Human language, however, is highly discrete, symbolic, and combinatorial. A word or grammatical rule functions more like an isolated mathematical point, making standard 1D SAEs highly effective for LLMs. Audio introduces strict temporal dynamics and overlapping frequencies, requiring yet another geometric framework.
As researchers look to entirely different modalities, they will need to hunt for tailored geometric structures, such as tracking time in video or musical keys in audio. Goodfire is already exploring these directions, recently uncovering rich temporal structures in language models through complementary research on Temporal Feature Analysis.
“One example might be video, audio, or world models, which we expect to have internal representations of time and how things change over time, unlike the static image models we study in this paper,” Byun said. “Other domains might also have different symmetries: for example, we might expect a music generation model to have conserved representations for the same melody played in different keys.”