Language models are increasingly used in text generation, decision support, and automated interaction, creating a need for localized, reversible, and selective control of their behavior. Activation steering demonstrates that internal interventions can alter the behavior of frozen models, but how to select and evaluate compact, multi-site control interfaces remains an open question. I introduce sparse internal control, a framework for steering target behavior by applying inference-time interventions to a small set of internal model nodes.
Formulation
A local target-controllability formulation characterizes candidate site--direction pairs by their behavioral effects and specifies constraints on:
- Target recovery
- Off-target preservation
- Intervention energy
Selectors
Two complementary selectors instantiate this formulation:
- Driver-OMP selects nodes for sparse reconstruction of a desired behavioral displacement.
- Coverage-Driver favors stable coverage across prompts.
Evaluation protocol
I propose a six-axis control-evaluation protocol measuring:
- Reachability
- Signed reversal
- Dose response
- Feedback controllability
- Off-target preservation
- Held-out reuse
Experimental evidence
Exploratory experiments on:
- Indirect object identification
- MMLU multiple-choice question answering
- Sentiment steering
- Refusal steering
provide evidence for the feasibility of steering task-defined behavior through sparse internal interventions.
Characterization beyond target movement
The six-axis evaluation characterizes these interventions beyond target movement, revealing distinct control properties that depend on the task and available direction information.
Conclusion
Together, the formulation, reference algorithms, and evaluation protocol provide a foundation for the systematic study and development of internal control methods for language models.