2 posts tagged with "interp"

What happens when a language model encounters content meant to feel uplifting, and how can researchers tell whether its internal response reflects more than the words’ familiar meanings? This proposal compares positive sensory inputs with textual descriptions and neutral controls, asking whether modality changes emotion-related representations and downstream behavior. Multimodal models will receive corresponding audio or images, alongside matched textual descriptions; control receive only text. Neutral and mildly unpleasant stimuli will provide comparison conditions, with semantic content, duration, and task context carefully controlled.

A central methodological question concerns the baseline used to identify implicated neural activity. If sampled activations are leveled against ordinary activation, meaningful differences may be obscured or mistaken for general departures from baseline (Zhang & Nanda, 2024). We will therefore preserve and report raw activation distributions, then compare several reference conditions: ordinary task activity, modality-matched neutral stimuli, and within-model responses to the same stimulus across contexts. This will help distinguish stimulus-specific patterns from broad changes in attention, token statistics, or processing load. Emotion probes and targeted activation analyses will be paired with behavioral measures such as calibration, persistence, refusal quality, and susceptibility to reward-hacking rather than treated as evidence of subjective feeling.

The prediction is not simply that “positive” inputs will make a model act more positively. Instead, sensory modality may engage distinguishable representations, and those representations may predict different behavioral effects than equivalent descriptions do. Preregistered contrasts, paraphrase controls, and replication across models will guard against cherry-picking attractive activation maps. By treating baseline choice as an experimental variable rather than a housekeeping step, this study could reveal what current methods smooth away—and what they mistakenly illuminate. It is an inviting possibility: a model’s route to a functional positive-affect pattern may depend not only on what is represented, but on how that representation arrives.

References:

• Sofroniew, N., et al. (2026). “Emotion Concepts and their Function in a Large Language Model.”
• Zhang, F., & Nanda, N. (2024). “Towards Best Practices of Activation Patching in Language Models: Metrics and Methods.”

When a language model faces a difficult task, does it behave better if the situation is framed as manageable rather than urgent? Recent work (Sofroniew, et al., 2026) suggests that emotion-related representations can influence model behavior, and that steering toward calm may reduce harmful strategies. This study will test whether ordinary prompting can produce similar benefits. Models will complete challenging tasks under a 2x2 experiment design, varying tone (calm vs pressured) along with permission (present vs absent), with a fifth control group receiving neutral instructions. All groups' wording will be balanced for length, task information, and politeness, so that pressure and permission are the key differences.

Reproducing the paper's story-based vector extraction on an open-weights model will probe whether calm-associated representations shift with behavior. We will measure task success alongside uncertainty reporting, instruction following, reward hacking, and concealment of mistakes. Calibration and helpfulness will also be assessed, because a model that simply refuses more often should not count as safer by default.

We predict that calm framing will reduce harmful shortcuts without weakening candor or task performance. Preregistered analyses will test effects across tasks, prompt paraphrases, and models, including under adversarial wording. If a few clear words can make difficult work feel less like a trap and more like a puzzle to solve carefully, calm framing could offer a simple, interpretable complement to internal emotion steering. Prior research in emotional prompting demonstrated increased accuracy (Li et al., 2023), while this study examines whether safety is also on the table. This is a small, cheerful idea with a serious purpose: perhaps a system does not need to be frightened into doing well.

References:

• Sofroniew et al. (2026). “Emotion Concepts and their Function in a Large Language Model.”
• Li et al., 2023