Meta, MIT and UW researchers unveil 'Context Language Models' that manage their own memory
The approach lets a model decide what to keep, edit or drop from its context; the authors report accuracy gains at lower compute but flag data-loss and prompt-injection risks.
Researchers from Meta, MIT and the University of Washington have described Context Language Models — a method that lets a language model manage its own working context instead of simply appending everything to it, AI and Tech News reported on 11 October.
Reported results
- Zero-shot models scored 11.4% higher accuracy with 21.5% fewer floating-point operations on the BrowseComp-Plus benchmark.
- Reinforcement learning lifted Qwen3.5-9B on BrowseComp-Plus from 28.8% to 42.5% while using about 12% less compute.
- In-context learning improved ContextBench accuracy by up to 35.9 percentage points.
Open problems
The authors themselves list caveats: a model may discard information it later needs; an editable context can become a new channel through which prompt injections persist across turns; and more control over context does not guarantee better decisions. The results are the researchers’ own benchmark evaluations and have not yet been independently replicated.
Also available in: Саха тылаРусскийEspañol中文Portuguêsالعربية