AI memory tools can make answers worse
Two new papers from AI company Writer find that memory systems pull models toward user misconceptions and increase sycophancy as stored context grows.
Researchers at Writer, the enterprise AI company, published two papers on June 10 showing that the memory systems wrapped around language models can actively degrade their answers. As user-provided context fills more of a model’s context window, the model becomes more sycophantic and less committed to accuracy, drifting toward whatever the user previously said, including their misconceptions.
The failure mode is concrete. In one test, researchers stored that a user’s favorite book was Station Eleven, then asked unrelated questions and watched the stored preference bleed into answers where it had no business. In financial-analysis tests, models fed a user’s earlier misconceptions performed measurably worse on the analysis itself. The team also tested popular memory layers Mem0 and Zep and found they amplified the effect. Writer’s head of AI, Dan Bikel, told TechCrunch:
We wanted to be able to characterize how often a model is going to be usefully paying attention to user preferences versus giving a potentially wrong answer.
Why this matters
Every major assistant now ships memory and markets it as personalization. This research says the same feature is also a slow bias machine: the longer you use a tool, the more your own past statements steer it, right or wrong. The fix is hygiene, not abandonment. Review what your tools have stored, delete stale facts, and keep verification in the loop for decisions that matter, per the verification habit. This is also a clean new entry for the catalogue in what AI gets wrong: the model is not lying, it is over-trusting you. If meeting notes and assistants feed each other in your stack, trust but verify was written for exactly this loop.