LongMem: displaceable longterm artificial memory

LongMem is a framework that proposes augmenting language models with long-term memory so they can use long past context despite fixed input-length limits. The authors (Weizhi Wang, Li Dong, Hao Cheng, Xiaodong Liu, Xifeng Yan, Jianfeng Gao and Furu Wei) design a decoupled architecture in which the original backbone LLM stays frozen as a memory encoder, while an adaptive residual side-network retrieves and reads from the memory. The framework can enlarge long-form memory to 65k tokens, and the article reports that it outperforms strong long-context models on the ChapterBreak benchmark and improves memory-augmented in-context learning. The code is open-sourced.