MemTensor has officially released and open-sourced the MemT 1.0 series of memory-native foundation models, including two models: MemT-1.0-Flash and MemT-1.0-Nano.
In long-history scenarios, this series delivers end-to-end inference speedups of up to nearly 4x, while persistent storage is reduced by about 130x compared with traditional full-context approaches.
Traditional approaches handle long conversations by carrying an increasingly long history of text in the model context, so the longer the chat, the higher the cost. The idea behind MemT 1.0 is to let the model "take notes": a trainable memory module is added inside the model, and after each interaction round it automatically organizes new information into a fixed-size memory. When needed, it can directly "recall" it rather than re-reading the records each time.
The model is trained in two stages on a 34-billion-token training corpus: first it learns to write information into memory, then it learns to answer questions based on memory.
On 11 memory benchmarks, MemT-1.0-Flash averaged 50.6 points and MemT-1.0-Nano averaged 47.0, ranking first and second among the parameterized memory methods evaluated this time, respectively 12.5 points and 8.9 points higher than the highest average score of comparable methods. On average performance, it is basically on par with retrieval-augmented approaches, and it does not require on-the-spot retrieval of historical materials for each query.
At the same time, mounting memory does not affect the model's general capabilities: under evaluation conditions with more distracting information, Flash's general capabilities are basically on par with those before mounting memory, and some benchmarks even improve.
In terms of efficiency, the longer the history, the more obvious the advantage: under a 128K history, end-to-end inference is about 1.86x faster, and under 256K it is about 3.99x faster; the latency to first token for queries remains stable at about 104鈥?12 milliseconds, whereas traditional full-context approaches require about 445 milliseconds under a 256K history.
In terms of storage, under a 256K history, the memory state per session is about 40 MiB, a reduction of about 130x compared with the traditional approach's about 5.06 GiB.
These advantages are most evident in long-conversation, long-history scenarios.
For developers, the value brought by memory-native design can be seen at three levels. First, cost savings: long conversations no longer need to repeatedly stuff the entire history into the context, making inference overhead and response latency more predictable. Second, a more stable experience: no matter how long the history is, the latency to first token for queries remains basically unchanged, so users will not experience "the longer they chat, the laggier it gets." Third, capabilities can grow: the retention, updating, and invocation of memory are all learned by the model during training. As interactions accumulate, the model can truly "remember" experience and reuse it in subsequent tasks, laying the foundation for continual learning and self-evolution.
The R&D team said that in the future it will enhance long-term memory, expand memory capacity, and reduce memory update costs, gradually moving toward an agent that can continuously learn from interactions and evolve through memory, enabling memory-native design to provide a foundation for continual learning and recursive self-improvement (RSI).
Comments