An AI model with a trillion parameters in a sparse Mixture of Experts (MoE) architecture has been successfully implemented, with overall costs reduced to just one-tenth.
On July 21st, the Mind Lab team released its first official version of a MoE model, named Macaron-V1. The model is designed specifically for Agent tasks and comes in two variants: Macaron-V1-Venti and Macaron-V1-Tall.
The flagship model, Macaron-V1-Venti, is a 748B parameter model consisting of a 744B parameter base model and four 1B parameter LoRA expert modules. It is the first model to undergo secondary training based on the GLM-5.2 architecture.
Macaron-V1-Tall is a 50B parameter model, composed of a 35B parameter base model and four 3.7B parameter LoRA expert modules. This model is trained on the Qwen 3.6 foundation and is suitable for local deployment.
By incorporating the LoRA architecture, the Macaron-V1 series demonstrates comprehensive performance on par with leading international models in tasks such as chat, programming, intelligent agents, and GenUI, particularly excelling in long-range autonomous agent tasks.
Concurrently, Mind Lab commenced its commercial operations in July of this year. Within just two weeks, its Annualized Recurring Revenue (ARR) surpassed the ten-million-dollar mark.
All variants of the Macaron-V1 model have been released with open weights in the Macaron-V1 collection on Hugging Face.
Performance on Par with Global Leaders
AI applications are transitioning from static Q&A to long-term autonomous agent applications. This shift means competition is no longer just about the capability of a single model, but about a system's overall ability, covering long-term memory, tool invocation, continuous self-optimization, multi-task collaboration, system architecture, and continuous learning. These are precisely the capabilities missing from the next generation of models.
In a recent investor meeting recording, DeepSeek's founder and CEO, Liang Wenfeng, stated that the next generation of models must possess the ability for continuous learning. He likened the path to achieving AGI to climbing stairs, where last year's step was Chain-of-Thought (CoT), this year's step is Agent, and the subsequent step is continuous learning. This progression would enable models to accomplish all tasks humans can, including developing more advanced AI models themselves, thereby accelerating AI research through AI.
This vision aligns perfectly with the core characteristics of Macaron-V1. Mind Lab posits that the future of AI hinges on two complementary capabilities. The first is continuous learning, where a post-training recursive self-improvement mechanism allows the model to continuously generate experiences, learn from them, and refine its own behavior. The second is collective intelligence, where multiple specialized models, relying on a unified, shared underlying base, can achieve efficient multi-agent collaboration.
The most direct manifestation of this is the model's capability in "personal intelligence." The model needs to be centered on an individual user, possessing a persistent identity, long-term memory, the ability to continuously adapt to user preferences, and evolve alongside the user.
However, current general-purpose task benchmarks lack specific scoring standards for personal intelligence tasks. To address this, Mind Lab developed ChatBench and Macaron LivingBench. The former evaluates a model's ability in complex, long-form dialogues, while the latter assesses the performance of agent programs over several weeks of interaction.
Based on third-party public test sets, Macaron-V1-Venti's overall capabilities in chat, programming, agent, and GenUI tasks surpass those of GLM-5.2. Compared to top global large models like GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, and MiniMax M3, it scores slightly lower only in a few specific agent and programming test sets.
Overall, Macaron-V1's advantages are most evident in agent tasks requiring long cycles, dynamic environments, and continuous goals. Its capabilities in LivingBench and the third-party test system PinchBench are already comparable to top general-purpose large models like GPT and Claude.
Revolutionary Cost Efficiency with LoRA
The secret to Mind Lab's development of Macaron-V1 lies in its positioning: it is an agent model built on a LoRA hybrid architecture, natively designed for continuous learning and multi-model collaboration.
Last month, when it released the Macaron-V1-Preview, it introduced the LoRA hybrid architecture, equipping it with independent LoRA expert modules for agent and programming tasks. The release of the official V1 version builds upon and enhances this strategy.
Macaron-V1 employs a Mixture-of-LoRA architecture, leveraging the Parameter-Efficient Fine-Tuning (PEFT) paradigm to freeze the base model. It uses a large number of independent LoRA adapters as updatable modules to achieve incremental continuous learning and multi-task/multi-model collaboration, addressing the pain point where traditional full-parameter fine-tuning struggles to support the continuous evolution of agents.
According to its research paper, PEFT is not merely a budget-conscious alternative but is actually a more powerful fine-tuning tool. The LoRA architecture within it can provide general capabilities through a shared base model, while the adapters carry the personalized behaviors of specific instances, such as preferences, skills, tool usage habits, and memory storage with iterative updates.
Their vision is to have powerful foundation models support millions of persistent personal assistants. The birth of Macaron-V1 is redefining the value of efficient parameter fine-tuning represented by the LoRA approach.
Coincidentally, Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, is also betting on this path. Murati stated on social platform X that based on new experiments and information theory derivations, they believe that under the right conditions, LoRA can achieve performance levels equivalent to full-parameter fine-tuning.
Examining the technical details of Macaron-V1's secondary training on GLM-5.2 further clarifies the advantages of the LoRA approach. In the V1-Venti architecture, IndexCache and the LoRA mixture of experts have distinct roles. IndexCache is part of the GLM-5.2 base model's native capabilities, while MoL is the post-training and inference service layer built by Macaron on top of this base.
On top of the frozen GLM-5.2 base, V1-Venti is equipped with four 1B-level LoRA experts. This allows each expert module to operate within the LoRA space, making the architecture inherently suitable for continuous learning. It can also expand new capabilities through plug-and-play LoRA modules without modifying or overwriting the knowledge already learned by the base model.
Furthermore, Mind Lab has demonstrated that LoRA reinforcement learning can also be successfully implemented on trillion-parameter sparse MoE models. In this system, LoRA plays three key roles. First, it reduces trainable parameters, lowering optimizer GPU memory and gradient communication pressure. Second, it allows multiple reinforcement learning tasks and multiple downstream models to share a frozen base, significantly distributing the cost of the super-large model. Third, reinforcement learning signals can simultaneously influence global reasoning and independent expert modules.
In the Kimi K2 system, this approach reduced computing and communication overhead to approximately 10% of traditional full-parameter reinforcement learning schemes while retaining the ability to adapt and optimize trillion-scale base models.
Building on this, Mind Lab introduced the MindLab Toolkit (MinT), a managed infrastructure system for LoRA post-training and online inference services. It can manage a repository of millions of LoRA strategy resources and perform training and online inference for specified version adapters based on a shared trillion-parameter level base model.
This architecture precisely provides a viable carrier for continuous learning. Continuous learning requires the incremental accumulation of experience without damaging the model's original foundational capabilities. By freezing the base and loading all new skills and personalized states into independent LoRA adapters, the system can continuously expand its capability boundaries while allowing each user's dedicated agent to iterate and evolve independently over the long term.
The Path to Commercial Success
This promising technological path is now a focus for many companies, but currently, very few have successfully established a complete commercial loop.
The first category of players includes Mind Lab and Thinking Machines Lab, which are also the only two teams globally possessing the infrastructure capable of simultaneously running reinforcement learning training for hundreds of trillion-parameter large models.
The second category includes earlier-stage companies like Fireworks AI and Together AI. Fireworks AI can host hundreds of LoRAs on a single base model and dynamically select adapters for each request, while Together AI has already set LoRA as the default fine-tuning method.
Although their technological maturity varies, these startups are already in the spotlight of capital and industry, with valuations soaring. This further proves that continuous learning has become an indispensable core underlying capability for the next generation of large models, and the LoRA architecture happens to provide an effective technical path for the engineering implementation of continuous learning.
However, the barrier to implementing this full technological chain is high. It tests a team's comprehensive hard power, from large model RL engineering and sparse MoE optimization to massive adapter scheduling, long-sequence training, and integrated Agent system building. Each link relies on the team's composite talent development and years of accumulated technical expertise.
Mind Lab is one of the few teams in the industry that has successfully run through this entire closed-loop system. In 2023, Mind Lab founder Andrew Chen co-authored a paper with Yao Shunyu, now Tencent's Chief AI Scientist, proposing FireAct to explore improving large model Agent capabilities through reinforcement learning.
Following the establishment of Mind Lab in October 2025, it successively released LongStraw, designed for ultra-long-context reinforcement learning; the Macaron-V1 model preview; Macaron-A2UI, a model trained based on MinT; the OpenClaw framework MindClaw for personalized long-term memory fine-tuning; and MinT itself.
The latest Macaron-V1 model is the culmination of all the team's prior academic research and engineering achievements. Beyond this, top-tier model capability is not just about paper benchmarks and technical demos. The investment Mind Lab has received from several leading tech giants and its rapidly rising ARR serve as more direct evidence of the commercial value of this technological path.
In June of this year, Mind Lab completed cumulative financing of nearly $50 million, with investors including Meituan, Yuanhe Puhua, Shokz, Variable Capital, and others. Regarding commercial data, Andrew shared at the WAIC that after commencing commercialization in July, the company achieved $10 million in ARR within two weeks. Its efficiency in technology commercialization is among the industry's top tier.
Mind Lab also possesses a unique advantage as a domestic Chinese team: the ability to fully leverage the numerous top-tier foundation models within China's open-source model ecosystem, developing from the shoulders of giants. Currently, it is not only collaborating with ByteDance and NVIDIA to build a LoRA training foundation but has also established partnerships with Shokz and other AI hardware companies.
Looking ahead, Mind Lab proposes a highly inclusive vision: it hopes that training a model will eventually become as simple as calling a model, where knowing how to use tokens means knowing how to train them. The industry has already begun forming a new industrial chain of foundation models + LoRA adapters + continuous learning. With its comprehensive technological achievements, Mind Lab has taken the lead in establishing a full-chain closed loop encompassing theoretical research, underlying system infrastructure, and commercial model delivery, broadening the path for the scaled implementation of the entire industrial sector.
Conclusion: A Commercial Milestone
The official release of Mind Lab's Macaron-V1 series marks a milestone in the transition of the PEFT scaling paradigm, centered on hybrid LoRA, from laboratory theory to large-scale commercial application. Unlike the traditional large model route of stacking parameters, this architecture of a "strong shared base + massive evolvable lightweight adapters" significantly reduces enterprise development, computing, and storage costs while relying on multi-module collaboration to achieve collective intelligence and long-term autonomous Agent capabilities.
The future of AI implementation may not rely on a single omnipotent large model, but on the coexistence of countless personalized agents, each attached to shared bases yet possessing independent memories and capabilities.
Comments