On November 20, Shanghai Mifeng Embodied Intelligence Technology Co., Ltd. (hereinafter "Mifeng Technology") officially released a real-machine data quality study. The team said the study is based on 207,000 hours of real-machine collection records and more than 20,000 task records, using a fine-tuned π0.5 model for real-machine validation.
π0.5 is a general-purpose robot foundation model developed by Physical Intelligence. The Mifeng team fine-tuned it with self-collected demonstration data and evaluated it on real robots.
The team placed three existing research directions — robot body adaptation, teleoperation systems, and imitation learning data quality — into the same real-machine data production pipeline for discussion, and used specific experiments to quantify some of the effects.
The related paper was submitted to arXiv on November 17. The study's conclusion is that a robot successfully completing a task does not necessarily mean good training data was obtained, and successful demonstrations can also teach the model bad habits.
The paper is authored by the Mifeng real-machine operations team and is a technical report that has not yet undergone peer review. In the paper's contribution statement, the research team divided participants into categories including core contributors, team leaders, contributors, and Sponsors (project support).
The project support list includes Mifeng Technology Chairman and CEO, Zhiyuan co-founder, co-president, and Embodied Business System President Yao Maoqing, as well as Zhiyuan Embodied Research Center Vice President Ren Guanghui and others, without specifying the form of support. Mifeng Technology is a data services company incubated by Zhiyuan Robotics, and the paper's first page also lists the Zhiyuan Robotics teleoperation website domain.
The paper describes three groups of controlled experiments.
The first group is the observation gap experiment. The team placed blocks outside the robot's field of view and compared three collection protocols.
From the experimental results, when the operator directly sees the target but the robot cannot, the model's arm-selection accuracy was 50%, close to random guessing; when the operator and robot share the same viewpoint and select the arm directly based on priors, it was 53%; when the robot is first allowed to actively search and confirm the target before selecting the arm, it reached 100%.
Each of the three groups conducted 30 real-machine evaluations, with complete pick-and-place success rates of 33%, 40%, and 60% respectively, and the 95% confidence interval for the difference was 27 to 64 percentage points.
The experiment shows that it is not enough for the operator and robot to see the same thing; the demonstration must record the basis for decisions into the visual frame.
The second group is the calibration experiment. The team artificially added zero-position offsets by batch to the state data of 14 joints, simulating calibration differences across different collection batches, with 100 demonstrations per group.
The paper states that when all experiments used clean data consistent with the deployment zero position, the success rate was 63.3%; when 75% offset data was mixed in but 25% aligned anchors were retained, it was 66.7%; without clean anchors, it dropped to only 20% to 26.7%. When the inference-side robot itself had a zero-position deviation, all 30 tests failed.
The team stated that for post-training, calibration consistency is a prerequisite for reliable execution of the model on the deployment body.
The third group is the redundancy experiment. The team mixed 0% to 80% of demonstrations containing pauses and detours into the training data. The model retained about 90% of the added path length, and execution time increased from 25.3 seconds to 32.6 seconds.
This group of experiments corresponds to real scenarios at collection sites. The experimental results show that the danger of post-training data lies in possibly confusing task success rate with action quality. Detours and pauses that operators make to compensate for equipment limitations may enter the training set along with successful demonstrations.
The three types of problems affect model learning in different ways. Observation gaps let the model see only the operation result while lacking the information needed for decision-making; calibration inconsistency makes the same joint values correspond to different actual postures; redundant demonstrations hand the operator's remedial actions to the model as if they were the correct path.
To explain the principle behind "invisible bad data," the team analyzed two levels: body design and teleoperation control. The paper's authors stated that end-to-end causal verification remains future work.
At the body level, the team analyzed 25 mainstream dual-arm robots, including Zhiyuan G1/G2, Unitree G1-D, Galaxea, AgileX, ARX, and Franka. The results show that after lengthening the gripper, the robot can reach farther positions, but this may come with rising joint-limit costs.
The paper proposes that robot selection should not only compare arm length, but should also comprehensively evaluate the position, gripper orientation, and joint margin required by the task; when workspace is insufficient, waist movement should be prioritized.
At the control level, experiments found that extra motions introduced by the control system will ultimately be solidified into the model's action policy by post-training.
The team mentioned that a filtering algorithm may need less than 1 millisecond to complete computation, yet bring more than 200 milliseconds of phase lag to actual motion. After the image seen by the operator lags behind the robot's real motion, it is easy to overshoot the target, wait for feedback, and then correct in reverse.
Therefore, when evaluating the quality of a collection system, it is not enough to consider only network latency, algorithm runtime, and task success rate; it is also necessary to measure the entire process from operation command, robot response, to visual feedback return.
Regarding how the quality of human demonstration data affects robot imitation learning, there has already been related academic research, mainly involving the model's recognition of decision bases, the design of teleoperation collection systems, and the impact of different demonstration methods on downstream policies.
The significance of Mifeng's study lies in placing the three types of variables — body, control, and data — into the same data production framework and providing quantified real-machine results. The paper points out that the experiments used only a single policy model and a single action representation, some offsets were manually injected, and the universality of the effects needs validation across more tasks and architectures.
In September this year, Mifeng released the crowdsourcing platform Mifeng Pai. Mifeng stated that the real judge of data quality is always whether downstream models are willing to repurchase and how much value is generated after use, and platform settlement is executed according to the duration of valid data after review.
Mifeng previously promoted million-hour-level body-free data capacity and a goal of ten-million-hour level within the year, while the 207,000 hours disclosed in the paper belongs to the real-machine teleoperation category. The two are not the same statistical basis, and it was not stated whether the statistical scopes overlap.
The paper's acceptance checklist proposes checking completion time, trajectory length, motion continuity, and decision information completeness. How this connects with settlement standards remains to be confirmed.
For the data services industry, the implication of this study is that acceptance standards may expand from whether the task succeeds to a multidimensional standard of observation completeness, calibration consistency, and path efficiency. Whether it can enter procurement and pricing processes remains to be verified by Mifeng's clients.
The observation points are whether peer data service providers follow up with similar quality metrics, whether model companies raise corresponding requirements in procurement acceptance, and whether third parties reproduce these effects on architectures beyond π0.5.
Comments