Apple’s new CEO has recently been a little dissatisfied with the progress of his own AI.
According to the latest news from Bloomberg, Apple CEO John Ternus has asked software director Federighi to speed up the pace of AI development and come up with bolder plans. He cannot just upgrade Siri with a few functions.
Compared with the speed of Apple's AI implementation, Apple's research progress actually exceeds many expectations, including embodied intelligence. Apple is also studying how to give AI a body that can act in the real world.
Recently, the Apple research team disclosed a study on a humanoid robot called HALO, which directly uses Yushu G1 as the body and a Mac mini M4 Pro strapped to the back, which still runs Alibaba’s Qianwen model.
This seemingly hybrid system can already allow robots to walk into strange homes autonomously, find chairs to sit on, get under tables, and push boxes away. The average success rate for 360 tests in 6 real-life environments that had never been seen and not scanned in advance was 91%.
Picture | Project address: https://apple-aiml-research.github.io/ml-halo/What’s even more interesting is that Apple didn’t even find someone to remotely control the robot to collect these training data.
The research team used hundreds of GPUs to batch generate robot motion and visual data in thousands of virtual homes, and then directly moved the learned skills into the real world.
This means that humanoid robots that originally required manual instruction by humans now have the opportunity to learn by relying on computing power.
[demo-1080.mp4]
To teach a robot to sit down, you first need to photograph what it sees.
When we hear "go sit on a chair," we usually look at where the chair is, then walk over, turn around, and then put the weight of our body on the seat.
For robots, this process requires a very specific training data. A common data sample must include not only the instructions it heard and the first-person view images, but also the records of how the joints of the whole body move when completing the task. Relying on human remote operation to collect, every time there is an additional sample, it has to be done again.
The data of this paired robot is called "Vision-Language-Action, VLA". Unlike training large language models that can use inexhaustible text on the Internet, there is no existing VLA corpus data in the real world.
Each training sample must be specially produced by someone: a remote operating system (such as ALOHA) or a wearable motion capture device (such as UMI). Producing one hour of data requires at least one hour of manpower. This is also a famous "data bottleneck" in the field of humanoid robots.
The solution given by Apple's paper HALO isNo more human data, all synthesized by simulation.
The research team used a motion diffusion model to generate full-body movements, and used a programmed generator (INFINIGEN) to create 1,000 virtual homes, with living rooms, offices, and bedrooms, with random furniture density. They then put the movements into the scene in Isaac Sim, rendered the first-person view of the robot's head camera, and finally handed it over to the automatic verifier to screen out bad samples that wear molds or slip.
For example, when sitting on a chair, the researchers first specified the stages of walking over, turning around, sitting down, and the final posture of sitting firmly.The action generation model gives multiple sets of trajectories based on this, the program checks whether the soles of the feet have slipped, whether the body has passed through the ground, and whether the joints have exceeded the range of motion, and then put the screened movements and chairs into the virtual room to avoid surrounding furniture.
Next, the simulator captured the scene from the position of the robot's head camera. The team worked on approximately 1,000 procedurally generated rooms. 163,000 items, nearly 300 hoursPaired samples, the data size is only limited by computing power.
The paper also mentioned that it took 250 H100 GPUs and 9 hours to generate 50,000 samples of the SIT (sit down) skill alone; the generation of all 1,178 rooms took about 300 GPUs and 26 hours.
Room variety itself also plays a role. The research team kept the amount of action data unchanged and only adjusted the number of training scenes. In 100 unseen simulated rooms, the success rate of sitting in a chair increased from 43% when only 1 room was used for training to 81% when 100 rooms were used. Adding 1,000 more scenes, the score rose to 84%.
Apple’s robot ecosystem runs on its own chips
HALO’s autonomy is accomplished through the collaboration of a three-tier structure, andAll run on the robot body, no internet connection, no remote control, no need for external motion capture.
- Orchestrator (approx. 0.5 Hz): A visual language model (the default is Qwen3.6-35B-A3B running locally) looks at the screen every two seconds and can break down free instructions such as "help me push the box to the table" into subtask sequences;
- Motion generator (3 Hz): The frozen Qwen3.5-2B VLM encoder plus a diffusion Transformer decoder trained from scratch generates a full-body motion plan for the next 5 seconds based on the current first-person view image and subtasks, that is, 29 joint angles on the robot body plus pelvic posture;
- Motion tracker (50 Hz): Combines depth vision and proprioception to implement movement plans into real joint commands. It has undergone three-stage training of "privileged expert training → multi-teacher distillation → multi-critic reinforcement learning fine-tuning" and is the lowest level for solving "physical problems".
In general, the upper two levels solve the "semantic" problem, that is, which chair the instruction refers to; the bottom layer solves the problem of how to make the butt land firmly on the chair surface without losing balance.
The interesting part of this arrangement is the mix of software and hardware. It uses the body of Yushu Unitree G1, with a Mac mini M4 Pro (64GB unified memory) on the back for thinking, and the original robot Jetson Orin NX is responsible for 50Hz real-time control. A 120° wide-angle camera is installed on the head, and the whole body is wireless, no cables, and no external motion capture.
The brain on the back of this Apple robot, the Mac mini M4 Pro, sports an open-source model from Alibaba. Of course, the orchestrator also reserves the option to call the cloud's cutting-edge large model, and completely relies on the local model when offline.
The Mac mini draws power directly from the robot's (Yushu G1) internal power bus, with no separate battery or power adapter.
When the real machine is running, the Mac mini on the back will determine which chair the person is talking about and what to do first. The action model takes over the current step and generates full-body movements for the next 5 seconds based on the camera footage, updating approximately 3 times per second.
The trouble is that while the robot is waiting for a new plan, it may have already put one foot up. If the new plan follows and the foot placement is changed, the movement will be sudden.
HALO uses asynchronous reasoning + training-time prefix constraints for the motion generator. When generating a new plan, it will retain the half-second actions that have been decided in the old plan and only recalculate the next 4.5 seconds. In this way, the robot can continue walking, and new plans can follow the actions it has already started.
Further down, the Jetson chip that comes with G1 combines the downward-looking depth camera and body status to update the position target 50 times per second for 29 joints. The sentence "sit down" in the upper-level plan has now turned into a series of actions in reality including how much to bend the knees and how to stabilize the torso. In the simulated chair test, if this layer of depth input is removed, the success rate will drop from 84% to 74%.
Most previous similar work was tested in rooms that had been scanned in advance, while HALO’s evaluation environment was 6 real homes that never appeared in the training and were never scanned or mapped。
In these environments, the robot completed six tasks: sitting down, walking towards a designated object, sideways through a half-open door, squatting/crawling under the table, and pushing boxes. The average success rate of 360 physical experiments was 91%, which is exactly the same as 91% in the simulation.
When the skills are strung together into combined tasks (such as "drill across the table → walk to the box → push the box"), the success rate drops to about 70%. The reason for the failure is very interesting: the robot was facing the wrong direction at the end of the previous action, the next target was not in the field of view at all, and the robot was "led astray" by other objects. But the solution is also easy. The paper mentioned that adding a simple recovery behavior of "turning in circles to find the target" can alleviate the problem.
Even more impressive is that in controlled experiments, when the number of training scenes increased from 1 to 100, the robot's success rate soared from 43% to 81%; and then increased to 1,000, reaching 84%.
This isn’t Apple’s first move
Although Apple’s AI has been delayed many times, looking through Apple’s public papers in the past two years, the layout of embodied intelligence has gradually become clearer:
- November 2024, ARMOR(Cooperating with CMU): Covering the arms of Fourier GR1 with 40 low-cost ToF radars like skin, solving the occlusion problem of humanoid robots that cannot see their own arms, and establishing a first-perspective airborne perception route;
- January 2025, ELEGNT: The Pixar desk lamp robot that refreshes the screen explores the motion design of "function + expression", allowing the robot to nod, approach, and use lights to guide our attention. This is also Apple's thinking on the form of consumer robots;
- May 2025, EgoDex and PH2D: Vision Pro was used to collect 829 hours of first-person dexterity operation data, and a human trainer wearing a headset directly trained the humanoid robot.
HALO looks like a brief end to this line. Perceptually, it inherits the first-person perspective philosophy. The scene is aimed at the family depicted in ELEGNT. In the most critical link of data, Apple gave up human collection this time and bet on simulation.
Of course, HALO still has a long way to go before it can truly enter thousands of households.
It can now sit on a chair, crawl under a table, and push boxes, but there are still a lot of skills it needs to learn before it can tidy up a messy bedroom or get a bottle of drink from the refrigerator. Even with each new skill added, researchers still need to manually design motion cues and keyframes.
However, HALO at least proves one thing:Humanoid robots learning how to interact with the real world could have an easier path to scale。
In the past, if you wanted a robot to accumulate 300 hours of experience, someone had to practice and record it with you. Today, researchers can use hundreds of GPUs to generate the same amount of training data in just over a day, and then hand that experience over to a robot that has never set foot in a real home.
What’s even more interesting is that Apple doesn’t even have its own robot body this time around. Yushu's G1, Alibaba's Qianwen, Nvidia's Jetson, and a Mac mini form a system that can perceive and act autonomously.
When robot training begins to rely more and more on computing power, the competition in the humanoid robot industry in the future may also extend from whose body is more flexible to who can make the robot accumulate experience faster and cheaper.
# Welcome to follow the official WeChat public account of aifaner: aifaner (WeChat ID: ifanr). More exciting content will be provided to you as soon as possible.
