Paper Figures 1 and 3 are reproduced from the cited paper for commentary. Copyright remains with the original authors or rights holders.
Today I am reading GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. I came to this after reading Diffusion Policy, so my first question was simple: what changes when an action-generation policy becomes part of a robot foundation model?
This is an English adaptation of my Korean reading notes. The scope is the N1 paper: its introduction, architecture, training data, evaluation, and limitations. The GR00T project continues beyond N1; I return briefly to the later versions at the end.
The Korean post also links an accompanying GR00T video.
1. The problem: data islands
A robot's body is part of its learning problem. Different robots have different joints, action dimensions, sensors, and camera placements. A demonstration collected on one platform cannot simply be treated as a demonstration on another.
The paper calls attention to these separated pools of experience. Robot data is already expensive; splitting it by embodiment makes each pool smaller still. I found this a useful starting point because it changes what we should look for in the model. The question is also how to make more experience usable together.
2. Two systems, one policy
GR00T N1 separates vision-language understanding, called System 2, from action generation, called System 1. System 2 processes the image and instruction. System 1 uses that representation to generate robot actions. The modules are coupled and trained together.
The names borrow Daniel Kahneman's distinction in Thinking, Fast and Slow: System 1 is fast, automatic, and intuitive; System 2 is slower and more deliberate. Understanding “this is a cup, and the instruction says to move it right” is the vision-language module's role. Producing the joint commands to grasp it is the action module's role. This is an architectural analogy, not a claim that the network reproduces human cognition.
System 2: a representation for control
The paper uses NVIDIA Eagle-2 as its vision-language backbone. Of GR00T-N1-2B's 2.2 billion parameters, 1.34 billion belong to the VLM. Images are encoded at resolution and reduced through pixel shuffle to 64 image tokens per frame.
One choice caught my attention: it uses the 12th language-model layer, rather than the final layer. The paper reports that intermediate features gave both faster inference and higher downstream policy success.
I still find that choice less intuitive than it first sounds. A feature useful for producing text need not be the feature that best supports a downstream motor policy. The practical question is the policy's behavior and computation time, not which layer feels most complete.
System 1: generating an action chunk
The action module uses a diffusion transformer, , with alternating cross-attention and self-attention blocks. The paper relates its structure to Flamingo and VIMA. It predicts an action chunk:
So the connection to Diffusion Policy is easy to see: the output is a short trajectory rather than a single isolated command. The training objective is flow matching. The paper writes Equation (1) as:
Here is the vision-language representation and is the robot's proprioceptive state. At , the interpolated action is pure noise; at , it is the clean action chunk. The network learns a vector field along the line joining noise and actions. There is a sign detail to watch: with this interpolation, the direction from noise to actions is , whereas the printed training target is . The sampling update must use a consistent sign convention; these two vectors point in opposite directions.
The paper reports that four inference steps worked well across embodiments, and sampling a 16-action chunk took 63.9 ms on an L40 GPU using bf16.
Why the two systems run at different rates
The paper describes System 2 at 10 Hz and motor action generation at 120 Hz. A robot does not necessarily need to reinterpret the full instruction every time it issues another motor command. The scene and instruction often change more slowly than the joint commands.
This reminded me of Diffusion Policy's observation and action-execution horizons: keep the current scene representation while executing a short sequence, then observe and replan. The rates refer to different parts of the pipeline. The 63.9 ms above measures generation of an entire chunk; it does not mean a fresh full chunk is inferred every second.

3. A pyramid of training data
This was the most interesting part of the paper for me. Its training material includes real robot demonstrations, human video, and synthetic trajectories. Each source contributes something different, and each comes with a different gap to bridge.

Human video and latent actions
Human video from datasets such as Ego4D and EPIC-KITCHENS does not contain the robot's joint commands. The paper trains a VQ-VAE whose encoder takes two frames, and , and compresses the change between them into a latent action . The decoder uses that latent and the first frame to reconstruct the later frame.
After training, the encoder acts as an inverse dynamics model: it infers what happened between the frames. The policy uses the continuous embedding before quantization as a latent-action label, with the same flow-matching objective but a separate embodiment identity. The representation provides supervision from visual change; it should not be confused with measured robot controls.
That distinction matters. Calling both things “actions” makes the pipeline sound simpler than it is. One is an observed command in a particular body; the other is a learned description of what happened between images.
Neural trajectories: from 88 hours to 827 hours
The paper fine-tunes video-generation models on 88 hours of real teleoperation data. Given an existing starting frame and a new language prompt, those models generate 827 hours of video, roughly ten times the original duration.
A multimodal LLM judges eight sampled frames from each video to check whether it follows the instruction. Filtering and re-captioning help manage quality. Generating this dataset required approximately 1.5 days on 3,600 L40 GPUs. That scale made me stop for a moment.
The generated videos still need action labels. Besides latent actions, the pipeline can use a separate inverse dynamics model trained on real robot data to predict robot-action labels. A plausible-looking video is not, by itself, a physically valid control trajectory.
Simulation trajectories: nine months in eleven hours
The other synthetic source is DexMimicGen. It splits a small set of human demonstrations into object-centric subtasks, transforms each segment to a new object arrangement while preserving the relative end-effector pose, and replays it in simulation. Only successful demonstrations are retained.
Across pre-training and post-training data, the paper reports 780,000 simulated trajectories, equivalent to about nine continuous months of human demonstration, generated in 11 hours. The pre-training subset contains 540,000 demonstrations: 10,000 for each of 54 source/target receptacle combinations. These are related totals with different scopes, not two counts of the same dataset.
Sharing a model across bodies
Small, embodiment-specific MLPs convert states and actions of different dimensions into a shared embedding size; an embodiment-specific decoder maps the output back to the appropriate action space. This lets the large model share learned structure while smaller components account for differences between robot bodies.
Training in stages
The paper distinguishes three settings:
- Pre-training: mix real robot data, human video, and synthetic data across embodiments using the flow-matching objective.
- Post-training: fine-tune for one target robot. The language component of the vision-language backbone stays frozen; the rest of the model can adapt.
- Post-training with neural trajectories: mix real and generated trajectories at a 1 sampling ratio to supplement scarce target-task data.
In my notes, I think of this as learning broadly, adapting to a body, and then testing how much generated experience can help. Pre-training used up to 1,024 H100 GPUs and approximately 50,000 H100 GPU-hours.
4. What the evaluation tells me
The simulation benchmarks are RoboCasa Kitchen (24 tasks), the DexMimicGen Cross-Embodiment Suite (9 tasks), and GR-1 Tabletop (24 tasks). On the real Fourier GR-1 humanoid, the paper evaluates 13 tasks covering pick-and-place, articulated objects, industrial objects, and bimanual coordination. The baselines are BC-Transformer and the Diffusion Policy we read across the previous two posts.
For the simulation comparison with 100 demonstrations per task, Table 2 reports:
| Benchmark | GR00T-N1-2B | Diffusion Policy |
|---|---|---|
| RoboCasa | 32.1% | 25.6% |
| DexMimicGen | 66.5% | 56.1% |
| GR-1 Tabletop | 50.0% | 32.7% |
For the real robot, Table 3 reports:
| Target-task demonstration data | GR00T-N1-2B | Diffusion Policy |
|---|---|---|
| Full data | 76.8% | 46.4% |
| 10% of data | 42.6% | 10.2% |
The last two rows impressed me most. With only 10% of the target-task data, GR00T-N1-2B is just 3.8 percentage points below Diffusion Policy trained on all of it. This is a comparison of target-task data efficiency: GR00T also benefits from broad pre-training, so the models have not seen equal total training data.
The neural-trajectory ablation also reports gains. On RoboCasa, adding generated trajectories improves average success by 4.2, 8.8, and 6.8 percentage points with 30, 100, and 300 demonstrations per task, respectively. Across eight real-robot tasks, the average gain is 5.8 percentage points. Generated data helps in these settings, although these numbers do not show a monotonic “less data means a larger gain” relationship.
I read each result together with its task set, training data, and evaluation conditions. A benchmark result is not a guarantee on an arbitrary robot. The numerical tables and settings are in Section 4 of the paper.
5. Limitations
N1 focuses on relatively short tabletop manipulation. Walking while manipulating objects, extended tasks, and broader whole-body behavior require more work. Synthetic experience also inherits limitations from its generator or simulator.
I appreciated that the paper leaves these boundaries visible. “Generalist” is a direction for the model, not evidence that every kind of robot behavior has already been solved.
My takeaway
Reading this after Diffusion Policy helped me see the continuity. Generating a sequence of actions remains central. What expands is the conditioning information, the range of bodies, and the effort spent bringing different kinds of experience into the training process.
That is the question I want to carry into the next reading: when a robot foundation model improves, how much comes from the policy architecture, and how much comes from the experience it can now learn from?
After N1
The Korean notes also point ahead to GR00T N1.5, its Eagle-2.5 backbone, and the GR00T-Dreams data-generation pipeline. The N1.5 project page is the follow-up reference. Later versions, including the N1.6 update mentioned in the Korean post and the N2/DreamZero direction, deserve their own reading rather than being folded into N1's experiments.