Will my robot policy run on my hardware?

It looks like I'll be working with SO-ARM-101 for my graduation project this semester. I haven't really started setting it up yet, but I wanted to open a new series to record the practical things I run into along the way. Think of these as notes on the small questions that come up in a robotics project—the ones that don't feel so small once you're stuck on them.

Here's the first question I wanted to ask:

Will the model I train actually run inference on my hardware?

Before committing to a policy model, it's easy to spend ages comparing GPU specifications, only to hit an out-of-memory error when you finally try it. I've run into this too, so I wanted to share what I looked up before starting the project.

A training hardware guide is a starting point

One useful reference is LeRobot's official Compute HW Guide for LeRobot Training. It groups policies by their approximate training memory requirements.

The word training matters here. Those estimates don't directly answer whether a trained policy fits in memory or responds quickly enough during inference. So I went looking for inference information for each policy. The situations were quite different.

ACT: a promising starting point, with a backend caveat

ACT is one of the lighter options I looked at. It predicts an action chunk in a forward pass, rather than repeatedly denoising it.

There is a useful edge deployment example in Bimanual Manipulation Within an 8 GB Budget. On a Jetson Orin Nano Super, its ACT implementation took 114.02 ms per inference with FP32 PyTorch, compared with 17.93 ms with FP16 TensorRT. The robot's configured control loop ran at 10 Hz; the paper also explains how executing a chunk before predicting again changes the inference deadline.

That is encouraging, but it also changes how I'd phrase the conclusion: the execution backend and chunking setup matter. A fast optimized implementation is evidence that ACT can run on that edge device under those conditions, rather than a guarantee for every desktop, laptop, or Jetson setup.

Diffusion Policy: watch the denoising steps

Diffusion Policy has another kind of bottleneck. Its inference procedure repeatedly removes noise from an action sequence, so the number of denoising steps is a major part of the latency budget. GPU speed still matters, as do the model and observation setup.

The original Diffusion Policy paper, Section 3.3, reports about 0.1 seconds of inference latency on an RTX 3080, using DDIM with 10 inference iterations after training with 100 diffusion iterations. That is a result for the authors' configuration, not a timing that transfers directly to another setup. I covered the underlying formulation in my first Diffusion Policy reading note.

My own experience made the step count feel much more concrete. I trained Diffusion Policy with a Franka, and the architecture I propose in my paper is also based on Diffusion Policy. In my experiments, both needed at least 64 denoising steps at inference. As the number of steps increased, the delay became something I had to pay attention to.

For the unmodified Diffusion Policy, my measured setup was:

This is the measurement I reported in the Korean post. I haven't reproduced it for this English edition, and the original note does not specify the timing protocol or what the ± term represents. It is a useful data point from my setup, rather than a controlled comparison with the paper's RTX 3080 result.

Reducing the step count is worth investigating, but the result also has to remain a usable policy. In my case, I couldn't simply assume the paper's shorter sampling schedule would be enough.

SmolVLA: a smaller VLA

The name “vision-language-action model” can make a policy sound heavy before you've even tried it. SmolVLA is a reminder to check the actual model size: the SmolVLA paper describes its main model as 450 million parameters.

The team's official introduction also describes asynchronous inference, where the robot executes actions while the next chunk is being computed. That makes the deployment setup part of the story, alongside model size.

For me, this puts SmolVLA among the smaller VLA candidates worth investigating for a local project. But parameter count alone doesn't establish a fixed VRAM requirement, a particular GPU's latency, or the success rate of a deployment on an edge board. Those need measurements for the checkpoint, precision, camera inputs, and runtime I actually plan to use.

Larger VLAs: the weights still have to fit

With models such as π₀ and π₀.₅, I would pay closer attention to the inference memory budget from the beginning. The π₀ paper uses a 3-billion-parameter PaliGemma backbone, plus a 300-million-parameter action expert, for 3.3 billion parameters in total.

As a concrete implementation reference, the official openpi repository currently lists more than 8 GB of GPU memory for inference. That is the repository's estimate for its supported setup, rather than a universal minimum for every variant or runtime.

The same caution applies when comparing other large VLAs, such as OpenVLA and OpenVLA-OFT. A memory number is only useful if I know which model, precision, and implementation it describes.

Why doesn't “training is heavy, inference is light” settle the question?

Training uses memory for gradients, optimizer state, and the intermediate values needed for backpropagation. Inference can dispense with those training costs, but the model weights still have to be available, and the forward computation still needs working memory. Removing the optimizer doesn't turn a large model into a small one.

What I'll check before committing to a policy

My starting question is now a little more specific: what size of policy am I planning to use, and how will it run?

ACT, Diffusion Policy, and SmolVLA give me smaller models to investigate, but each has its own constraints. ACT's backend and action chunking matter; Diffusion Policy adds a sampling-step budget; SmolVLA still needs a concrete inference configuration. Larger VLAs make checking the memory budget especially important, even if training happens elsewhere.

For this project, I'd start by loading the intended checkpoint with representative observations on the hardware available to me, then measure memory use and time to produce an action chunk. That answers much more than a training VRAM table alone. I also need to compare that delay with how the robot executes and refreshes its actions: an inference call and the robot's control loop are not necessarily running at the same frequency.

Had I skipped this reading, I probably would have gone in thinking, “Surely inference will be easy once training is over.” My Franka timing was a good reminder to check.

I'll be back with more notes once I start setting up the SO-ARM-101 and run into the next round of practical problems.

Good luck with your projects too!

References

• LeRobot: Compute HW Guide for LeRobot Training. Training estimates; not an inference benchmark.
• Singh et al., Bimanual Manipulation Within an 8 GB Budget. ACT deployment settings and backend-specific inference measurements, especially Table IV.
• Chi et al., Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Section 3.3 reports the DDIM iteration count and RTX 3080 latency.
• SmolVLA paper and official introduction. Model size and the asynchronous inference approach.
• π₀ paper and openpi requirements. Architecture size and implementation-specific hardware estimates.