RLDS Is Archived: What Format Should You Ship Robot Data In Now?

On November 29, 2025, Google archived the RLDS GitHub repository, so the library behind the RLDS format is now read-only. That matters because Open X-Embodiment, a shared collection of robot learning data, pooled data from 22 robots across 21 institutions and stores every episode in RLDS.

If you are about to buy, collect or deliver robot data, you need a format your training code can still read in two years. This guide explains what the archive changes, which formats teams use now, and how to pick one for your stack.

What happened to the RLDS format?

The RLDS format still exists, but its main library is no longer maintained. GitHub shows the repo as archived by its owner on November 29, 2025, which means no new commits, fixes or pull requests.

RLDS (Reinforcement Learning Datasets) was built to store episodic data for reinforcement learning, learning from demonstrations and imitation learning. Its design is simple:

• Episode: a dictionary holding metadata (such as episode_id or agent_id) and a dataset of steps.
• Step: one moment in time, with fields like observation, action, reward, discount, is_first, is_last and is_terminal.
• Loader: every dataset is read as a TensorFlow tf.data.Dataset, so RLDS assumes a TensorFlow and TFDS toolchain.

That last point is the real issue. If your training code doesn't run on TensorFlow, a read-only TensorFlow-based library is a risky base for new data.

Can you still use RLDS format datasets?

Yes. Archiving freezes the code; it does not delete data. Existing RLDS format datasets still load with the tools that read them today.

The Open X-Embodiment repository is not archived and still describes each episode using the RLDS episode format. So if your team already trains on Open X-Embodiment or other RLDS data, nothing breaks overnight.

The risk is slower. If a TensorFlow or TFDS update breaks something, there is no maintainer to fix the RLDS library. For data you collect from now on, choose a format with an active project behind it.

What format should you ship robot data in now?

For most new robot learning work, ship LeRobotDataset and keep raw sensor arrays in HDF5 or zarr alongside it. Pick by your training stack and the job the data has to do:

• Imitation learning on the LeRobot stack: LeRobotDataset, the format of Hugging Face's LeRobot library.
• Training beyond local disk: WebDataset, tar shards you can stream from object storage.
• Raw arrays such as depth or high-rate IMU: HDF5 on local disk, or zarr for chunked arrays in the cloud.
• Looking at episodes before you train: Rerun recordings, which put every stream on one timeline.
• An existing TensorFlow pipeline: keep the RLDS format, but plan an exit.

You can ship more than one. A practical pairing is LeRobotDataset for training plus HDF5 for streams, like depth, that the video layout leaves out.

What is LeRobotDataset, and why is it the default?

LeRobotDataset is Hugging Face's standard format for robot learning data. Version 3.0 splits each dataset into three parts:

• Tabular data: states, actions and timestamps in Apache Parquet.
• Video: camera frames encoded into MP4, sharded per camera.
• Metadata: JSON and Parquet files that describe the schema, frame rates, normalization stats and where each episode starts and ends.

The big change in v3.0 is file-based storage. Version 2 used one file per episode; v3.0 packs many episodes into each Parquet and MP4 file and finds episodes through metadata. That keeps file counts sane at millions of episodes, and datasets can be streamed from the Hugging Face Hub without downloading them first.

Version matters when you buy data. LeRobot's docs say v3.0 ships in lerobot 0.4.0 or later, and they provide a converter (convert_dataset_v21_to_v30) for older v2.1 datasets. Ask any vendor which version they export.

When do WebDataset, HDF5, zarr or Rerun make more sense?

Each one solves a narrower job than LeRobotDataset. Use them for that job, not as your only format.

• WebDataset: stores samples in numbered tar files, where files that belong to one sample share a base name. It reads with plain sequential I/O from local disk or cloud storage, which suits very large pretraining runs.
• HDF5: a self-describing file format (.h5 or .hdf5) with a BSD-style license. It holds n-dimensional arrays and their metadata in one file, which is ideal for depth maps, joint angles and calibration.
• zarr: an open specification for chunked, compressed n-dimensional arrays, with libraries in 10 languages. It works well when arrays live in object storage and are read in pieces.
• Rerun: an open-source SDK for logging and visualizing multi-rate, multimodal data. Use it to check that cameras, poses and actions line up before you spend GPU hours.

If your work involves hands, the annotation format matters as much as the container. Our guide to hand pose annotation formats compares MANO, 21-keypoint, UmeTrack and ARKit.

How do you move RLDS format data to LeRobotDataset?

Map the RLDS structure onto LeRobot's three parts, then validate with a real training run. The concepts line up closely:

1. Episodes: each RLDS episode becomes one LeRobot episode, with its boundaries recorded in metadata instead of separate files.
2. Steps to rows: each RLDS step becomes one row of tabular data, with state, action and timestamp columns.
3. Images to video: per-step camera images become one MP4 stream per camera.
4. Flags: is_first and is_last become episode start and end offsets; keep is_terminal if your policy uses it.
5. Extras: fields LeRobot's video layout does not carry, such as depth, go into a side file like HDF5.

Before you convert a full dataset, convert a few episodes, load them in your training code, and check that frame counts and timestamps match the source.

What should you ask a data vendor about formats?

A good vendor exports the format your stack already reads, from one consistent schema. Ask these before you sign:

• Which formats and versions? "LeRobot" is not enough; ask whether it is v2.1 or v3.0.
• Where does depth live? Video-based layouts often leave it out.
• How are timestamps aligned? Ask for the clock source and the units.
• Is there a sample? Load real episodes in your own pipeline before buying hours.
• Are fields identical across formats? Switching containers later should not change your data.

What Firsthand can and can't do

Firsthand delivers every episode from one canonical schema in five containers: RLDS, LeRobot, WebDataset, HDF5 and zarr, and Rerun .rrd. The fields, timestamps and units are the same in each, so you choose by training stack, not by what data exists. See our data format documentation and the RLDS vs LeRobot field-by-field comparison.

Here is what we can't do, stated plainly:

• LeRobot version: our LeRobot export uses the v2.1 layout today. If your stack expects v3.0, run LeRobot's v2.1-to-v3.0 converter on delivery.
• Depth in LeRobot: our LeRobot files do not embed depth. A policy that needs depth should use the HDF5 export alongside them.
• No format benchmark: we have not measured how training results differ between formats. The Open X-Embodiment and LeRobot figures in this article come from their own projects, not from Firsthand data.

You can point your loader at our sample pack of 40 real episodes, which ships in every format, and confirm the fit before you commit.

The bottom line

The RLDS format isn't dead, but its library is frozen. Keep reading the RLDS data you already have, and stop building new pipelines on it. For new robot data, LeRobotDataset v3.0 is the safest default, with HDF5 or zarr for raw arrays, WebDataset for streaming and Rerun for inspection. Before you buy, ask the vendor which formats and versions they export, then test a sample in your own loader.