47 Open Egocentric and Robotics Datasets (With Licenses)

Finding robotics or first-person video data is easy. Finding a dataset whose license actually matches what you want to do with it is much harder.

Some datasets are openly downloadable but restricted to non-commercial research. Others require users to sign a separate data agreement. Some repositories use permissive software licenses even though the videos, annotations, 3D assets, or demonstrations distributed alongside the code have different terms.

This egocentric dataset list brings together 47 useful egocentric, human-object interaction, robot manipulation, embodied AI, and multimodal datasets while putting licensing at the center of the comparison.

Important: Licenses and dataset terms can change. Always check the current official license and data-use agreement before training, redistributing, or commercializing a model. This article is an informational research guide, not legal advice.

Why Dataset Licensing Matters for Robotics and Embodied AI

A dataset can be technically perfect and legally unusable for your project.

This is particularly important for embodied AI because modern training pipelines increasingly combine multiple data sources: human first-person video, robot trajectories, simulation, language instructions, depth, pose, gaze, and object annotations.

Suppose a company combines five datasets into one pretraining corpus. Four permit commercial use, but the fifth is licensed only for non-commercial research. That single dataset can create a licensing question for the entire training pipeline.

The safest approach is therefore to treat licensing as part of dataset selection—not as paperwork to review after training.

47 Egocentric and Robotics Datasets to Know

The following egocentric dataset list is organized by dataset type. “License/access” is a practical summary, not a substitute for reviewing the complete current terms.

Egocentric and Human-Activity Video Datasets

Ego4D is one of the largest foundational resources in this category, with thousands of hours of first-person video. Its related Ego-Exo4D project adds synchronized egocentric and exocentric perspectives, along with modalities such as audio, IMU, gaze and 3D information. Access requires following the projects' licensing procedures rather than simply treating the datasets as unrestricted public video.

EPIC-KITCHENS provides a particularly clear example of why “public” and “commercial” must be separated. EPIC-KITCHENS-100 contains approximately 100 hours of unscripted kitchen recordings, but the public dataset is distributed under CC BY-NC 4.0. The project explicitly directs commercial users to request a separate commercial license.

Assembly101 follows a similar non-commercial model: its official annotation repository identifies the dataset as CC BY-NC 4.0.

HoloAssist takes a different approach. Its project page states that the dataset is released under CDLA v2 and describes that release as permissive. The dataset contains 169 hours from 350 instructor-performer pairs and includes synchronized signals useful for interactive AI assistance.

Robot Manipulation and Embodied AI Datasets

What Makes Open X-Embodiment Different?

Open X-Embodiment is especially important because it represents a move away from training robot policies on one robot in one laboratory.

The project standardizes many robot datasets using a common RLDS-style representation, making cross-embodiment training substantially easier. The official repository describes the project as an effort to provide open-sourced robotic data in a unified format.

However, aggregation creates an important licensing lesson.

Do not assume that one umbrella project automatically replaces the terms of every contributing dataset.

Commercial teams should trace each dataset in a training mixture back to its original source and record its applicable license, because individual components, assets or original releases can have different restrictions.

Which Robotics Datasets Have Especially Clear Commercial-Use Signals?

RH20T provides an unusually useful example because its maintainers explicitly divide the dataset according to licensing.

The RH20T-C portion covering specified scenes is licensed under CC BY-SA 4.0, while RH20T-NC is distributed under non-commercial terms. The maintainers specifically state that commercial use of the non-commercial subset—or models trained on that subset—is not allowed under those terms.

That distinction is valuable for AI developers because it addresses a question that many academic datasets leave unclear: whether restrictions apply only to redistributing the raw dataset or extend into model development.

ManiSkill demonstrates another complication. Its rigid-body environments may use permissive licensing, while assets are identified as CC BY-NC 4.0 and other components can follow different licenses.

The lesson is simple: a dataset does not always have one license.

How Should You Evaluate an Egocentric Dataset License?

Before downloading anything from this egocentric dataset list, create a licensing record for each dataset.

1. Identify the actual data license

Start with the dataset website, license agreement, dataset card or official repository.

Do not stop when you find a LICENSE file on GitHub. That file may apply only to downloader scripts or evaluation code.

2. Check commercial training rights

Look specifically for language covering commercial use, machine-learning training, model development, internal R&D and production systems.

“Available for research” should not be interpreted as permission for commercial model training.

3. Check downstream model rights

Training permission and deployment permission are different questions.

Your review should determine whether you may train the model, own or commercialize model weights, provide API access, distribute derivatives, and deploy the model in a paid product.

4. Review redistribution restrictions

Many licenses permit researchers to work with a dataset but prohibit redistributing the original videos or images.

This matters when using external annotation companies, cloud providers, contractors or other third parties.

5. Examine human-subject and privacy conditions

Egocentric footage can contain faces, voices, homes, screens, bystanders, personal belongings and other sensitive information.

Licensing should therefore be evaluated alongside consent, privacy and provenance—not separately from them.

Open Access vs. Open Source vs. Commercially Usable

These terms should never be treated as synonyms.

Open access may simply mean researchers can obtain the dataset.

Open source generally describes software whose source code is distributed under an open-source license.

Commercially usable means the applicable rights actually permit your intended commercial activity.

A research group can therefore publish its downloader under MIT while distributing the associated video dataset under completely different conditions.

EPIC-KITCHENS illustrates this clearly: the dataset is publicly available for research, yet its public materials are explicitly non-commercial under CC BY-NC 4.0.

Which Dataset Should You Choose?

There is no universal “best” dataset.

For broad first-person perception, Ego4D is a natural starting point.

For synchronized first- and third-person demonstrations, Ego-Exo4D is more specialized.

For kitchen actions, object interactions and action anticipation, EPIC-KITCHENS remains highly useful.

For detailed 4D hand-object interaction, HOI4D deserves consideration.

For assisted procedural tasks with multiple sensor streams, HoloAssist offers a different supervision structure.

For robot-policy pretraining across many embodiments, Open X-Embodiment is particularly relevant.

And for real robot trajectories, datasets such as DROID, BridgeData V2 and RH20T move closer to the action-conditioned data required for robot learning.

The correct choice depends on the target environment, task, embodiment, sensor configuration, annotation requirements—and license.

Conclusion

The growing number of public egocentric and robotics datasets is accelerating research in embodied AI, multimodal models, manipulation, activity recognition and humanoid robotics.

But dataset size should not be the only deciding factor.

Licensing, provenance and downstream rights matter just as much as hours of video or number of robot trajectories.

Use this egocentric dataset list as a starting point for discovering relevant data, then verify the official terms before incorporating any dataset into a production training pipeline. For commercial AI teams in particular, the safest workflow is to document data access → training rights → model rights → deployment rights → redistribution rights before training begins.