What Hardware Sync Buys You That a Phone App Can't

In August 2026, Figure announced that its Index app had crossed 16 million uploaded videos from 264,000 downloads across 108 countries, processing 30 minutes of footage every second. No instrumented rig fleet on earth comes close to that throughput. If you are deciding where your data budget goes, that number deserves a straight answer rather than a dismissal.

The honest version is that the two methods buy different things, and the difference is not image quality. A phone app buys coverage: environments, objects and habits you could never specify in advance, which is exactly what real-world video data is for. Multimodal sensor sync AI data buys correspondence — the guarantee that the pixel, the depth value, the wrist view and the IMU sample you are about to fuse describe the same instant and the same geometry. This article is about what that guarantee costs to produce, and when you actually need it.

What does hardware sync actually mean?

It means one clock and one trigger line. A single timing source tells every sensor when to capture, and every sample lands in the same time domain, so no software has to guess afterward which frame belongs with which depth map.

That is the whole idea, and it is narrower than the word "synced" usually implies in a vendor deck. Ten sensors recording at once are ten separate recordings until something forces them onto a shared timebase. Two things do that job:

• A hardware trigger: a physical signal that fires the sensors together, rather than each sensor free-running and being matched up later.
• A shared clock domain: usually PTP (IEEE 1588), with a grandmaster clock that every device disciplines itself to, so the timestamps written into the file mean the same thing across streams.

Purpose-built research hardware is built exactly this way. Meta's Project Aria glasses timestamp sensor data on a dedicated microcontroller rather than in the Android OS, and Meta documents the resulting error: for the global-shutter SLAM and eye-tracking cameras, the timestamping error is bounded at 19 microseconds. Meta also states that Aria's sensors are temporally aligned to a shared device-time domain, and that multiple devices can be aligned to each other the same way.

The word to watch for is post-hoc. If streams are aligned after capture — by matching audio claps, by cross-correlating motion, by trusting file creation times — you do not have hardware sync. You have an estimate, and nobody has told you its error bars.

What is a phone app genuinely good at?

Diversity and throughput, by a margin no rig programme can close. A phone is already in the pocket of everyone who knows how to do the task you want recorded, which is why app-based collection reaches environments a rig fleet never books.

Figure's published Index figures are the clearest public evidence of this, and they are worth reading carefully rather than waving away:

• 16 million videos uploaded by Creators, with the pipeline processing 30 minutes of video every second.
• 264,000 downloads across 108 countries, with over 44,000 weekly active users contributing.
• Per 1,000 hours collected: 373 unique tasks, 1,146 unique manipulated objects, 116 unique environments. That density of long-tail variation is the hard part to buy, not the easy part.
• $15M paid to Creators to date, with a stated commitment to spend over $1B on data and compute in the following 12 months.

Figure is also explicit that it tried the alternative first and that vendors could not hit its throughput, diversity or quality bar, so it built the app pipeline itself. Anyone selling instrumented capture who pretends this is not a real result is not arguing in good faith.

What the app approach is buying is prior over the world: what a kitchen looks like in 108 countries, how many ways a person opens a carton, which objects actually sit on real counters. For pretraining a visual policy on human behaviour, that distribution matters more than millimetres.

What can a phone app not deliver, however good the software is?

Four things, and none of them are fixable in an app update: cross-subsystem timing you can trust, reliable depth, a second viewpoint on the hands, and solved extrinsics between the two.

Timestamps you can compare across subsystems. This is the one people assume software solves. Android's own camera API says otherwise. A camera device reports its timestamp source, and where that source is SENSOR_INFO_TIMESTAMP_SOURCE_UNKNOWN, Google's documentation states the timestamps are monotonic but "can not be compared to timestamps from other subsystems" — accelerometer and gyro are the examples given. On such a handset, fusing the video with the phone's own IMU means fusing two clocks whose relationship the platform declines to guarantee. You can estimate the offset. You cannot look it up.

Depth, except conditionally. Apple's ARKit provides scene depth from the LiDAR scanner, and Apple's documentation is explicit that ARKit supports scene depth only on LiDAR-capable devices. A collection programme running on whatever phone a contributor already owns gets depth on some submissions and not others, at varying quality, which is a sampling problem rather than a format problem.

A wrist view. At the moment of contact, the hand and the object are frequently occluded from a head-mounted or hand-held viewpoint. A wrist camera sees the grasp the head camera cannot. One phone is one viewpoint, and asking a contributor to hold a second phone defeats the point.

Extrinsics. Even with two devices, you need the rigid transform between them, solved against a calibration target, or the two views cannot be related geometrically. A pair of unregistered video streams is not a multi-view dataset.

None of this makes app footage useless. It makes it single-stream RGB, and it should be priced and labelled as single-stream RGB.

Why does global shutter vs. rolling shutter matter for manipulation data?

Because a rolling shutter does not capture one instant. It reads the sensor row by row, so a frame taken during fast hand or head motion is a composite of slightly different moments, and the geometry in it is skewed.

A global shutter exposes every pixel at the same time, which is why research hardware uses it for the streams that feed pose and tracking. The split shows up inside Aria's own design: Meta's documentation notes that the SLAM and eye-tracking cameras use electronic global shutter sensors whose timestamps mark the centre of the exposure window, while the RGB camera uses an electronic rolling shutter whose timestamp marks the centre of the exposure of the middle row. That phrasing is the admission: with a rolling shutter, the top and bottom of the frame were not taken at the timestamp written in the file.

Phone cameras are overwhelmingly rolling shutter. For a slow, deliberate demonstration that costs you little. For the fast reach-and-grasp segments that manipulation policies actually need, it means the hand position implied by the pixels and the hand position implied by the IMU disagree, and the disagreement varies with where in the frame the hand happens to be.

Global shutter is not free — the sensors are larger, more expensive and generally noisier in low light at equal resolution, which is precisely why phones do not use them. That trade is a reasonable one for a consumer camera and a bad one for training data on fast motion.

How do you verify a sync claim instead of trusting it?

Ask for per-stream nanosecond timestamps in the delivered files and recompute the error distribution yourself. Any vendor claiming hardware sync who cannot hand you the timestamps is asking you to take timing on faith.

The check is short. For each frame of the reference camera, find the nearest sample in every other stream, take the absolute offset, and look at the median and the p99 rather than the best case. What you want back from a vendor, before you sign anything:

• A stated ceiling, not an adjective. "Tightly synchronised" is not a number. Firsthand publishes a 2.0 ms hardware-triggered ceiling with a measured median of 1.12 ms and p99 of 1.94 ms across 12,480 frames sampled from shipped episodes — the method and raw CSV are on the capture methodology page.
• The sampling basis. Frames drawn at random from delivered episodes, not from a demo reel.
• What happens to breaches. A ceiling with no rejection rule attached is decoration. Ask whether out-of-spec segments are recaptured and whether the reject log ships with the batch.
• Calibration at both ends of the session. An opening solve lets everyone assume nothing shifted. A closing solve turns that assumption into a drift number you can filter on.
• Timestamps in the delivery format. timeStampNs as int64 per stream, in whatever container you take — RLDS, LeRobotDataset, WebDataset, HDF5 — so the check is reproducible on your side.

If your own recomputation disagrees with the vendor's figure on a delivered batch, that is a defect, and it should be treated as one contractually.

Which one should you actually buy?

Match the tier to what the model has to learn. If the training signal is "what do humans do and where", buy breadth. If the training signal is "where exactly was the hand relative to the object at this millisecond", buy sync.

• Pretraining a visual or semantic policy on human behaviour: app-collected single-stream RGB, at volume. The distribution is the product, and Figure's Index numbers show what that scale looks like.
• Manipulation, grasping, contact-rich tasks: instrumented rigs. You need depth, a wrist view, hand pose and a shared clock, because the model is learning geometry and timing rather than context.
• Evaluation and held-out sets: instrumented, and held out by site rather than by clip, so you are not training and evaluating in the same kitchen.
• Most real programmes: both, bought deliberately and labelled by tier, so you can filter on capture method instead of inferring it from the footage.

What hardware sync cannot do for you. It does not scale like an app, and anyone implying otherwise is selling. A rig programme is gated on recruiting, training and retaining operators — Firsthand's own published funnel shows 4,180 applicants producing 287 currently active collectors, roughly 7 percent. It costs more per hour, because operators are paid for setup and calibration time, not per clip. It reaches fewer countries than a phone app does. And Firsthand's second, contributor-worn 360 tier explicitly gives up depth, wrist views and sub-millisecond sync in exchange for that reach, which is why every multi-stream claim on our capture rig specifications is scoped to the instrumented tier only.

One more honest note: the third-party results cited here — Figure's Index figures, Meta's Aria timestamping bounds — were measured on those organisations' own systems and datasets, not on Firsthand data. They establish how the technology behaves. They are not benchmarks of what Firsthand delivers, and nobody should read them that way.

The bottom line

A phone app and an instrumented rig are not competing on quality. They are competing on what they guarantee. The app guarantees reach — 16 million videos across 108 countries is a real achievement and a real asset. Hardware sync guarantees correspondence: one trigger, one clock, published error bars, and a calibration solve at both ends of every session.

Buy the app tier when the model needs to see the world. Buy the synced tier when the model needs to know exactly where and exactly when. And in either case, ask for the timestamps and check the number yourself.