Your robot has a camera. You are storing what it sees as one image per frame, and that works until someone asks to watch it. Then you write a script that extracts ten thousand JPEGs from storage, sorts them by timestamp, and calls ffmpeg. You write that script again the next time anyone wants thirty seconds of footage.

Storing the encoded stream removes that step. ReductStore stores H.264 chunks as time-indexed records, and the ReductVideo extension returns an MP4 you can open in a player.

A rosbag recording is ultimately written to files, and a single file cannot grow forever. Something has to close it at some point, so rosbag2 makes you pick: ros2 bag record -a -b 100000 closes one every 100 kilobytes, or -d 9000 closes one every 9000 seconds. Either way, a recording that runs for hours comes out as a directory of many small files instead of one huge file that becomes awkward to work with.

Splitting a recording into five minute bags seems like a simple solution. The problem starts when you need to turn them back into one file.

A rosbag2 user reported that merging 300 GB of split recordings required roughly 600 GB of disk capacity. The original 300 GB stays on disk while another 300 GB is written out as the merged bag.

Splitting solves one problem. Merging creates another, and it is easy to underestimate.

We originally built ReductStore to store data on edge devices and read it back by time interval. But a device only has so much disk space. With cameras and sensors writing all the time, we used a FIFO quota to remove the oldest records and make room for new ones. That left us with another problem: how to get the data off the device before it was deleted.

At first, transferring data from an edge device to central storage was a manual job. We used the CLI client or scripts and copied data when somebody remembered to do it. It was always problematic because data collection never stopped. Sometimes we had only two or three days to copy it before it was gone, and a temporary network problem or a missed run could make that window disappear completely.

Automatic replication was the next logical step. The edge is producing a stream of new data, while the central store has more capacity and can have different criteria for what it keeps. We needed to forward new records and configure source-side filters for each destination. Keeping two identical replicas was not the goal.

We also had to keep in mind that the source is usually on the edge. Very often it has no public IP address, and its network connection can be unstable, slow, or absent for long periods. Replication therefore could not make local ingestion wait for the central store. The device still needed to accept data first and deliver it later when a connection became available.