Anthony Cavin

Your robot has a camera. You are storing what it sees as one image per frame, and that works until someone asks to watch it. Then you write a script that extracts ten thousand JPEGs from storage, sorts them by timestamp, and calls ffmpeg. You write that script again the next time anyone wants thirty seconds of footage.

Storing the encoded stream removes that step. ReductStore stores H.264 chunks as time-indexed records, and the ReductVideo extension returns an MP4 you can open in a player.

A rosbag recording is ultimately written to files, and a single file cannot grow forever. Something has to close it at some point, so rosbag2 makes you pick: ros2 bag record -a -b 100000 closes one every 100 kilobytes, or -d 9000 closes one every 9000 seconds. Either way, a recording that runs for hours comes out as a directory of many small files instead of one huge file that becomes awkward to work with.

Splitting a recording into five minute bags seems like a simple solution. The problem starts when you need to turn them back into one file.

A rosbag2 user reported that merging 300 GB of split recordings required roughly 600 GB of disk capacity. The original 300 GB stays on disk while another 300 GB is written out as the merged bag.

Splitting solves one problem. Merging creates another, and it is easy to underestimate.

TimescaleDB is an open-source time-series database optimized for fast ingest and complex queries. It is engineered up from PostgreSQL and offers the power, reliability, and ease-of-use of a relational database, combined with the scalability typically seen in NoSQL systems. It is particularly suited for storing and analyzing things that happen over time, such as metrics, events, and real-time analytics.

Since TimescaleDB is based on PostgreSQL, it supports blob data and can be used to store a history of unstructured data such as images, binary sensor data, or large text documents. In this article, we will use the database as a time-series blob storage and compare its performance with ReductStore, which is designed specifically for this use case.

TimescaleDB and ReductStore both have Python Client SDKs. We'll create simple Python functions to read and write data, then compare performance with different blob sizes. To repeat these benchmarks on your own machine, use this repository.