Anthony Cavin

ROS2 is widely used for building robotic systems with sensors like cameras, LiDAR, and IMUs. While it's great for communication (e.g., publishing and subscribing to topics), it lacks a built-in solution for storing large amounts of unstructured data, such as images.

Bag files are commonly used to store data in ROS2, but they aren't a good fit for long-term storage or real-time streaming. They're mainly meant for recording and replaying mission data or episodes, not for managing large volumes of unstructured data.

Addressing this challenge, this blog post will guide you through setting up ROS2 with ReductStore a high-performance storage and streaming solution optimized for unstructured, time-series data.

We will focus specifically on image data, but if you are interested in a more general overview you can read How to Store and Manage Robotic Data which covers the challenges and strategies for storing and managing robotic data in general.

For the full code example, we will be using the reduct-ros-example repository, which provides a complete implementation of the concepts discussed in this article.

At a previous company, we used MQTT to send industrial data, such as vibration readings, images and log files. However, maintaining a history of this data proved challenging. Initially, we used a combination of a time-series database and an object store, but we struggled to ingest blob data quickly enough, and the system was difficult to maintain.

To help you avoid a similar experience, this article will recommend the most suitable database for your IoT or Industrial IoT (IIoT) project. We will look at different ways of storing data from IoT devices that communicate with each other via MQTT.

MQTT stands for Message Queuing Telemetry Transport and is a lightweight messaging protocol designed to be efficient, reliable, and scalable, making it ideal for collecting and transmitting data from sensors in real time.

Why is this important when choosing a database?

Well, MQTT is format-agnostic, but it works in a specific way. We should therefore be aware of its architecture, how it works, and its limitations to make the right choice. This is what this article is about, we will try to cut through the fog and explore some key factors to consider when selecting the right option.

Let's get started!

As data-driven applications evolve, the need for efficient storage solutions continues to grow. ReductStore and MinIO are two powerful solutions designed to handle massive amounts of unstructured data, but they serve different purposes.

While ReductStore is optimized for time-series object storage with a focus on unstructured data such as sensor logs, images, and machine-generated data for robotics and IIoT, MinIO is a high-performance object storage system built for scalable, cloud-native applications with a focus on S3 compatibility and enterprise-wide storage needs.

In this article, we'll explore the differences between ReductStore and MinIO, examine where each excels, and discuss how they can be used together to build a more comprehensive data storage solution.