Gracija Nikolovska

Modern robots collect a lot of data from sensors, cameras, logs, and system outputs. Managing this data well is important for debugging, performance tracking, and training machine learning models.

Over the past few years, we've been building a storage system from scratch. As part of that work, we spoke with many robotics teams across different industries to understand their challenges with data management.

Here's what we heard often:

• Only a subset of what robots generate is actually useful
• Network connections are not always stable or fast
• On-device storage is limited (hard drive swaps is not practical)
• Teams rely on manual workflows with scripts and raw files
• It's hard to find and extract the right data later
• ROS bag files get large quickly and are difficult to manage

In this article, we compare four tools built to handle robotics data: ReductStore, Foxglove, Rerun, and Heex. We look at how they work, what they're good at, and which use cases they support.

If you're working with robots and need to organize, stream, or store data more effectively, this overview should help.

The Robot Operating System (ROS) is a powerful framework for developing and managing robotic systems. It simplifies integration, communication, and development through various tools and libraries. ROS is built around a communication system that uses a publish-subscribe model to connect components, where some, like sensors or cameras, act as publishers, and others, like motors or processors, are subscribers. The data shared between these components is organized into topics.

To make the most of this data, especially when it's needed later for analysis, debugging, or sharing, it's crucial to store it efficiently. In this article, we'll dive into three methods for storing ROS topics, comparing their benefits and limitations to help you choose the best one for your needs. In case you need to gain a broader understanding of how to handle robotics data effectively, make sure to check out our article on storing and managing robotics data first.

Robots generate massive amounts of data that must be managed effectively. Challenges like limited on-device storage, the need for real-time processing, and high cloud storage costs make it essential to find efficient solutions. Balancing edge and cloud storage while keeping data synchronized is a key part of effective management.

This article begins by outlining these challenges and offering practical strategies, such as using time-series databases and implementing retention policies. We will then introduce ReductStore, a specialized database designed to meet the unique needs of robotic systems. With features like real-time ingestion, efficient querying with batching, smart retention policies, and edge-to-cloud replication, ReductStore offers a cost-effective and high-performance solution for storing and managing robotic data.

We’ll also explore a hand-on example where we’ll show how you can set up ReductStore and use it for storing and managing data. Finally, we will compare ReductStore with MongoDB, explaining why ReductStore is the better choice for robotics. This comprehensive guide is designed to help engineers and developers overcome the challenges of robotic data management and optimize their systems.