ReductStore is designed to efficiently handle time series unstructured data, making it an excellent choice for storing high frequency vibration sensor measurements. This article is the second part of How to Store Vibration Sensor Data | Part 1, where we discussed the benefits of storing both raw measures and pre-processed metrics, the advantages of time series databases, and efficient storage and replication strategies.

In this post, we'll dive into a practical example of storing and querying vibration sensor readings using ReductStore and Python. To follow along, you can find the full source code for this example at GitHub's reduct-vibration-example repository.

Our example will show you how to:

1. Store simulated sensor values in 1-second chunks
2. Compute and store associated labels for each chunk
3. Query and retrieve stored measurements within a specified time range
4. Set up replication using the ReductStore web console

Efficient and effective storage of vibration data is important to a wide range of industries, particularly where accurate and complex predictive maintenance or optimization is required.

This blog post looks at best practices for managing vibration data, starting with storing both raw and pre-processed metrics to take advantage of their unique benefits. We'll explore the differences between time series object stores and traditional time series databases, and highlight optimal data flow processes.

We'll also cover strategies for eliminating data loss through volume-based retention policies, guide you through setting up an effective data retention frameworks.

We are pleased to announce the release of the latest minor version of ReductStore, 1.10.0. ReductStore is a time series database designed for storing and managing large amounts of blob data.

To download the latest released version, please visit our Download Page.

What's new in 1.10.0?​

ReductStore v1.10.0 introduces new query and replication parameters that can downsample data when querying or replicating to another database. In addition, we have optimized the operation of the storage and replication engines, which should improve the overall performance of the database.

Photo by Jan Antonin Kolar on Unsplash

Since joining ReductStore's project, I've been exploring alternative solutions to get a better understanding about how the project fits into current echosystem.

To give you some context, we will look at solutions to store data from IoT devices (e.g. sensors, cameras, etc.) that commonly use MQTT to communicate with each other.

MQTT stands for Message Queuing Telemetry Transport and is a lightweight messaging protocol designed to be efficient, reliable, and scalable, making it ideal for collecting and transmitting data from sensors in real time.

Why is this important when choosing a database?

Well, MQTT is format-agnostic, but it works in a specific way. We should therefore be aware of its architecture, how it works, and its limitations to make the right choice. This is what this article is about, we will try to cut through the fog and explore some key factors to consider when selecting the right option.

Let's get started!

A year ago, we migrated ReductStore, time series database for unstructured data, from C++ to Rust. Now, reflecting on that decision after a year, it's clear that it was the right move, yielding the following benefits:

• Fewer bugs related to memory management, move semantics, and threading
• Easier codebase porting to MacOS and Windows
• Better dependency management with Cargo compared to what we had with C++

Today, I'm pleased to announce that we've rewritten the CLI client from Python to Rust. Our primary motivation was distribution. Although Python is one of the most widely used programming languages, we don't want to require users to install an interpreter to use our tools, and standalone installers are too bulky for us. With Rust, we can build compact and blazingly fast executable binaries for most popular platforms.

wget https://github.com/reductstore/reduct-cli/releases/latest/download/reduct-cli.linux-amd64.tar.gz
tar -xvf reduct-cli.linux-amd64.tar.gz
chmod +x reduct-cli
sudo mv reduct-cli /usr/local/bin

In edge computing, managing time series blob data efficiently is critical for performance-sensitive applications. This blog post will compare ReductStore, a specialized time series database for unstructured data, and MongoDB, a widely-used NoSQL database.

Using Docker containers for straightforward setup, we'll examine the speed of each system. We'll go through setting up ReductStore buckets and preparing MongoDB collections, focusing on how to effectively store and access blob data for time series scenarios.

By conducting performance tests on binary data insertion and retrieval, we aim to provide insights into which system might best serve your application's needs.

For those interested in replicating our benchmarks or conducting their own evaluations, we've made our methods easily accessible through this repository.

TimescaleDB is an open-source time-series database optimized for fast ingest and complex queries. It is engineered up from PostgreSQL and offers the power, reliability, and ease-of-use of a relational database, combined with the scalability typically seen in NoSQL systems. It is particularly suited for storing and analyzing things that happen over time, such as metrics, events, and real-time analytics.

Since TimescaleDB is based on PostgreSQL, it supports blob data and can be used to store a history of unstructured data such as images, binary sensor data, or large text documents. In this article, we will use the database as a time-series blob storage and compare its performance with ReductStore, which is designed specifically for this use case.

TimescaleDB and ReductStore both have Python Client SDKs. We'll create simple Python functions to read and write data, then compare performance with different blob sizes. To repeat these benchmarks on your own machine, use this repository.

The Robot Operating System (ROS) stands as a versatile framework for developing sophisticated robotic applications with various sensors, including cameras. These cameras are relatively inexpensive and widely used as they can provide a wealth of information about the robot's environment.

Processing camera output with computer vision requires efficient solutions to handle massive amounts of data in real time. ROS 2 is designed with this in mind, but it is a communication middleware and does not provide a built-in solution for storing and managing large volumes of image data.

Addressing this challenge, this blog post will guide you through setting up ROS 2 with ReductStore—a time-series database for unstructured data optimized for edge computing, ensuring your robotic applications can process and store camera outputs effectively.

We are pleased to announce the release of the latest minor version of ReductStore, 1.9.0. ReductStore is a time series database designed for storing and managing large amounts of blob data.

To download the latest released version, please visit our Download Page.

What’s New in 1.9.0?​

This release, version 1.9.0, introduces several key improvements and features to enhance the overall performance and user experience. These updates include optimizations for disk space management, the inclusion of replication support in the Web Console, and the provision of license information in the HTTP API.

The MQTT protocol is a simple way to connect different sources of data to applications, which makes it very popular for IoT (Internet of Things) applications. Some MQTT brokers can save messages for a while, even when the MQTT client is offline. However, sometimes, you need to keep this data for a longer time. In those cases, it's a good idea to use a time series database.

There are many time series databases available, but if you need to store a history of images, sensor data, or Protobuf messages, you might want to use ReductStore. This database is designed to store a lot of blob data and works well with IoT and edge computing.

ReductStore has client SDKs (Software Development Kits) for many programming languages. This means you can easily use it in your existing system. In this example, we'll be using the Python SDK from ReductStore.

Now, let's make a simple MQTT application to see how this all works.