Skip to main content

Query CSV and Parquet Records in Place with ReductSelect

· 10 min read
Anthony Cavin
Co-founder & CEO - Data, ML & Robotics Systems

Machines often send several measurements at once. A controller might send a CSV packet every 200 milliseconds, with several readings in each packet. ReductStore stores each packet as a timestamped record, without splitting it into individual measurements. This keeps the original packets intact, even as the number of records grows.

Later, you might want to find readings above a threshold, calculate an average over a time window, or combine the packets into a Parquet file. Rather than downloading all the raw data and processing it in your application, you can use ReductSelect to process the records directly on the server.

Disaster Recovery for Time-Indexed Sensor Data

· 7 min read
Alexey Timin
Co-founder & CTO - Database & Systems Engineering

A robot or production line writes sensor data continuously. Eventually, a machine crashes, a disk fails, or a bad deployment corrupts an index. The real question is how much data loss and downtime you can afford when that happens.

The Disaster Recovery guide is the product reference: it describes the available mechanisms and their settings. This post focuses on choosing and deploying them. It covers cold backup, hot backup with the CLI, hot standby, and read-only replicas, including the downtime each requires and when each is appropriate for a robot or plant that cannot stop.

Unsupervised anomaly detection with Anomalib in Python

· 10 min read
Anthony Cavin
Co-founder & CEO - Data, ML & Robotics Systems

Anomaly detection pipeline: a camera feeds Anomalib, which scores each frame, and ReductStore keeps every frame with its anomaly labels

Anomalib makes it relatively easy to train an anomaly detection model on your own images. But once you want to use it in production, you also need to handle inference, evaluate predictions, and keep track of the images that triggered them.

In this tutorial, we'll use anomalib to train and run an anomaly detection model in Python. Then we'll look at how to store images alongside their anomaly scores, so you can review detections and use them for future training.