Using ReductStore in ROS applications

info

Some features described in this guide are available only in ReductStore Pro, including the ReductROS and ReductSelect extensions. Check the pricing page for more information about available features and extensions.

ReductStore is designed for robotics applications. It makes ROS messages, sensor payloads, and recorded robotics data queryable by storing them as timestamped records with labels and attachments.

This guide covers everything you need to know to use ReductStore in a ROS application: collecting data on a robot with ReductBridge, storing it in a local ReductStore instance, selectively replicating it to a cloud or central on-premises instance, and querying, visualizing, and exporting the data from there.

The following sections walk through each step, starting with an overview of the architecture and then diving into setup, replication, querying, and export.

How to read this guide

Use this page for a quick, runnable walkthrough of the complete ROS data flow, then follow the linked standalone guides for in-depth configuration and concepts.

Overview

ReductStore stores data as binary records and indexes each record by timestamp. It allows you to store a history of robotics data in its original format, while labels provide metadata for filtering and attachments store shared metadata such as ROS message schemas.

The ROS workflow uses the following components:

• ReductBridge subscribes to ROS topics on the robot, writes messages to ReductStore, assigns labels, and stores the schema metadata required to decode raw ROS messages.
• ReductStore stores the message payloads as timestamped records in a bucket. On the robot side, it runs locally on the same robot or edge computer.
• Replication tasks run on the robot-side ReductStore instance and copy only selected entries or records with matching labels to a cloud or central on-premises ReductStore instance.
• Cloud or central on-premises ReductStore stores replicated data from one or more robots and provides the main query and visualization endpoint for operators, dashboards, and analytics tools.
• ReductROS decodes stored ROS data when it is queried. It can extract messages from raw ROS records, MCAP files, and rosbag archives, and export filtered raw ROS data as MCAP.
• ReductSelect runs after ReductROS in the query pipeline. It treats the extracted JSON fields as SQL columns so queries can select, filter, aggregate, and reorder the decoded message data.

The diagram below shows the robot-side pipeline and the central storage and query layer. ROS nodes publish topics, ReductBridge subscribes to selected topics, and local ReductStore stores one entry per topic with the raw payload, labels, and $schema attachment. A replication task on the robot copies only the selected entries or label-matched records to the cloud or central on-premises ReductStore instance. When users query that data, ReductROS extracts the messages as JSON before ReductSelect runs SQL over the decoded fields.

ROS data collection, replication, and query workflow with ReductStore

The following sections examine each part of the workflow in more detail.

Data collection

Start by setting up data ingestion on the robot to see how ReductBridge simplifies collecting, labeling, and storing ROS data in ReductStore.

Clone the documentation repository, enter the example directory, and start the complete stack, including the ROS publisher, ReductBridge, and ReductStore:

git clone https://github.com/reductstore/website.git
cd website/docs/ros/example
docker compose up -d

Running ReductBridge

ReductBridge is an open-source project that can be installed on the robot or edge computer. It is available as a prebuilt binary for Linux, macOS, and Windows, or as a Docker container. You can find the installation instructions in the ReductBridge documentation. This example uses the Docker container with Docker Compose:

reduct-bridge:
  image: reduct/bridge:latest-ros2-jazzy
  restart: unless-stopped
  depends_on:
    - reductstore
    - ros2publisher
  network_mode: host
  environment:
    ROS_DOMAIN_ID: "0"
    ROS_HOME: /tmp/ros
    FASTDDS_BUILTIN_TRANSPORTS: UDPv4
  volumes:
    - ./bridge.toml:/etc/reduct-bridge/config.toml:ro
  command: ["reduct-bridge", "/etc/reduct-bridge/config.toml"]

See the full Docker Compose example.

The publisher and ReductBridge containers use the same ROS 2 environment:

• ROS_DOMAIN_ID: "0" places both processes in the same DDS domain.
• ROS_HOME: /tmp/ros provides a writable directory for ROS runtime files.
• FASTDDS_BUILTIN_TRANSPORTS: UDPv4 enables UDP/IPv4 transport between the isolated containers.

See the Data collection guide for details about these variables.

Configuring ReductBridge

ReductBridge uses one TOML file to connect three layers: an input subscribes to ROS 2 topics, a pipeline enriches and routes the records, and a remote writes them to ReductStore. Save the following file as bridge.toml next to docker-compose.yml. The Compose service mounts it at /etc/reduct-bridge/config.toml.

# ROS 2 input named "robot".
[inputs.ros2.robot]
# Use the same domain as the publishers.
domain_id = 0
node_name = "reduct_bridge_robot"
queue_size = 128
# Resolve ROS message schemas from this installation.
schema_paths = ["/opt/ros/jazzy"]

# Subscribe to camera images.
[[inputs.ros2.robot.topics]]
name = "/camera/image_raw"
entry_name = "/camera/image_raw"
# Use the message acquisition time.
# timestamp-start
timestamp = { field = "header.stamp", format = "ros_stamp" }
# timestamp-end
# Add labels that are constant for this topic.
# camera-static-start
labels = [
  { static = { source = "ros2", topic = "/camera/image_raw", sensor = "camera" } }
]
# camera-static-end

# Subscribe to GPS fixes.
[[inputs.ros2.robot.topics]]
name = "/gps/fix"
entry_name = "/gps/fix"
timestamp = { field = "header.stamp", format = "ros_stamp" }
# Extract coordinates and add constant labels.
# gps-dynamic-start
labels = [
  { field = "latitude", label = "x" },
  { field = "longitude", label = "y" },
  { field = "altitude", label = "z" },
  { static = { source = "ros2", topic = "/gps/fix", sensor = "gps" } }
]
# gps-dynamic-end

# Subscribe to IMU measurements.
[[inputs.ros2.robot.topics]]
name = "/imu/data"
entry_name = "/imu/data"
timestamp = { field = "header.stamp", format = "ros_stamp" }
labels = [
  { static = { source = "ros2", topic = "/imu/data", sensor = "imu" } }
]

# Route the ROS input to the local remote.
# pipeline-route-start
[pipelines.ros_to_reductstore]
remote = "local"
inputs = ["robot"]
# pipeline-route-end
# Copy the latest GPS coordinates and identify the robot.
# pipeline-labels-start
labels = [
  { from = "/gps/fix", labels = ["x", "y", "z"], to = "/camera/image_raw" },
  { from = "/gps/fix", labels = ["x", "y", "z"], to = "/imu/data" },
  { static = { robot = "robot-1" }, to = "*" }
]
# pipeline-labels-end

# Write records to the local ReductStore instance.
# remote-start
[remotes.reduct.local]
url = "http://127.0.0.1:8383"
token_api = "my-token"
bucket = "robot-data"
prefix = ""

# Create a rolling bucket when it is missing.
[remotes.reduct.local.create_bucket]
quota_type = "FIFO"
quota_size = "20GB"
# remote-end

The source is also available as the full bridge.toml example.

At a high level, this configuration subscribes to camera, GPS, and IMU topics; stores their original CDR payloads and ROS schemas; assigns sensor timestamps and labels; propagates the latest GPS coordinates to camera and IMU records; and writes everything to a rolling robot-data bucket.

For a field-by-field explanation of subscriptions, schema attachments, timestamp assignment, static and dynamic labels, cross-topic label propagation, and ReductStore output, see the standalone Data Collection guide.

Buffering, replication, and monitoring

ReductBridge handles data collection, but it needs a ReductStore instance for persistent storage. In a ROS application, the robot-side ReductStore has three roles: it buffers robotics data on local disk, replicates selected records to a cloud or central on-premises instance, and records operational diagnostics.

Running ReductStore

ReductStore is available as a Docker image, a Snap package, and prebuilt binaries for Linux, macOS, and Windows. See the Getting Started guide for all installation options. This example runs ReductStore with Docker Compose:

reductstore:
  image: reduct/store:latest
  restart: unless-stopped
  ports:
    - "127.0.0.1:8383:8383"
  environment:
    RS_API_TOKEN: my-token
    # Keep diagnostic entry paths stable across container recreation.
    RS_INSTANCE_NAME: ${ROBOT_NAME:-robot-1}

    # bucket-buffering-start
    # Keep the newest 20 GB of robot data on local disk.
    RS_BUCKET_1_NAME: robot-data
    RS_BUCKET_1_QUOTA_TYPE: FIFO
    RS_BUCKET_1_QUOTA_SIZE: 20GB
    # bucket-buffering-end

    # replication-start
    # Send selected topics to the public demo instance.
    RS_REPLICATION_1_NAME: robot-to-central
    RS_REPLICATION_1_SRC_BUCKET: robot-data
    RS_REPLICATION_1_DST_BUCKET: demo
    RS_REPLICATION_1_DST_HOST: https://play.reduct.store
    RS_REPLICATION_1_DST_TOKEN: reductstore
    RS_REPLICATION_1_DST_PREFIX: ${ROBOT_NAME:-robot-1}
    RS_REPLICATION_1_ENTRIES: "/camera/image_raw,/imu/data"
    # replication-end

    # compression-start
    # Compress local data after one day.
    RS_LIFECYCLE_1_NAME: compress-robot-data
    RS_LIFECYCLE_1_TYPE: compress
    RS_LIFECYCLE_1_BUCKET: robot-data
    RS_LIFECYCLE_1_OLDER_THAN: 1d
    RS_LIFECYCLE_1_INTERVAL: 1h
    # compression-end
  volumes:
    - reduct-data:/data

The Docker Compose example runs ReductStore on the robot, publishes its HTTP API on port 8383, and mounts a named volume at /data. The volume keeps records, bucket settings, and replication progress across container restarts.

Public demo server

The example replicates data to the demo bucket on the public Play server using its reductstore demo token. Do not send sensitive data to this shared server.

For production, replace these values with your destination URL, bucket, and a token that has permission to write to that bucket. The local API token is my-token, matching the ReductBridge configuration from the previous section.

At startup, the service provisions a rolling 20 GB robot-data bucket, a task that replicates camera and IMU records under a robot-1 destination prefix, and a lifecycle policy that compresses local data after one day. It also records replication, lifecycle, usage, and log events in the $system bucket.

For a complete explanation of local buffering, selective fleet replication, compression, failure behavior, and monitoring, see the standalone Buffering, Replication, and Monitoring guide.

Data querying and visualization

ROS 2 messages are stored as binary CDR payloads, which preserves the original data but cannot be plotted or analyzed directly. ReductROS uses the stored $schema attachment to decode selected messages on the server and return JSON. The querying application therefore does not need ROS or the original message packages installed locally.

A typical query follows four stages:

1. Select entries and a time range, such as /gps/fix for the last hour.
2. Filter records by stored labels such as robot, sensor, x, or y before reading their payloads.
3. Use ReductROS to resolve the entry's $schema attachment and decode matching CDR messages as JSON. Binary fields can remain arrays, be encoded as base64, or be converted to JPEG for supported image messages.
4. Return the decoded records to an SDK or Grafana. When more complex processing is required, ReductSelect can apply SQL to the extracted JSON before it leaves the server.

This order is important for large robotics datasets: time, entry, and stored-label filters reduce the input first; only the remaining payloads are read, decoded, and transferred. See the Data Querying guide and Conditional Query Reference for the common time-range, entry-pattern, and label-filtering options.

Using SDKs

Official SDKs can use ReductROS to query stored robotics data and receive decoded ROS messages as JSON. Applications and browser-based tools can then process the JSON directly. See the standalone Using SDKs guide for a runnable Python example using the public ROS dataset.

Using Grafana

The ReductStore Grafana data source can visualize decoded ROS fields directly from a ReductStore query. See the standalone Using Grafana guide for a runnable example that plots interval-averaged GPS latitude and longitude from the public ROS dataset.

Using Foxglove

ReductROS can export a bounded selection of raw ROS records as MCAP for remote inspection in Foxglove. See the standalone Using Foxglove guide for a runnable workflow using the public ROS dataset, a temporary query link, and an infrared Image panel.

Export data

For scripted or local exports, install the ReductStore CLI and use its cp command. The examples use the public orion dataset and include the Play server's public reductstore token. Do not use the demo token for private or production data.

Export records in their native format

This command selects entries with a prefix wildcard and copies one record from each entry:

reduct-cli cp \
  https://reductstore@play.reduct.store/replica/orion \
  ./orion-native \
  --entries 'right_ir/rotated/*' \
  --limit 1

The CLI creates a directory for each selected entry under ./orion-native and writes each original record payload without converting it. Each filename uses the record timestamp, and its extension is inferred from the content type. Add --with-meta to write record labels, content type, and content length to a JSON file alongside each payload.

Keep wildcard arguments quoted so that the shell passes them to reduct-cli instead of expanding them as local filenames. Adjust the entry prefix, limit, and destination for your dataset. Add --start and --stop to bound the time range when needed.

Export records as MCAP

To convert raw ROS records while downloading them, pass the ReductROS #ext directive as the --when condition:

reduct-cli cp \
  https://reductstore@play.reduct.store/replica/orion \
  ./orion-mcap \
  --entries 'right_ir/rotated/image_raw' \
  --limit 1 \
  --when '{"#ext":{"ros":{"export":{"format":"mcap"}}}}'

ReductROS reads the entry's $schema attachment and returns an application/mcap result. The CLI recognizes that content type and writes the result with an .mcap extension under ./orion-mcap.

The #ext directive runs ReductROS as part of the conditional query. You can open the downloaded MCAP file directly in Foxglove.

Offboarding without vendor lock-in

Both export options support offboarding from ReductStore. Native export preserves each record's original bytes; use --with-meta to retain record metadata, and export the $schema entry attachment separately with the CLI attachment command when the destination must decode raw ROS payloads. MCAP packages ROS messages, topics, and schemas in an open format supported by robotics tools.