ReductStore's Blog

Mar 9, 2023. | By: Alexey Timin

Hey, I’ve released version 0.8.0 of Reduct CLI, the Python package for managing data stored in ReductStore. This release includes two new features that will be particularly helpful for our public datasets hosted on ReductStore, where metadata can be used to provide important context for the data.

1. The rcli export folder command now has a new option --with-metadata, which exports meta information about a record and its labels into a JSON file along with the content of the record.

2. The rcli export commands now accept integers for --start and --stop options, which means you can specify a time interval in Unix time or use it as a range of record IDs if you don’t care about time:

    # Export records with ID from 0 to 5 
    rcli export folder instance/bucket ./export_path --start 0 --stop 5
   

I hope these new features make it even easier to work with ReductStore and manage your data. As always, if you have any questions or feedback, please don’t hesitate to reach out!

Thanks for using ReductStore!

[Read More]

Mar 7, 2023. | By: Alexey Timin

The ReductStore Project hosts the free “cats” dataset, which contains about 10K photos of cats in JPEG format with eye coordinates, a month, and ears as labels. In this example, we can learn how to download the dataset from the ReductStore instance and draw the features using the OpenCV library.

Installing Dependencies

First, we need to install the ReductStore Client SDK for Python to download the photos and labels. We also need the OpenCV Python library for drawing features and Pillow to display images in the Jupyter environment:

pip install reduct-py opencv-python Pillow

Getting Data

To retrieve data we need the URL of the ReductStore instance, the bucket name, where we store our datasets and an API token with read access, so that we can connect to the database by using the Client class:

from reduct import Client, Bucket

HOST = "https://play.reduct.store"
API_TOKEN = "dataset-read-eab13e4f5f2df1e64363806443eea7ba83406ce701d49378d2f54cfbf02850f5"
BUCKET = "datasets"

client = Client(HOST, api_token=API_TOKEN)

bucket: Bucket = await client.get_bucket(BUCKET)

[Read More]

Mar 5, 2023. | By: Alexey Timin

Using snap, users can now easily install and manage ReductStore on various Linux distributions.

Snap is a universal package manager developed by Canonical, the company behind Ubuntu Linux. It allows developers to package their applications and dependencies into a single package that can be installed on any Linux distribution that supports snap, without worrying about different packaging formats and dependency conflicts.

[Read More]

Feb 22, 2023. | By: Alexey Timin

We are excited to announce the release of ReductStore Client SDK for C++ v1.3.0! This release includes support for the ReductStore API v1.3.0 with labels and content type.

[Read More]

Feb 19, 2023. | By: Alexey Timin

Hey everyone,

I’m happy to announce that we have released Reduct CLI client v0.7.0 with some minor improvements and bug fixes. We started using the tool in real applications and faced some problems exporting data from a ReductStore instance when the connection is slow and we have many entries to download asynchronously.

First of all, it wasn’t very convenient to count all needed entries in the rcli export command. Now we can use wildcards:

rcli export folder instance/bucket ./export_path  --entries=sensor-*

[Read More]

Feb 10, 2023. | By: Alexey Timin

Hey everyone,

we are pleased to announce the release of version 1.3.0 of the ReductStore SDK for JavaScript. This version supports the new features of ReductStore v1.3, including labels and content type.

Now you can write a record with MIME type and labels to ReductStore:

const client = new Client("https://play.reduct.store");

const bucket = await client.getOrCreateBucket("bucket");
    
const record = await bucket.beginWrite("entry-1", {
    contentType:"text/plain", 
    labels: {type:"example"}
})

await record.write("Some text");

[Read More]

Feb 7, 2023. | By: Alexey Timin

Hello everyone,

We’re excited to announce the release of version 0.6.0 of the Reduct CLI! This release brings a number of new features for working with the new ReductStore API v1.3.

[Read More]

Jan 26, 2023. | By: Alexey Timin

We are happy to announce the release of version 1.3.0 of the ReductStore SDK for Python! This release introduces several new features to help users better organize and filter their data.

One of the most notable new features is the ability to attach labels to data when writing and querying. Labels are key-value pairs that can be used to classify and categorize data. For example, you might use labels to store metadata about a record, such as its md5 sum or class. To start using labels, you need the version of the ReductStore database higher than 1.3.0.

[Read More]

Jan 26, 2023. | By: Alexey Timin

Hello everyone,

We are excited to announce the release of version 1.3.0 of the ReductStore database! This update brings a number of new features and improvements that we believe will enhance your experience with the database.

License Change

First and foremost, we’ve changed the project license, switching from AGPLv3 to MPL-2.0 which allows usage of the program as a service over the network in proprietary software. This change is made to avoid any misunderstanding in the future, and to align with our goal of encouraging contributions back to the project while allowing everyone to use it for free.

[Read More]

Jan 20, 2023. | By: Alexey Timin

Hello everyone,

We’re excited to announce the release of version 0.5.0 of the Reduct CLI! This release brings a number of new features to improve data export and mirroring. Here are the highlights:

Limit parallel tasks

We added a global option --parallel to limit the number of entries to be exported or mirrored in parallel. This can be useful when a bucket has a large number of entries and the network connection is slow. Usage example:

rcli export --parallel 5 folder server_1/bucket_1 . 

[Read More]

Jan 15, 2023. | By: Alexey Timn

This quick start guide will walk you through the process of installing and using the ReductStore JavaScript client SDK to interact with a ReductStore instance.

Installing the SDK

To install the ReductStore SDK, you will need to have Node.js 16 or higher installed on your machine. Once Node.js is installed, you can use the npm package manager to install the reduct-js package:

npm install reduct-js

[Read More]

Jan 8, 2023. | By: Alexey Timn

This quick start guide will walk you through the process of installing and using the ReductStore Python client SDK to interact with a ReductStore instance.

Installing the SDK

To install the ReductStore SDK, you will need to have Python 3.7 or higher installed on your machine. Once Python is installed, you can use the pip package manager to install the reduct-py package:

pip install reduct-py

[Read More]

Jan 7, 2023. | By: Alexey Timin

We often use blob storage like S3, if we need to store data of different formats and sizes somewhere in the cloud or in our internal storage. Minio is an S3 compatible storage which you can run on your private cloud, bare-metal server or even on an edge device. You can also adapt it to keep historical data as a time series of blobs. The most straightforward solution would be to create a folder for each data source and save objects with timestamps in their names:

bucket
 |
 |---cv_camera
        |---1666225094312397.bin
        |---1666225094412397.bin
        |---1666225094512397.bin

[Read More]

Jan 4, 2023. | By: Alexey Timin

I think, NGINX doesn’t need any introductions. It is one of the most widely used HTTP servers and reverse proxies. You can route your microservices or monolith application through it and make it responsible for:

• TSL encryption
• Basic HTTP authorization
• Map public URL to services or applications
• Load balancing

[Read More]

Dec 29, 2022. | By: Alexey Timin

The MQTT protocol is very popular in IoT applications. It is a simple way to connect different data sources with your application by using a publish/subscribe model. Sometimes you may want to keep a history of your MQTT data to use it for model training, diagnostics or metrics. If your data sources provide different formats of data that can not be interpreted as time series of floats, ReductStore is what you need.

Let’s make a simple MQTT application to see how it works.

[Read More]

Dec 26, 2022. | By: Alexey Timin

Hello everyone,

We’re excited to announce the release of version 0.4.0 of the Reduct CLI! This release includes a new rcli export folder command that allows you to export data from a bucket in your storage engine to a local folder on your computer. This can be useful if you want to make a copy of your data for backup or offline access. For more information, see the documentation.

[Read More]

Dec 23, 2022. | By: Alexey Timin

Hello, everyone! ReductStore has released v1.2.0 of its JavaScript SDK. This update includes support for ReductStore API version 1.2 with the new Client.me method, which allows you to retrieve information about your current API token and its permissions.

[Read More]

Dec 22, 2022. | By: Alexey Timin

Hello, everyone! We are excited to announce the release of the Python client SDK v1.2.0! This release includes support for the ReductStore HTTP API v1.2 and several other improvements.

[Read More]

Dec 20, 2022. | By: Alexey Timin

We are excited to announce the release of ReductStore Client SDK for C++ v1.2.0! This release includes updated documentation after we renamed the project from “Reduct Storage” to “ReductStore”, and supports ReductStore HTTP API 1.2.0 with the endpoint GET /api/v1/me.

[Read More]

Dec 19, 2022. | By: Alexey Timin

Hello, everyone!

We’re excited to announce the release of ReductStore v1.2.0. In this update, we’ve made some important changes to improve the way tokens and buckets are managed. Specifically, when you create a new token, the database will now check to ensure that the buckets specified in the token’s permissions actually exist. This will help prevent errors and ensure that users have the proper access to the appropriate buckets.

[Read More]

Dec 17, 2022. | By: Alexey Timin

This guide will help you get started with the ReductStore C++ SDK.

Requirements

The ReductStore C++ SDK is written in C++20 and uses CMake as a build system. To install it, you will need:

• C++ compiler with C++20 support (we use GCC-11.2)
• cmake 3.18 or higher
• conan 1.40 or higher (it is optional, but conan is a convenient package manager)

Currently, we have only tested the SDK on Linux AMD64, but if you need to port it to another operating system or platform, you can create an issue for assistance.

Installing

To install the ReductStore C++ SDK, follow these steps:

[Read More]

Dec 10, 2022. | By: Alexey Timin

When it comes to computer vision, data storage is a critical component. You need to be able to store images for model training, as well as the results of the processing for model validation. There are a few ways to go about this, each with its own advantages and disadvantages. In this post, we’ll take a look at three different ways to store data in computer vision applications: a file system, an S3-like object storage and ReductStore. We’ll also discuss some of the pros and cons of each option.

A Simple Computer Vision Application

For demonstration purposes, we’ll use a simple computer vision application which is connected to a CV camera and runs on an edge device:

The camera driver captures images from the CV camera every second and sends them to the model. The model detects something and shows the results in the user interface.

So far it isn’t too complicated, let’s see how we can deal with the data here.

[Read More]

Recent Posts

• CLI Client for ReductStore v0.8.0 has been released

  #News #Cli

• How to Use 'Cats' dataset with Python ReductStore SDK

  #Tutorials #Computer-vision #Sdks

• Streamline your edge computing workflows with ReductStore, now available on Snap

  #News #Tutorials

• ReductStore Client SDK for C++ v1.3.0 with Labels Support

  #News #Sdks #Tutorials

Popular Tags

Tutorials (14) Computer-vision (3) Sdks (10) News (14) Cli (5) Iot (2) Comparisons (1)