LogSpace

LogSpace is a DBMS developed by Positive Technologies specifically for storing large volumes of event information from diverse sources for security teams' needs

Overview

Challenges in processing and storing large volumes of event information:

Data storage requires excessive disk space.

Events vary greatly in their structure.

Due to string data storage, traditional DBMSs cannot process tens of thousands of events per second.

Why LogSpace is needed

A SIEM system processes hundreds of millions of records from various logs daily, totaling hundreds of gigabytes. Even with modern hardware capabilities, processing and storing such data volumes present serious engineering challenges. SIEM developers fall into two groups: proprietary database owners and open-source solution users.

Challenges with open-source DBMSs:

Columnar DBMSs handle structured information well but lose efficiency with raw text event data.
Text-oriented solutions (such as Elasticsearch) are significantly less efficient than columnar DBMSs for structured information storage.
Users have no influence on development progress or the ability to guarantee data storage quality.

We have addressed these open-source DBMS issues by developing our own database that uses a columnar format while accounting for log storage specifics.

LogSpace Advantages

C++ development

This high-performance language is crucial for developing high-load systems.

Columnar storage format

This format enables you to handle hundreds of millions of records daily while significantly reducing the SIEM system load.

SQL for data operations

It provides a familiar user interface.

High compression density

It offers a 6x increase in data volume and storage duration compared to open-source DBMSs.

Auto-adaptability

Different encoding methods per column block allow for working with any event type.

Secure backup

A flexible zero-overhead backup system supports partial archiving and instant recovery.

Thinking about the best way to protect your company?

During the consultation we'll propose a solution precisely tailored to your organization.