Stroom – A Robust Platform For Data Processing, Storage, And Analysis

Stroom is a data processing, storage and analysis platform. It is scalable – just add more CPUs / servers for greater throughput.

It is suitable for processing high volume data such as system logs, to provide valuable insights into IT performance and usage.

Stroom provides a number of powerful capabilities:

• Data ingest. Receive and store large volumes of data such as native format logs. Ingested data is always available in its raw form.
• Data transformation pipelines. Create sequences of XSL and text operations, in order to normalise or export data in any format. It is possible to enrich data using lookups and reference data.
• Integrated transformation development. Easily add new data formats and debug the transformations if they don’t work as expected.
• Scalable Search. Create multiple indexes with different retention periods. These can be sharded across your cluster.
• Dashboards. Run queries against your indexes or statistics and view the results within custom visualisations.
• Statistics. Record counts or values of items over time, providing answers to questions such as “how many times has a specific machine provided data in the last hour/day/month?”

Get Stroom

To run Stroom in docker do the following:

# Download and extract Stroom v7.0 stack
bash <(curl -s https://gchq.github.io/stroom-resources/v7.0/get_stroom.sh)

# Navigate into the new stack directory
cd stroom_core_test/stroom_core_test*

# Start the stack
./start.sh