Send Logs to Elasticsearch

This integration is part of the Pro edition.

This integration enables you to send schema-free JSON documents directly to any Elasticsearch local or cloud-hosted instance by using the vendor.elastic helper.

Sending Select Traffic Logs to Elasticsearch

A ready-to-run script that enables you to send select traffic logs to Elasticsearch can be found here.

var kflQuery    = "gql and (src.name == 'my-pod-name' or dst.name == 'my-pod-name')";
var ACTIVE      = true;  // change to false to disable this script

// Use environment variables (recommended) or change these variables locally
var elaIdx      = env.ELASTIC_INDEX;
var elaCloId    = env.ELASTIC_CLOUD_ID;
var elaApiKey   = env.ELASTIC_API_KEY;

function onItemCaptured(data) {
    if (!ACTIVE) return;
    try{
        if (kflQuery.match(kflQuery, data)){
            vendor.elastic(
                "",     // URL is ignored for Elastic Cloud
                elaIdx,
                data,   // Payload
                "",     // Username is ignored for Elastic Cloud
                "",     // Password is ignored for Elastic Cloud
                elaCloId,
                elaApiKey
            );
        }
    }
    catch(error){
        console.error("Elastic Traffic Logs", error);
    }
}

Change the KFL query and the Elasticsearch Authentication variables at the top of the script to fit your use-case and put the script in the script folder.

TL;DR

The script uses the onItemCaptured hook, the kfl.match helper in conjunction with a KFL query to identify the select traffic logs.

For example, the following KFL query can be used to filter pod specific ingress/egress GraphQL traffic:

gql and (src.name == "my-pod-name" or dst.name == "my-pod-name")

The vendor.elastic helper is used to send the traffic logs to an Elasticsearch cloud instance.

Prerequisites

Using the Elasticsearch integration requires the Pro edition and is currently supported only by the CLI. If you haven’t done so:

  1. Install the CLI, by following these instructions.
  2. Sign up to the Pro edition by running the pro command:
kubeshark pro

More information about upgrading to the Pro edition can be found in the Upgrading & Downgrading section.

Ensure your configuration file includes the following configuration at a minimum:

license: FT7YKAYBAE****************AA=
scripting:
    env:
        ELASTIC_CLOUD_ID: <your-elastic-cloud-id>
        ELASTIC_API_KEY:  <your-elastic-api-key>
        ELASTIC_INDEX:    <your-elastic-index-name>
    source: /path/to/your/script/folder/
    watchScripts: true