Data exporters
The data exporters are the way to export alerts and events from Otoroshi to an external storage.
To try them, you can folllow this tutorial.
Common fields
Type
: the type of event exporterEnabled
: enabled or not the exporterName
: given name to the exporterDescription
: the data exporter descriptionTags
: list of tags associated to the moduleMetadata
: list of metadata associated to the module
All exporters are split in three parts. The first and second parts are common and the last are specific by exporter.
Filtering and projection
: section to filter the list of sent events and alerts. The projection field allows you to export only certain event fields and reduce the size of exported data. It’s composed ofFiltering
andProjection
fields. To get a full usage of this elements, read this sectionQueue details
: set of fields to adjust the workers of the exporter.Buffer size
: if elements are pushed onto the queue faster than the source is consumed the overflow will be handled with a strategy specified by the user. Keep in memory the number of events.JSON conversion workers
: number of workers used to transform events to JSON format in paralellSend workers
: number of workers used to send transformed eventsGroup size
: chunk up this stream into groups of elements received within a time window (the time window is the next field)Group duration
: waiting time before sending the group of events. If the group size is reached before the group duration, the events will be instantly sent
For the last part, the Exporter configuration
will be detail individually.
Matching and projections
Filtering is used to include or exclude some kind of events and alerts. For each include and exclude field, you can add a list of key-value.
Let’s say we only want to keep Otoroshi alerts
{ "include": [{ "@type": "AlertEvent" }] }
Otoroshi provides a list of rules to keep only events with specific values. We will use the following event to illustrate.
{
"foo": "bar",
"type": "AlertEvent",
"alert": "big-alert",
"status": 200,
"codes": ["a", "b"],
"inner": {
"foo": "bar",
"bar": "foo"
}
}
The rules apply with the previous example as event.
{
"<key>": "<value>"
}
a field key with value
{
"foo": "bar"
}
Keep events with foo as key and bar as value
{
"<key>": {
"$wildcard": "<value>*"
}
}
a field starting with value
{
"type": {
"$wildcard": "Alert*"
}
}
Keep events with type field starting with Alert
{
"<key>": {
"<key2>": "<value>"
}
}
a sub-field with value
{
"inner": {
"foo": "bar"
}
}
Keep events with sub field foo at value bar
{
"<key>": "<number>"
}
a field with the specific value as number
{
"status": 200
}
Keep events with status code at 200 (as number check)
{
"<key>": {
"$gt": "<number>"
}
}
a field with number value greater than number
{
"status": {
"$gt": 100
}
}
Keep events with status code greater than 100
{
"<key>": {
"$gte": "<number>"
}
}
a field with number value greater or equal to number
{
"status": {
"$gte": 100
}
}
Keep events with status code greater or equal to 100
{
"<key>": {
"$lt": "<number>"
}
}
a field with number value lower than number
{
"status": {
"$lt": 100
}
}
Keep events with status code lower than 100
{
"<key>": {
"$lte": "<number>"
}
}
a field with number value lower or equal to number
{
"status": {
"$lte": 100
}
}
Keep events with status code lower or equal to 100
{
"<key>": {
"$between": {
"min": "<number>",
"max": "<number>"
}
}
}
a field with value between two values (exclusive)
{
"status": {
"$between": {
"min": 100,
"max": 200
}
}
}
Keep events with status code between 100 and 200 (100 and 200 won't match)
{
"<key>": {
"$between": {
"min": "<number>",
"max": "<number>"
}
}
}
a field with value between two values (inclusive)
{
"status": {
"$between": {
"min": 100,
"max": 200
}
}
}
Keep events with status code between 100 and 200 (100 and 200 will match)
{
"<key>": {
"$and": [
{
"<key2>": "<value>"
},
{
"<key3>": "<value>"
}
]
}
}
an object with two fields with values
{
"inner": {
"$and": [
{
"foo": "bar"
},
{
"bar": "foo"
}
]
}
}
Keep events matching the list of key-value
{
"$or": [
{
"<key2>": "<value>"
},
{
"<key3>": "<value>"
}
]
}
an object matching at least one condition of the list
{
"$or": [
{
"method": "DELETE"
},
{
"protocol": "http"
}
]
}
Keep event whose method is http OR method is DELETE OR both
{
"$nor": [
{
"<key2>": "<value>"
},
{
"<key3>": "<value>"
}
]
}
an object that matches no conditions of the list
{
"$nor": [
{
"method": "DELETE"
},
{
"protocol": "http"
}
]
}
Keep events whose method is not DELETE AND protocol is not http
{
"<key>": [
"<value>",
"<value2>"
]
}
an array field with values
{
"codes": [
"a",
"b"
]
}
Keep events with an array codes which strictly containing values a and b
{
"<key>": {
"$contains": "<value>"
}
}
an array field containing a specific value
{
"codes": {
"$contains": "a"
}
}
Keep events with an array codes containing an a value
{
"<key>": {
"$contains": {
"key": "<subkey>",
"value": "<value>"
}
}
}
an object containing a key subkey with given value
{
"target": {
"$contains": {
"key": "scheme",
"value": "https"
}
}
}
Keep events whose target contains a field 'scheme' valued to 'https'
{
"<key>": {
"$all": [
"<value>",
"<value>"
]
}
}
and array field containing all specific values
{
"codes": {
"$all": [
"a",
"b"
]
}
}
Keep events with an array codes containing at minima a and b values
{
"<key>": {
"$regex": "<value>"
}
}
a string field whose value match given regular expression
{
"url": {
"$regex": ".*api.*"
}
}
Keep events with url containing 'api'
{
"<key>": {
"$in": [
"<value>",
"<value2>"
]
}
}
a field containing one of the given value
{
"method": {
"$in": [
"PUT",
"POST"
]
}
}
Keep events whose method is PUT or POST
{
"<key>": {
"$nin": [
"<value>",
"<value2>"
]
}
}
a field containing none of the given value
{
"method": {
"$nin": [
"PUT",
"POST"
]
}
}
Keep events whose method is neither PUT nor POST
{
"<key>": {
"$size": "<number>"
}
}
an array field whose size is given value
{
"headers": {
"$size": 12
}
}
Keep events with exactly 12 headers
{
"<key>": {
"$not": "<condition>"
}
}
an object that does not satisfy condition
{
"url": {
"$not": {
"$regex": ".*api.*"
}
}
}
Keep events whose url does not contain 'api'
{
"<key>": {
"$eq": "<value>"
}
}
a field key with value
{
"foo": {
"$eq": "bar"
}
}
Keep events with foo as key and bar as value
{
"<key>": {
"$ne": "<value>"
}
}
a field key whose value is not provided value
{
"foo": {
"$ne": "bar"
}
}
Keep events with foo field not equal to bar
{
"<key>": {
"$exists": "<entry>"
}
}
an object field containing given entry as key
{
"target": {
"$exists": "scheme"
}
}
Keep events whose target object contains a schema field
Projection is a list of fields to export. In the case of an empty list, all the fields of an event will be exported. In other case, only the listed fields will be exported.
"<field>": true
Description
{
"<field>": true
}
Include given field in result.
Expression
{
"foo": true
}
Event
{
"headers": [
{
"key": 1,
"value": "1"
},
{
"key": 2,
"value": "2"
}
],
"foo": 1
}
Result
{
"foo": 1
}
$at
Description
{
"<target>": {
"$at": "<location>"
}
}
Values <target> with value <at> location
Expression
{
"h1": {
"$at": "headers.0.value"
}
}
Event
{
"headers": [
{
"key": 1,
"value": "1"
},
{
"key": 2,
"value": "2"
}
]
}
Result
{
"h1": "1"
}
$atIf
Description
{
"<target>": {
"$atIf": {
"path": "<path>",
"predicate": {
"at": "<at>",
"value": "<value>"
}
}
}
}
Put <path> value in <target> if value at <at> match <value>
Expression
{
"r": {
"$atIf": {
"path": "hs.0.value",
"predicate": {
"at": "scheme",
"value": "HTTPS"
}
}
}
}
Event
{
"hs": [
{
"key": 1,
"value": "1"
},
{
"key": 2,
"value": "2"
}
],
"scheme": "HTTPS"
}
Result
{
"r": "1"
}
$pointer
Description
{
"<target>": {
"$pointer": "<jsonPointer>"
}
}
Allow to get a json value using JSON pointer spec
Expression
{
"h1": {
"$pointer": "/headers/0/value"
}
}
Event
{
"headers": [
{
"key": 1,
"value": "1"
},
{
"key": 2,
"value": "2"
}
]
}
Result
{
"h1": "1"
}
$pointerIf
Description
{
"<target>": {
"$pointerIf": {
"path": "<path>",
"predicate": {
"pointer": "<pointer>",
"value": "<value>"
}
}
}
}
Put value at <path> in <target> field if and only if value at <pointer> equals <value>. <path> and <pointer> fields are resolved using json pointer spec.
Expression
{
"h1": {
"$pointerIf": {
"path": "/headers/0/value",
"predicate": {
"pointer": "/scheme",
"value": "HTTP"
}
}
}
}
Event
{
"headers": [
{
"key": 1,
"value": "1"
},
{
"key": 2,
"value": "2"
}
],
"scheme": "HTTP"
}
Result
{
"h1": "1"
}
$path
Description
{
"<target>": {
"$path": "<jsonPath>"
}
}
Put value located at <jsonPath> in <target>. <jsonPath> is resolved using json path specification.
Expression
{
"hs": {
"$path": "$.headers[1:]"
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
]
}
Result
{
"hs": [
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
]
}
$pathIf
Description
{
"<target>": {
"$pathIf": {
"path": "<jsonPath>",
"predicate": {
"path": "<predicateJsonPath>",
"value": "<value>"
}
}
}
}
Put value located at <jsonPath> in <target>, if and only if value at <predicateJsonPath> equals <value>.
Expression
{
"test": {
"$pathIf": {
"path": "$.headers[1:]",
"predicate": {
"path": "$.scheme",
"value": "HTTPS"
}
}
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
],
"scheme": "HTTPS"
}
Result
{
"test": [
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
]
}
$compose (array result)
Description
{
"<target>": {
"$compose": [
{
"<subtarget>": {
"$path": "<path>"
}
}
]
}
}
Build an array in <target> field, by composing severals operators : $path, $at, $pointer.
Expression
{
"result": {
"$compose": [
{
"values": {
"$path": "$.headers[0:].value"
}
},
{
"keys": {
"$path": "$.headers[0:].key"
}
}
]
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
],
"scheme": "HTTPS"
}
Result
{
"result": [
{
"values": [
"v1",
"v2",
"v3"
]
},
{
"keys": [
"k1",
"k2",
"k3"
]
}
]
}
$compose (object result)
Description
{
"<target>": {
"$compose": {
"<subtarget>": {
"$path": "<path>"
}
}
}
}
Build an object in <target> field, by composing severals operators : $path, $at, $pointer.
Expression
{
"result": {
"$compose": {
"values": {
"$path": "$.headers[0:].value"
},
"keys": {
"$path": "$.headers[0:].key"
}
}
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
]
}
Result
{
"result": {
"values": [
"v1",
"v2",
"v3"
],
"keys": [
"k1",
"k2",
"k3"
]
}
}
$value
Description
{
"<target>": {
"$value": "<value>"
}
}
Put <value> in <target> field. <value> can be either a primtive type, an object or an array.
Expression
{
"headers": {
"$value": []
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
},
{
"key": "k3",
"value": "v3"
}
]
}
Result
{
"headers": []
}
$spread
Description
{
"$spread": true
}
Include all properties of source object, this can be used as base for futher transformation with below operators/
Expression
{
"$spread": true
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS",
"foo": 1
}
Result
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS",
"foo": 1
}
{"<field>": false}
Description
{
"$spread": true,
"<field>": false
}
Exclude <field> field from resulting object
Expression
{
"$spread": true,
"foo": false
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS",
"foo": 1
}
Result
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS"
}
$remove
Description
{
"$spread": true,
"<field>": {
"$remove": true
}
}
Exclude <field> field from resulting object, like for {"<field>": false}
Expression
{
"$spread": true,
"foo": {
"$remove": true
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS",
"foo": 1
}
Result
{
"headers": [
{
"key": "k1",
"value": "v1"
}
],
"scheme": "HTTPS"
}
$header
Description
{
"<target>": {
"$header": {
"path": "<path>",
"name": "<name>"
}
}
}
Put specified header value in <target>. <path> indicate an array of key/value headers, <name> indicate the name of the header.
Expression
{
"hostValue": {
"$header": {
"path": "headers",
"name": "Host"
}
}
}
Event
{
"headers": [
{
"key": "Host",
"value": "otoroshi.oto.tools:9999"
},
{
"key": "Accept",
"value": "application/json"
}
]
}
Result
{
"hostValue": "otoroshi.oto.tools:9999"
}
$includeAllKeysMatching
Description
{
"$spread": true,
"<subname>": {
"$includeAllKeysMatching": [
"<expression>"
]
}
}
Filter an object entries based on StartsWith, Wildcard or Regex expression. This filter must be used in a sub-object whose name <subname> doesn't matter, since it won't appear in resulting projection. This operator is an array, which mean you can have several filters. A logical OR will be applied between filters, therefore an entry just need to match one filter to be kept.
Expression
{
"$spread": true,
"_": {
"$includeAllKeysMatching": [
"Wildcard(f*)",
"StartsWith(fi)",
"bar",
"Regex(.*lo)"
]
}
}
Event
{
"foo": 1,
"foobar": 2,
"bar": 3,
"baz": 4,
"hello": "world",
"fifou": "test"
}
Result
{
"foo": 1,
"foobar": 2,
"bar": 3,
"hello": "world",
"fifou": "test"
}
$excludeAllKeysMatching
Description
{
"$spread": true,
"<subname>": {
"$excludeAllKeysMatching": [
"<expression>"
]
}
}
Filter an object entries, excluding them based on StartsWith, Wildcard or Regex expression. This filter must be used in a sub-object whose name <subname> doesn't matter, since it won't appear in resulting projection. This operator is an array, which mean you can have several filters. A logical OR will be applied between filters, therefore an entry just need to match one filter to be deleted.
Expression
{
"$spread": true,
"_": {
"$excludeAllKeysMatching": [
"Wildcard(fo*)",
"StartsWith(fi)",
"bar",
"Regex(.*lo)"
]
}
}
Event
{
"foo": 1,
"foobar": 2,
"bar": 3,
"baz": 4,
"hello": "world",
"fifou": "test"
}
Result
{
"baz": 4
}
$jq
Description
{
"<target>": {
"$jq": "<jqExpression>"
}
}
Fill target field with provided JQ selector
Expression
{
"headerKeys": {
"$jq": "[.headers[].key]"
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
}
]
}
Result
{
"headerKeys": [
"k1",
"k2"
]
}
$jqIf
Description
{
"<target>": {
"$jqIf": {
"filter": "<jqExpression>",
"predicate": {
"path": "<path>",
"value": "<value>"
}
}
}
}
Fill target field with provided JQ selector if and only if value located at <path> is equal to <value>
Expression
{
"headerKeys": {
"$jqIf": {
"filter": "[.headers[].key]",
"predicate": {
"path": "target.scheme",
"value": "https"
}
}
}
}
Event
{
"headers": [
{
"key": "k1",
"value": "v1"
},
{
"key": "k2",
"value": "v2"
}
],
"target": {
"scheme": "https"
}
}
Result
{
"headerKeys": [
"k1",
"k2"
]
}
Elastic
With this kind of exporter, every matching event will be sent to an elastic cluster (in batch). It is quite useful and can be used in combination with elastic read in global config
Cluster URI
: Elastic cluster URIIndex
: Elastic indexType
: Event type (not needed for elasticsearch above 6.x)User
: Elastic User (optional)Password
: Elastic password (optional)Version
: Elastic version (optional, if none provided it will be fetched from cluster)Apply template
: Automatically apply index templateCheck Connection
: Button to test the configuration. It will displayed a modal with checked point, and if the case of it’s successfull, it will displayed the found version of the Elasticsearch and the index usedManually apply index template
: try to put the elasticsearch template by calling the api of elasticsearchShow index template
: try to retrieve the current index template presents in elasticsearchClient side temporal indexes handling
: When enabled, Otoroshi will manage the creation of indexes. When it’s disabled, Otoroshi will push in the same indexOne index per
: When the previous field is enabled, you can choose the interval of time between the creation of a new index in elasticsearchCustom TLS Settings
: Enable the TLS configuration for the communication with ElasticsearchTLS loose
: if enabled, will block all untrustful ssl configsTrustAll
: allows any server certificates even the self-signed onesClient certificates
: list of client certificates used to communicate with elasticsearchTrusted certificates
: list of trusted certificates received from elasticsearch
Webhook
With this kind of exporter, every matching event will be sent to a URL (in batch) using a POST method and an JSON array body.
Alerts hook URL
: url used to post eventsHook Headers
: headers add to the post requestCustom TLS Settings
: Enable the TLS configuration for the communication with ElasticsearchTLS loose
: if enabled, will block all untrustful ssl configsTrustAll
: allows any server certificates even the self-signed onesClient certificates
: list of client certificates used to communicate with elasticsearchTrusted certificates
: list of trusted certificates received from elasticsearch
Pulsar
With this kind of exporter, every matching event will be sent to an Apache Pulsar topic
Pulsar URI
: URI of the pulsar serverCustom TLS Settings
: Enable the TLS configuration for the communication with ElasticsearchTLS loose
: if enabled, will block all untrustful ssl configsTrustAll
: allows any server certificates even the self-signed onesClient certificates
: list of client certificates used to communicate with elasticsearchTrusted certificates
: list of trusted certificates received from elasticsearchPulsar tenant
: tenant on the pulsar serverPulsar namespace
: namespace on the pulsar serverPulsar topic
: topic on the pulsar server
Kafka
With this kind of exporter, every matching event will be sent to an Apache Kafka topic. You can find few tutorials about the connection between Otoroshi and Kafka based on docker images.
Kafka Servers
: the list of servers to contact to connect the Kafka client with the Kafka clusterKafka topic
: the topic on which Otoroshi alerts will be sent
By default, Kafka is installed with no authentication. Otoroshi supports the following authentication mechanisms and protocols for Kafka brokers.
SASL
The Simple Authentication and Security Layer (SASL) [RFC4422] is a method for adding authentication support to connection-based protocols.
SASL username
: the client usernameSASL password
: the client usernameSASL Mechanism
:PLAIN
: SASL/PLAIN uses a simple username and password for authentication.SCRAM-SHA-256
andSCRAM-SHA-512
: SASL/SCRAM uses usernames and passwords stored in ZooKeeper. Credentials are created during installation.
SSL
Kafka keypass
: the keystore password if you use a keystore/truststore to connect to Kafka clusterKafka keystore path
: the keystore path on the server if you use a keystore/truststore to connect to Kafka clusterKafka truststore path
: the truststore path on the server if you use a keystore/truststore to connect to Kafka clusterCustom TLS Settings
: enable the TLS configuration for the communication with ElasticsearchTLS loose
: if enabled, will block all untrustful ssl configsTrustAll
: allows any server certificates even the self-signed onesClient certificates
: list of client certificates used to communicate with elasticsearchTrusted certificates
: list of trusted certificates received from elasticsearch
SASL + SSL
This mechanism uses the SSL configuration and the SASL configuration.
Mailer
With this kind of exporter, every matching event will be sent in batch as an email (using one of the following email provider)
Otoroshi supports 5 exporters of email type.
Console
Nothing to add. The events will be write on the standard output.
Generic
Mailer url
: URL used to push eventsHeaders
: headers add to the push requestsEmail addresses
: recipients of the emails
Mailgun
EU
: is EU server ? if enabled, *https://api.eu.mailgun.net/* will be used, otherwise, the US URL will be used : *https://api.mailgun.net/*Mailgun api key
: API key of the mailgun accountMailgun domain
: domain name of the mailgun accountEmail addresses
: recipients of the emails
Mailjet
Public api key
: public key of the mailjet accountPrivate api key
: private key of the mailjet accountEmail addresses
: recipients of the emails
Sendgrid
Sendgrid api key
: api key of the sendgrid accountEmail addresses
: recipients of the emails
File
File path
: path where the logs will be writeMax file size
: when size is reached, Otoroshi will create a new file postfixed by the current timestamp
GoReplay file
With this kind of exporter, every matching event will be sent to a .gor
file compatible with GoReplay.
this exporter will only be able to catch TrafficCaptureEvent
. Those events are created when a route (or the global config) of the new proxy engine is setup to capture traffic using the capture
flag.
File path
: path where the logs will be writeMax file size
: when size is reached, Otoroshi will create a new file postfixed by the current timestampCapture requests
: capture http requests in the.gor
fileCapture responses
: capture http responses in the.gor
file
Console
Nothing to add. The events will be write on the standard output.
Custom
This type of exporter let you the possibility to write your own exporter with your own rules. To create an exporter, we need to navigate to the plugins page, and to create a new item of type exporter.
When it’s done, the exporter will be visible in this list.
Exporter config.
: the configuration of the custom exporter.
Metrics
This plugin is useful to rewrite the metric labels exposed on the /metrics
endpoint.
Labels
: list of metric labels. Each pair contains an existing field name and the new name.