Flow / Unit types / Control / Batch Detector
Batch Detector
Description
This unit is a Processor unit.
A unit that signals the start and end of a batch of events, where a batch is considered to be a sequence of events that share a common value in a given field. This field is identified as the Input batch field value in the unit properties.
There are two types of input events for this unit:
- Normal: input events that can produce both data and signal events on output.
- Signal: input events that can only produce signals.
Both of them should contain the field configured in the Input batch field option of the unit's settings.
Normal events enter through the in port. If they belong to the same batch as the previous event, they will be forwarded through the data port. Otherwise, signals will be generated on start/end ports, and then the input event will be forwarded through the data port.
Configuration
After dragging this unit into the Flow canvas, double-click it to access its configuration options. The following table describes the configuration options of this unit:
| Tab | Field | Description |
|---|---|---|
| General | Name | Enter a name for the unit. It must start with a letter, and cannot contain spaces. Only letters, numbers, and underscores are allowed. |
| Input batch field | This is where you configure which field will be used as the batch value, that is to say, the field that contains the value that all the events in a batch share. | |
| Output batch field | Name of the output field that will contain the relevant batch value. For start events, the batch that is starting; for end events, the batch that is ending; for data events, the current batch. |
Input ports
| Port | Description |
|---|---|
| in | "Normal" input events (will generate output signals if needed and then be forwarded through the data port). |
| signal | "Signal" input events (will generate output signals if needed, but won't be forwarded through the data port). |
Output ports
| Port | Description |
|---|---|
| start | Signals the start of a new batch. The output events are input events, enriched with batchIdField, containing the starting batch value. |
| data | Input events that are part of the current batch. The output events are input events, enriched with batchIdField, containing the starting batch value. |
| end | Signals when a batch ends. Events are clones of input events, enriched with batchIdField, containing the ending batch value. |
| error | Signals when an error occurred. Events are input events enriched with standard error fields. |
Example
Imagine you have a Devo domain with users from two different countries and need to know each time users from one country or the other one are working in the domain.
You can use the Batch Detector unit to get notified each time the value in the country column of the siem.logtrust.web.activity table changes. To do it, add a Devo Source unit to indicate the source table that will send the events. Then, connect it to the Batch Detector unit through its in input port. Finally, link the start output port of the unit to an Email Sink unit to get notified each time a new batch of events starts.
In the Batch Detector unit properties, you must select the country column of the table, whose values will be used to define the different batches.
You can try this flow by downloading this JSON file and uploading it to your domain using the Import option.