---
title: "AlienVault OTX Pulse collector"
canonical: "https://docs.devo.com/space/latest/94655429/AlienVault%20OTX%20Pulse%20collector"
format: markdown
---
> Macro (toc)
## Overview
AlienVault OTX provides open access to a global community of threat researchers and security professionals. It delivers community-generated threat data, enables collaborative research, and automates the process of updating your security infrastructure with threat data from any source. OTX enables anyone in the security community to actively discuss, research, and share the latest threat data, trends, and techniques, strengthening your defenses while helping others do the same.
This document provides information about the AlienVault-OTX Pulse Collector, which facilitates automated interactions, with an AlienVault-OTX server to perform operations, such as retrieving details for an ***indicator***, and for a ***pulse***.
## AlienVault OTX Pulse
Pulses are the format for the OTX community to share information about threats. Pulses provide you with a summary of the threat, a view into the software targeted, and the related indicators of compromise (IOC) that can be used to detect the threats. A pulse consists of at least one, but more often multiple Indicators of Compromise (IoCs). An IoC is an artifact observed on a network or in an endpoint judged with a high degree of confidence to be a threat vector. Examples of threat vectors include campaigns or infrastructures used by an attacker.
[The table in this article](https://cybersecurity.att.com/documentation/usm-appliance/otx/about-otx.htm) provides a list of IoC types.
## Data source description
| **Data source** | **Description** | **API endpoint** | **Collector service** | **Devo table** |
| --- | --- | --- | --- | --- |
| Pulses | Threat intelligence subscriptions
- All pulses by users you are subscribed to
- All pulses you are directly subscribed to
- All pulses you have created yourself
- All pulses from groups you are a member of | `/api/v1/pulses/subscribed` | `alienvault_otx` | `threatintel.alienvault_otx.pulses.indicators` |
The data retrieved from AlientVault OTX is adapted to be stored in Devo. Each received *pulse* contains several hundred or thousands of *indicators*. Each indicator is stored individually in the Devo table `threatintel.alienvault_otx.indicators`, but combined with its pulse information.
An example of the information item stored in Devo, containing both indicator and pulse data combined, can be seen in **Example 1**:
```
{"eventdate":"2022-03-31T10:40:09.509+0200","hostname":"2020-hostname",
"id":"32321","indicator":"malware.indicator.web","type":"domain","created":"2022-03-31T10:02:54.000+0200","content":"","title":"","description":"",
"expiration":"","is_active":"1","role":"malware_hosting",
"pulse_id":"1ff3a9a","pulse_name":"New Wave Of Phishing Campaign","pulse_description":"","pulse_author_name":"AlienVault",
"pulse_modified":"2022-03-31T10:02:52.294+0200","pulse_created":"2022-03-31T10:02:52.294+0200",
"pulse_revision":"1","pulse_tlp":"white","pulse_public":"1","pulse_adversary":"",
"pulse_tags":"["phishing", "trojan"]","pulse_targeted_countries":"[]","pulse_malware_families":"["Family 1"]","pulse_attack_ids":"["T1XX1","TA0XX2"]",
"pulse_references":"[\"https://reference.web\"]","pulse_industries":"[\"Finance\"]","pulse_extract_source":"[]"}
```
**Example 2**
```
{"eventdate":"2022-03-31T10:39:23.475+0200","hostname":"2020-hostname",
"id":"550876","indicator":"2.3.4.5","type":"IPv4","created":"2022-03-12T12:18:02.000+0100","content":"","title":"Blaster UDP, Trojan from scan.example.org port 48909","description":"",
"expiration":"2022-04-11T13:00:00.000+0200","is_active":"1","role":"trojan",
"pulse_id":"606d75c1189a9430","pulse_name":"Example Honeypot","pulse_description":"Honeypot","pulse_author_name":"john",
"pulse_modified":"2022-03-31T10:39:02.654+0200","pulse_created":"2021-04-07T11:05:05.353+0200",
"pulse_revision":"1","pulse_tlp":"white","pulse_public":"1","pulse_adversary":"",
"pulse_tags":"["honeypot", "rdp", "ssh"]","pulse_targeted_countries":"["Italy"]","pulse_malware_families":"[]","pulse_attack_ids":"[]",
"pulse_references":"[]","pulse_industries":"[]","pulse_extract_source":"[]"}
```
## Vendor setup
In order to enable AlienVault OTX Pulse Devo Collector, you will need to follow the following steps:
1. Go to [AlienVault Open Threat Exchange](https://otx.alienvault.com/).
2. Click on the **Login **tab. If you don’t have an account you can sign up, and you will be able to access detailed documentation as well as your API key via the dashboard.
3. Type your login ID and password (provided by AlienVault) and log in.
4. After the console loads, click** API Integration** on the menu.
5. This loads the **DirectConnect API **page. This section of the panel is also where you’ll be able to confirm from the OTX side that your connection is functional.
6. Copy your OTX API key. This is the value that you need to update the `api_key` value in your collector configuration file.
## Run the collector
> Macro (rw-ui-tabs-macro)
>
> > Macro (rw-tab)
>
> We use a piece of software called Collector Server to host and manage all our available collectors.
>
> To enable the collector for a customer:
>
> 1. In the **Collector Server** **GUI**, access the **domain** in which you want this instance to be created
> 2. Click **Add Collector** and find the one you wish to add.
> 3. In the **Version** field, select the latest value.
> 4. In the **Collector Name** field, set the value you prefer (this name must be unique inside the same Collector Server domain).
> 5. In the sending method select **Direct Send. Direct Send** configuration is optional for collectors that create `Table` events, but mandatory for those that create `Lookups`.
> 6. In the **Parameters** section, establish the **Collector Parameters** as follows below:
>
> ### Editing the JSON configuration
>
> ```
> {
> "global_overrides": {
> "debug": false
> },
> "inputs": {
> "alienvault_otx": {
> "id": ,
> "enabled": true,
> "requests_per_second": ,
> "override_base_url": ,
> "credentials": {
> "api_key":
> },
> "services": {
> "alienvault_otx": {
> "request_period_in_seconds": ,
> "start_date":
> }
> }
> }
> }
> }
> ```
>
> > ℹ️ All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.
>
> Please replace the placeholders with real world values following the description table below:
>
> | | | | | |
> | --- | --- | --- | --- | --- |
> | **Parameter** | **Data type** | **Type** | **Value range / Format** | **Details** |
> | `short_unique_id` | `int` | `Mandatory` | Minimum length: 1
> Maximum length: 5 | Use this param to give an unique id to this input service. |
> | `requests_per_second` | `int` | `Optional` | Minimum value: 1 | Customize the maximum number of API requests per second. If not used, the default setting will be used: `100000` requests/sec
> 📝 This parameter can be left blank, removed or commented. |
> | `override_base_url` | str | `Optional` | By default, this RegEx:
`^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.])([\/\w \.-]*)*([a-z])$` | URL of the AlientVault OTX API, provided by AlienVault. Usually it is not necessary to set this parameter, the default value should be adequate.
> 📝 This parameter can be left blank, removed or commented. |
> | `creds_api_key` | `str` | `Mandatory` | Any | API Key downloaded from AlienVault tenant. |
> | `period_in_seconds` | `int` | `Optional` | Minimum length: 1 | By default, this service will run every `300` seconds. This parameter allows you to customize this behavior.
> 📝 This parameter can be left blank, removed or commented. |
> | `start``_date` | `str` | `Optional` | Format expected: `YYYY-MM-DD HH:mm:ss` | Parameter used by the collector as the starting `UTC` point when it first pulls data. An example value can be `2022-03-25 18:00:00`
> 📝 This parameter can be left blank, removed or commented.
> ⚠️ Updating this value will produce the lost of all persisted data. |
>
> 
>
> > Macro (rw-tab)
>
> This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.
>
> ### Structure
>
> The following directory structure should be created for being used when running the* *collector:
>
> ```
>
> └── devo-collectors/
> └── devo-collector-alienvault-otx/
> ├── certs/
> │ ├── chain.crt
> │ ├── .key
> │ └── .crt
> └── config/
> └── config.yaml
> ```
>
> ### Devo credentials
>
> In Devo, go to **Administration → Credentials → X.509 Certificates**, download the **Certificate**, **Private key** and **Chain CA** and save them in `devo-collector-alienvault-otx/certs`. Learn more about security credentials in Devo [here](https://devodocs.atlassian.net/wiki/pages/createpage.action?spaceKey=NDT5&title=Security%20credentials).
>
> 
>
> ### Editing the config.yaml file
>
> ```
> globals:
> debug: false
> id: not_used
> name: alienvault_collector
> persistence:
> type: filesystem
> config:
> directory_name: state
>
> outputs:
> devo_1:
> type: devo_platform
> config:
> address:
> #address: collector-us.devo.io
> #address: collector-eu.devo.io
> port: 443
> type: SSL
> chain:
> cert: .crt
> key: .key
> inputs:
> alienvault_otx:
> id:
> enabled: true
> requests_per_second:
> override_base_url:
> credentials:
> api_key:
> services:
> alienvault_otx:
> request_period_in_seconds:
> start_date:
> ```
>
> Replace the placeholders with the required values:
>
> | **Parameter** | **Data type** | **Type** | **Value range** | **Details** |
> | --- | --- | --- | --- | --- |
> | `devo_address` | `str` | `Mandatory` | - [collector-us.devo.io](http://collector-us.devo.io)
> - [collector-eu.devo.io](http://collector-eu.devo.io) | Use this param to identify the Devo Cloud where the events will be sent. |
> | `chain_filename` | `str` | `Mandatory` | Minimum length: 4
> Maximum length: 20 | Use this param to identify the chain.cert file downloaded from your Devo domain. Usually this file's name is `chain.crt` |
> | `cert_filename` | `str` | `Mandatory` | Minimum length: 4
> Maximum length: 20 | Use this param to identify the `file.cert` downloaded from your Devo domain. |
> | `key_filename` | `str` | `Mandatory` | Minimum length: 4
> Maximum length: 20 | Use this param to identify the `file.key` downloaded from your Devo domain. |
> | `shor_unique_id` | `int` | `Mandatory` | Minimum length: 1
> Maximum length: 5 | Use this param to give a unique ID to this input service. |
> | `requests_per_second` | `int` | `Optional` | Minimum value: 1 | Customize the maximum number of API requests per second. If not used, the default setting will be used: 100000 requests/sec
This parameter can be left blank, removed or commented. |
> | `override_base_url` | `str` | `Optional` | By default, this RegEx:
`^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.])([\/\w \.-]*)*([a-z])$` | URL of the AlientVault OTX API, provided by AlienVault. Usually, it is not necessary to set this parameter, the default value should be adequate.
This parameter can be left blank, removed, or commented. |
> | `creds_api_key` | `str` | `Mandatory` | Any | API Key downloaded from AlienVault tenant. |
> | `period_in_seconds` | `int` | `Optional` | Minimum length: 1 | By default, this service will run every 300 seconds. This parameter allows you to customize this behavior.
This parameter can be left blank, removed or commented. |
> | `start_date` | `str` | `Optional` | Format expected: `YYYY-MM-DD HH:mm:ss` | Parameter used by the collector as the starting UTC point when it first pulls data. An example value can be `2022-03-25 18:00:00`
This parameter can be left blank, removed, or commented.
Updating this value will produce the loss of all persisted data. |
>
> ### Download the Docker image
>
> The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:
>
> | **Collector Docker image** | **SHA-256 hash** |
> | --- | --- |
> | [collector-alienvault-otx-docker-image-1.1.0.tgz](https://drive.google.com/file/d/1T0MxtrwuYsSEjnS91_Mwh3HAhTXb4ubk/view?usp=drive_link) | `8e2fe5117df3a65b62e1b8205c90f31c2cca5eeacdff3ff5c278d47da56660e0` |
>
> Use the following command to add the Docker image to the system:
>
> ```
> gunzip -c collector-alienvault-otx-docker-image-.tgz | docker load
> ```
>
> > ℹ️ Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace "``" with a proper value.
>
> The Docker image can be deployed on the following services:
>
> #### Docker
>
> Execute the following command on the root directory `/devo-collectors/devo-collector-alienvault-otx/`
>
> ```
> docker run \
> --name collector-alienvault-otx\
> --volume $PWD/certs:/devo-collector/certs \
> --volume $PWD/config:/devo-collector/config \
> --volume $PWD/state:/devo-collector/state \
> --env CONFIG_FILE=config.yaml \
> --rm -it devo.com/collectors/alientvault_otx_collector_if:
> ```
>
> > ⚠️ Replace `` with a proper value.
>
> #### Docker Compose
>
> The following Docker Compose file can be used to execute the Docker container. It must be created in the `/devo-collectors/devo-collector-alienvault-otx/` directory.
>
> ```
> version: '3'
> services:
> collector-alienvault-otx:
> image: devo.com/collectors/alientvault_otx_collector_if:${IMAGE_VERSION:-latest}
> volumes:
> - ./certs:/devo-collector/certs
> - ./config:/devo-collector/config
> - ./state:/devo-collector/state
> environment:
> - CONFIG_FILE=${CONFIG_FILE:-config.yaml}
> ```
>
> To run the container using docker-compose, execute the following command from the `/devo-collectors/devo-collector-alienvault-otx/` directory:
>
> ```
> IMAGE_VERSION= docker-compose up -d
> ```
>
> > ⚠️ Replace `` with a proper value.
## Change log
| **Release** | **Released on** | **Release t**
**ype** | **Details** | **Recommendations** |
| --- | --- | --- | --- | --- |
| `v1.1.0` | 8/25/2023 | [IMPROVEMENT] | **Improvements**
- **Upgraded DCSDK from 1.1.4 to 1.9.1**
- Store lookup instances into DevoSender to avoid creation of new instances for the same lookup
- Ensure service_config is a dict into templates
- Ensure special characters are properly sent to the platform
- Changed log level to some messages from info to debug
- Changed some wrong log messages
- Upgraded some internal dependencies
- Changed queue passed to setup instance constructor
- Added log traces for knowing the execution environment status (debug mode)
- Fixes in the current puller template version
- Improved log trace details when runtime exceptions happen
- Refactored source code structure
- New “templates” functionality
- Functionality for detecting some system signals for starting the controlled stopping
- Input objects sends again the internal messages to devo.collectors.out table
- Upgraded DevoSDK to version 3.6.4 to fix a bug related to a connection loss with Devo
- Refactored source code structure
- Changed way of executing the controlled stopping
- Minimized probabilities of suffering a DevoSDK bug related to “sender” to be null
- Ability to validate collector setup and exit without pulling any data
- Ability to store in the persistence the messages that couldn’t be sent after the collector stopped
- Ability to send messages from the persistence when the collector starts and before the puller begins working
- Ensure special characters are properly sent to the platform
- Added a lock to enhance sender object
- Added new class attrs to the `setstate` and `getstate` queue methods
- Fix sending attribute value to the `setstate` and `getstate` queue methods
- Added log traces when queues are full and have to wait
- Added log traces of queues time waiting every minute in debug mode
- Added method to calculate queue size in bytes
- Block incoming events in queues when there are no space left
- Send telemetry events to Devo platform
- Upgraded internal Python dependency Redis to v4.5.4
- Upgraded internal Python dependency DevoSDK to v5.1.3
- Fixed obfuscation not working when messages are sent from templates
- New method to figure out if a puller thread is stopping
- Upgraded internal Python dependency DevoSDK to v5.0.6
- Improved logging on messages/bytes sent to Devo platform
- Fixed wrong bytes size calculation for queues
- New functionality to count bytes sent to Devo Platform (shown in console log)
- Upgraded internal Python dependency DevoSDK to v5.0.4
- Fixed bug in persistence management process, related to persistence reset
- Aligned source code typing to be aligned with Python 3.9.x
- Inject environment property from user config
- Obfuscation service can be now configured from user config and module definitonStore lookup instances into DevoSender to avoid creation of new instances for the same lookup
- Ensure service_config is a dict into templates
- Ensure special characters are properly sent to the platform
- Changed log level to some messages from info to debug
- Changed some wrong log messages
- Upgraded some internal dependencies
- Changed queue passed to setup instance constructor
- Added log traces for knowing the execution environment status (debug mode)
- Fixes in the current puller template version
- Improved log trace details when runtime exceptions happen
- Refactored source code structure
- New “templates” functionality
- Functionality for detecting some system signals for starting the controlled stopping
- Input objects sends again the internal messages to devo.collectors.out table
- Upgraded DevoSDK to version 3.6.4 to fix a bug related to a connection loss with Devo
- Refactored source code structure
- Changed way of executing the controlled stopping
- Minimized probabilities of suffering a DevoSDK bug related to “sender” to be null
- Ability to validate collector setup and exit without pulling any data
- Ability to store in the persistence the messages that couldn’t be sent after the collector stopped
- Ability to send messages from the persistence when the collector starts and before the puller begins working
- Ensure special characters are properly sent to the platform
- Added a lock to enhance sender object
- Added new class attrs to the `setstate` and `getstate` queue methods
- Fix sending attribute value to the `setstate` and `getstate` queue methods
- Added log traces when queues are full and have to wait
- Added log traces of queues time waiting every minute in debug mode
- Added method to calculate queue size in bytes
- Block incoming events in queues when there are no space left
- Send telemetry events to Devo platform
- Upgraded internal Python dependency Redis to v4.5.4
- Upgraded internal Python dependency DevoSDK to v5.1.3
- Fixed obfuscation not working when messages are sent from templates
- New method to figure out if a puller thread is stopping
- Upgraded internal Python dependency DevoSDK to v5.0.6
- Improved logging on messages/bytes sent to Devo platform
- Fixed wrong bytes size calculation for queues
- New functionality to count bytes sent to Devo Platform (shown in console log)
- Upgraded internal Python dependency DevoSDK to v5.0.4
- Fixed bug in persistence management process, related to persistence reset
- Aligned source code typing to be aligned with Python 3.9.x
- Inject environment property from user config
- Obfuscation service can be now configured from user config and module definiton | `Recommended version` |