---
title: "Tenable.io collector"
canonical: "https://docs.devo.com/space/latest/94657360/Tenable.io%20collector"
format: markdown
---
> Macro (toc)

## Overview

Managed in the cloud and powered by Nessus technology, [http://Tenable.io](http://Tenable.io)  provides the comprehensive vulnerability coverage with the ability to predict which security issues to remediate first. It’s a complete end-to-end vulnerability management solution. Get a risk-based view of your entire attack surface—from IT to cloud to containers—so you can quickly identify, investigate and prioritize vulnerabilities.

## 1.x to 2.x migrating guide

If you are migrating from v1.x to v2.x, you can find a complete guide [in this article](https://devodocs.atlassian.net/wiki/spaces/latest/pages/1000013837).

## Devo collector features

| **Feature** | **Details** |
| --- | --- |
| Allow parallel downloading (`multipod`) | - `not allowed` |
| Running environments | - `collector server`
- `on-premise` |
| Populated Devo events | - `table` |
| Flattening preprocessing | - `no` |

## Data sources

This collector extracts data from multiple sources within [http://Tenable.io](http://Tenable.io) , enabling comprehensive vulnerability and asset management. Below is a detailed description of the data sources, their purpose, endpoints, collector service names, associated Devo tables, and the release version when they became available.

| **Data source** | **Description** | **API endpoint** | **Collector service name** | **Devo table** | **Available from release** |
| --- | --- | --- | --- | --- | --- |
| Agents | Nessus Agents collect vulnerability, compliance, and system data from hosts and report it back to [http://Tenable.io](http://Tenable.io) . | `PyTenable - Agents` | `agents` | `vuln.tenable.io.agents` | `v1.0.0` |
| Assets | Represents network entities such as servers, desktops, mobile devices, and cloud instances. Helps identify and mitigate potential security risks. | `PyTenable - Assets` | `export_assets` | `vuln.tenable.io.assets` | `v1.2.0` |
| Audit Log | Provides visibility into actions taken in the [http://Tenable.io](http://Tenable.io)  account, including user actions, timestamps, and affected entities. | `PyTenable - Audit Logs` | `audit_log` | `vuln.tenable.io.audit_log` | `v1.0.0` |
| Plugins | Programs written in NASL (Nessus Attack Scripting Language) for detecting vulnerabilities. Includes information about vulnerabilities and their remediation. | `PyTenable - Plugins` | `plugins` | `vuln.tenable.io.plugins` | `v1.0.0` |
| Scanners | Configuration and linking of Nessus, NNM, and cloud scanners for use in [http://Tenable.io](http://Tenable.io) . | `PyTenable - Scanners` | `scanners` | `vuln.tenable.io.scanners` | `v1.0.0` |
| Scans | Retrieves information about vulnerability scans conducted on the network. | `PyTenable - Scans` | `scans` | `vuln.tenable.io.scans             ` | `v1.0.0` |
| Vulnerabilities | Centralized data for vulnerabilities detected by Tenable and third-party scans. | `PyTenable - Vulnerabilities` | `export_vulnerabilities` | `vuln.tenable.io.vulnerabilities` | `v1.0.0` |
| WAS Vulnerabilities | Returns vulnerabilities detected by Tenable Web App Scanning API v2. | `/vulnerabilities/search` | `was_vulnerabilities` | `vuln.tenable.io.was_vulnerability ` | `v2.0.0` |

For more information on how the events are parsed, [visit our page](https://devodocs.atlassian.net/wiki/spaces/latest/pages/94666077).

## Flattening preprocessing

This collector does not implement flattening

## Vendor setup

To set up access to the [http://Tenable.io](http://Tenable.io)  API and configure the collector, follow the steps below.

### Obtaining API Keys

#### Verify Account Permissions

Ensure you have a valid user account with appropriate permissions by logging into [http://Tenable.io](http://Tenable.io) .

#### Generate API Keys

1. Generate the `access_key` and `secret_key` for your account. For more information, refer to **Generate API Keys** in the [http://Tenable.io](http://Tenable.io)  Vulnerability Management User Guide.
2. [http://Tenable.io](http://Tenable.io)  generates a unique set of API keys for each user account.
  - These keys allow your application to authenticate without creating a session.

#### Add API Keys to Requests

Include the `X-ApiKeys` header element in your HTTP request messages. Example:

```http
X-ApiKeys: accessKey=<your_access_key>; secretKey=<your_secret_key>
```

### Authorization Permissions

#### Basic Permissions

Users with the **Basic** role (Value: 16) can safely view and configure scan results.

#### Administrator Permissions

To request **Audit_log** data, you must have the **Administrator** role (Value: 64).

### Security Recommendations

- Treat your `access_key` and `secret_key` like passwords.
- Do not share your keys with unauthorized individuals.
- Store the keys securely in an environment variable or a secrets manager.
- Rotate your API keys periodically to enhance security.

## Minimum configuration required for basic pulling

Although this collector supports advanced configuration, the fields required to retrieve data with basic configuration are defined below.

> ℹ️ This minimum configuration refers exclusively to those specific parameters of this integration. There are more required parameters related to the generic behavior of the collector. Check setting sections for details.

| **Setting** | **Details** |
| --- | --- |
| `access_key` | The access key is required to authenticate API requests. |
| `secret_key` | The secret key is required to securely sign API requests. |

> ℹ️ See the **Accepted authentication methods** section to verify what settings are required based on the desired authentication method.

## Accepted authentication methods

The collector supports the following authentication method to connect to [http://Tenable.io](http://Tenable.io) :

| **Authentication Method** | **Details** |
| --- | --- |
| `access_key/secret_key` | The `access_key` serves as the user identifier, while the `secret_key` is used to securely sign API requests. |

## Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (<u>Cloud collector</u>), or deploy and host the collector in your own machine using a Docker image (<u>On-premise collector</u>).

> Macro (rw-ui-tabs-macro)
> 
> > Macro (rw-tab)
> 
> We use a piece of software called Collector Server to host and manage all our available collectors. If you want us to host this collector for you, [get in touch with us](https://devo.my.site.com/support/login?ec=302&startURL=%2Fsupport%2Fs%2F) and we will guide you through the configuration.
> 
> To enable the collector for a customer:
> 
> 1. In the **Collector Server** **GUI**, access the **domain** in which you want this instance to be created
> 2. Click **Add Collector** and find the one you wish to add.
> 3. In the **Version** field, select the latest value.
> 4. In the **Collector Name** field, set the value you prefer (this name must be unique inside the same Collector Server domain).
> 5. In the sending method select **Direct Send. Direct Send** configuration is optional for collectors that create `Table` events, but mandatory for those that create `Lookups`.
> 6. In the **Parameters** section, establish the **Collector Parameters** as follows below:
> 
> ### Editing the JSON configuration
> 
> ```
> {
>   "global_overrides": {
>     "debug": false
>   },
>   "inputs": {
>     "tenable_io": {
>       "id": "<short_unique_identifier>",
>       "enabled": true,
>       "credentials": {
>         "access_key": "<access_key_value>",
>         "secret_key": "<secret_key_value>"
>       },
>       "environment": "<dev_or_prod>",
>       "services": {
>         "agents": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "export_assets": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "audit_log": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "start_date_utc": "<start_date_utc_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "plugins": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "start_date_utc": "<start_date_utc_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "scanners": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "scans": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "override_tag": "<override_tag_value>"
>         },
>         "export_vulnerabilities": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "override_tag": "<override_tag_value>",
>           "severities": ["<severities_value>"],
>           "statuses": ["<statuses_value>"],
>           "since_epoch": "<since_epoch_value>",
>           "num_processing_threads": "<num_processing_threads_value>"
>         },
>         "was_vulnerabilities": {
>           "request_period_in_seconds": "<request_period_in_seconds_value>",
>           "start_date_utc": "<start_date_utc_value>",
>           "override_tag": "<override_tag_value>"
>         }
>       }
>     }
>   }
> }
> ```
> 
> The following table outlines the parameters available for configuring the collector. Each parameter is categorized by its necessity (mandatory or optional), data type, acceptable values or formats, and a brief description.
> 
> | **Parameter** | **Data Type** | **Requirement** | **Value Range / Format** | **Description** |
> | --- | --- | --- | --- | --- |
> | `id` | `string` | Mandatory | Min length: `1` | Unique identifier for the collector instance. |
> | `enabled` | `boolean` | Mandatory | `true, false ` | Indicates whether the collector is active. |
> | `access_key` | `string` | Mandatory | Min length: `1` | API access key for [http://Tenable.io](http://Tenable.io)  authentication. |
> | `secret_key` | `string` | Mandatory | Min length: `1` | Secret key used for signing API requests. |
> | `environment` | `string` | Optional | `dev, prod` | Specifies the environment. Use `dev` for development or `prod` for production. |
> | `request_period_in_seconds` | `integer` | Optional | `1` to `604800` | Frequency of requests, in seconds. |
> | `override_tag` | `string` | Optional | Example: `my.app.test.1 ` | Custom override tag for event categorization. |
> | `start_date_utc` | `string` | Mandatory (for audit_log, plugins, and was_vulnerabilities) | `"YYYY-MM-DD"` (for audit_log and plugins) or `"YYYY/MM/DD`" (for was_vulnerabilities) | Start date for pulling events. |
> | `severities` | `array` | Optional | `["critical", "high", "medium"] ` | List of severities to filter vulnerabilities. Default: All severities. |
> | `statuses` | `array` | Optional | `["open", "reopened"] ` | List of statuses to filter vulnerabilities. Default: All statuses. |
> | `since_epoch` | `integer` | Optional | Unix timestamp | Fetch data after a specific point. |
> | `num_processing_threads` | `integer` | Optional | Positive integer (default: `1`) | Number of threads for processing vulnerabilities. |
> 
> **Example**
> 
> ```
> {
>   "global_overrides": {
>     "debug": false
>   },
>   "inputs": {
>     "tenable_io": {
>       "id": "example-collector-id",
>       "enabled": true,
>       "credentials": {
>         "access_key": "example_access_key_12345",
>         "secret_key": "example_secret_key_ABCDE"
>       },
>       "environment": "prod",
>       "services": {
>         "agents": {
>           "request_period_in_seconds": 86400
>         },
>         "export_assets": {
>           "request_period_in_seconds": 86400
>         },
>         "audit_log": {
>           "request_period_in_seconds": 60,
>           "start_date_utc": "2024-11-10"
>         },
>         "plugins": {
>           "request_period_in_seconds": 86400,
>           "start_date_utc": "2024-11-10"
>         },
>         "scanners": {
>           "request_period_in_seconds": 43200
>         },
>         "scans": {
>           "request_period_in_seconds": 43200
>         },
>         "export_vulnerabilities": {
>           "request_period_in_seconds": 43200
>         },
>         "was_vulnerabilities": {
>           "request_period_in_seconds": 43200,
>           "start_date_utc": "2024/11/10"
>         }
>       }
>     }
>   }
> }
> ```
> 
> > Macro (rw-tab)
> 
> This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.
> 
> ### Structure
> 
> The following directory structure should be created for being used when running the collector:
> 
> ```
> <any_directory>
> └── devo-collectors/
>     └── <product_name>/
>         ├── certs/
>         │   ├── chain.crt
>         │   ├── <your_domain>.key
>         │   └── <your_domain>.crt
>         ├── state/
>         └── config/ 
>             └── config.yaml 
> ```
> 
> > ⚠️ Replace `<product_name>` with the proper value.
> 
> ### Devo credentials
> 
> In Devo, go to **Administration → Credentials → X.509 Certificates**, download the **Certificate**, **Private key** and **Chain CA** and save them in `<product_name>/certs/`. Learn more about security credentials in Devo [here](#).
> 
> ![image](media://f48d6294-a613-4bba-8f22-f5ad772ea95b)
> 
> > ⚠️ Replace `<product_name>` with the proper value.
> 
> ### Editing the config.yaml file
> 
> ```
> globals:
>   debug: false
>   id: not used
>   name: tenable_io
>   persistence:
>     type: filesystem
>     config:
>       directory_name: state
> outputs:
>   devo_1:
>     type: devo_platform
>     config:
>       address: <devo_address>
>       port: 443
>       type: SSL
>       chain: <chain_filename>
>       cert: <cert_filename>
>       key: <key_filename>
> inputs:
>   tenable_io:
>     id: <short_unique_identifier>
>     enabled: true
>     credentials:
>       access_key: <access_key_value>
>       secret_key: <secret_key_value>
>     services:
>       agents:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         override_tag: <override_tag_value>
>       export_assets:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         override_tag: <override_tag_value>
>       audit_log:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         start_date_utc: <start_date_utc_value>
>         override_tag: <override_tag_value>
>       plugins:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         start_date_utc: <start_date_utc_value>
>         override_tag: <override_tag_value>
>       scanners:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         override_tag: <override_tag_value>
>       scans:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         override_tag: <override_tag_value>
>       export_vulnerabilities:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         override_tag: <override_tag_value>
>         severities:
>           - <severity_value>
>         statuses:
>           - <status_value>
>         since_epoch: <since_epoch_value>
>         num_processing_threads: <num_processing_threads_value>
>       was_vulnerabilities:
>         request_period_in_seconds: <request_period_in_seconds_value>
>         start_date_utc: <start_date_utc_value>
>         override_tag: <override_tag_value>
> ```
> 
> > ℹ️ All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.
> 
> Replace the placeholders with your required values following the description table below:
> 
> | **Parameter** | **Data Type** | **Requirement** | **Value Range / Format** | **Description** |
> | --- | --- | --- | --- | --- |
> | `id` | `string` | Mandatory | Min length: `1` | Unique identifier for the collector instance. |
> | `enabled` | `boolean` | Mandatory | `true, false` | Indicates whether the collector is active. |
> | `access_key` | `string` | Mandatory | Min length: `1` | API access key for [http://Tenable.io](http://Tenable.io)  authentication. |
> | `secret_key` | `string` | Mandatory | Min length: `1` | Secret key used for signing API requests. |
> | `environment` | `string` | Optional | `dev, prod` | Specifies the environment. Use `dev` for development or `prod` for production. |
> | `address` | `string` | Mandatory | Example: [logs.example.devo.com](http://logs.example.devo.com) | The Devo address where logs will be sent. |
> | `port` | `integer` | Mandatory | `443` | Port for secure communication. |
> | `type` | `string` | Mandatory | `SSL` | Protocol used for communication. |
> | `chain_filename` | `string` | Mandatory | Min length: `4` | The filename of the chain certificate downloaded from Devo. Example: chain.crt. |
> | `cert_filename` | `string` | Mandatory | Min length: `4` | The filename of your certificate downloaded from Devo. Example: your_domain.crt. |
> | `key_filename` | `string` | Mandatory | Min length: `4` | The filename of your private key downloaded from Devo. Example: your_domain.key. |
> | `request_period_in_seconds` | `integer` | Optional | `1` to `604800` | Frequency of requests, in seconds. |
> | `override_tag` | `string` | Optional | Example: `my.app.test.1` | Custom override tag for event categorization. |
> | `start_date_utc` | `string` | Mandatory (for audit_log, plugins, and was_vulnerabilities) | `"YYYY-MM-DD"` (for audit_log and plugins) or `"YYYY/MM/DD"` (for was_vulnerabilities) | Start date for pulling events. |
> | `severities` | `array` | Optional | `["critical", "high", "medium"]` | List of severities to filter vulnerabilities. Default: All severities. |
> | `statuses` | `array` | Optional | `["open", "reopened"]` | List of statuses to filter vulnerabilities. Default: All statuses. |
> | `since_epoch` | `integer` | Optional | Unix timestamp | Fetch data after a specific point. |
> | `num_processing_threads` | `integer` | Optional | Positive integer (default: `1`) | Number of threads for processing vulnerabilities. |
> 
> **Example**
> 
> ```yaml
> globals:
>   debug: false
>   id: not used
>   name: tenable_io
>   persistence:
>     type: filesystem
>     config:
>       directory_name: state
> outputs:
>   devo_1:
>     type: devo_platform
>     config:
>       address: logs.example.devo.com
>       port: 443
>       type: SSL
>       chain: chain.crt
>       cert: your_domain.crt
>       key: your_domain.key
> inputs:
>   tenable_io:
>     id: example-collector-id
>     enabled: true
>     credentials:
>       access_key: example_access_key_12345
>       secret_key: example_secret_key_ABCDE
>     services:
>       agents: {}
>       export_assets: {}
>       audit_log:
>         start_date_utc: 2024-11-10
>       plugins:
>         start_date_utc: 2024-11-10
>       scanners: {}
>       scans: {}
>       export_vulnerabilities: {}
>       was_vulnerabilities:
>         start_date_utc: 2024/11/10
> ```
> 
> ### Download the Docker image
> 
> The collector should be deployed as a Docker container. Download the Docker image of the collector as a .tgz file by clicking the link in the following table:
> 
> | **Collector Docker image** | **SHA-256 hash** |
> | --- | --- |
> | [collector-tenable_io_if-docker-image-2.2.0.tgz](https://drive.google.com/file/d/1YJIlhhG8ydmCVpgHPGeIcl-OeXaQV_gT/view?usp=drive_link) | `21bb619f19df67f7ddee989bed5805718582658858b98e8f1c58da009c2ad77e` |
> 
> Use the following command to add the Docker image to the system:
> 
> ```
> gunzip -c <image_file>-<version>.tgz | docker load
> ```
> 
> > ⚠️ Once the Docker image is imported, it will show the real name of the Docker image (including version info). Replace `<image_file>` and `<version>` with a proper value.
> 
> The Docker image can be deployed on the following services:
> 
> #### Docker
> 
> Execute the following command on the root directory `<any_directory>/devo-collectors/<product_name>/`
> 
> ```
> docker run 
> --name collector-<product_name> 
> --volume $PWD/certs:/devo-collector/certs 
> --volume $PWD/config:/devo-collector/config 
> --volume $PWD/state:/devo-collector/state 
> --env CONFIG_FILE=config.yaml 
> --rm 
> --interactive 
> --tty 
> <image_name>:<version>
> ```
> 
> > ⚠️ Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.
> 
> #### Docker Compose
> 
> The following Docker Compose file can be used to execute the Docker container. It must be created in the `<any_directory>/devo-collectors/<product_name>/` directory.
> 
> ```
> version: '3'
> services:
>   collector-<product_name>:
>     image: <image_name>:${IMAGE_VERSION:-latest}
>     container_name: collector-<product_name>
>     volumes:
>       - ./certs:/devo-collector/certs
>       - ./config:/devo-collector/config
>       - ./credentials:/devo-collector/credentials
>       - ./state:/devo-collector/state
>     environment:
>       - CONFIG_FILE=${CONFIG_FILE:-config.yaml}
> ```
> 
> To run the container using docker-compose, execute the following command from the `<any_directory>/devo-collectors/<product_name>/` directory:
> 
> ```
> IMAGE_VERSION=<version> docker-compose up -d
> ```
> 
> > ⚠️ Replace `<product_name>`, `<image_name>` and `<version>` with the proper values.

## Collector services detail

This section is intended to explain how to proceed with specific actions for services.

### Common services detail

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Setup output

A successful run has the following output messages for the setup module:

```
INFO InputProcess::MainThread -> TenableIoPullerSetup(tenable_io#13245,scanners#predefined) -> Starting thread
WARNING InputProcess::TenableIoPullerSetup(tenable_io#13245,scanners#predefined) -> The token/header/authentication has not been created yet
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Waiting until setup will be executed
INFO InputProcess::TenableIoPullerSetup(tenable_io#13245,scanners#predefined) -> Setup for module <TenableIoStatelessPuller> has been successfully executed
```
</details>

<details>
<summary>Troubleshooting</summary>

This collector has different security layers that detect both an invalid configuration and abnormal operation. This table will help you detect and resolve the most common errors.

| **Error Type** | **Error ID** | **Error Message** | **Cause** | **Solution** |
| --- | --- | --- | --- | --- |
| InitVariablesError | 1 | `The init_start_time: {start_time_in_utc_str} is not in the correct format: {datetime_forma} ` | Incorrect format for `start_time_in_utc`. | Ensure the start_time_in_utc is in the correct format (`e.g., YYYY-MM-DD`). |
| 2 | `Invalid start_time_in_utc: {start_time_in_utc_str}. Must be a past date.      ` | The provided `start_time_in_utc` is not a valid past date. | Verify that start_time_in_utc is set to a valid past date. |
| PullError | 301 | `Response Content (not JSON) ` | The response from the server is not in the expected JSON format. | Check the response content and ensure it is properly formatted as JSON. |
| 302 | `Error: {response.status_code} - {response.text}` | The API response was an error. | Review the status code and message for more details on the failure. |
| SetupError | 101 | `Unexpected error occurred: {e}` | A general setup error occurred during initialization. | Inspect the setup process and correct any misconfigurations or issues. |
| ApiError | 400 | `Bad Request (400): The request was invalid.` | The API request was malformed or missing required parameters. | Verify the request parameters and ensure they match the API specifications. |
| 401 | `Unauthorized (401): Access key or secret key is incorrect.` | The access key or secret key is invalid. | Check that the correct access_key and secret_key are being used. |
| 403 | `Forbidden (403): Access is forbidden, check permissions.` | Insufficient permissions for the requested operation. | Ensure that the API user has the necessary permissions. |
| 404 | `Not Found (404): The requested resource does not exist.` | The resource requested could not be found. | Verify the resource ID or endpoint is correct. |
| 429 | `Too Many Requests (429): Rate limit exceeded.` | Too many requests were sent in a short period of time. | Review the rate limit and implement retry logic as needed. |
| 500 | `Server Error (500): An internal server error occurred.` | The server encountered an internal error while processing. | Retry the request or contact support if the issue persists. |
| 410 | `Unhandled HTTP Error ({status_code}): {error}` | An unhandled error occurred with a status code other than `400`-`500`. | Investigate the error code and consult the API documentation for details. |
| PullError | 313 | `Error during _request_job_creation. {e.cause}` | An error occurred while creating the job. | Investigate the error in job creation and check the API documentation. |
| 310 | `{export_type} job {job_uuid}: Error on Tenable API during job creation.` | A failure occurred during the job creation process. | Check the job creation API request for issues. |
| 311 | `{export_type} job {job_uuid}: Job creation was cancelled.` | The job creation was manually or automatically cancelled. | Verify the job status and resolve any issues that caused cancellation. |
</details>

### Scanners service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Reading persisted data
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Data retrieved from the persistence: None
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Running the persistence corrections steps
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> The persistence version value is <ZERO>, so no persistence will be allocated
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Pull Started
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733390827964):Number of requests made: 2; Number of events received: 31; Number of duplicated events filtered out: 0; Number of events generated and sent: 31; Average of events per second: 48.412.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733390827964):Number of requests made: 2; Number of events received: 31; Number of duplicated events filtered out: 0; Number of events generated and sent: 31; Average of events per second: 48.276.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> The data is up to date!
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Data collection completed. Elapsed time: 0.656 seconds. Waiting for 43199.344 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scanners#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733390827964):Number of requests made: 2; Number of events received: 31; Number of duplicated events filtered out: 0; Number of events generated and sent: 31; Average of events per second: 48.276.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

### Scans service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> TenableIoStatelessPuller(tenable_io#13245,scans#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Reading persisted data
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Data retrieved from the persistence: None
NING InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Running the persistence corrections steps
NING InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> The persistence version value is <ZERO>, so no persistence will be allocated
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> TenableIoStatelessPuller(tenable_io#13245,scans#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Pull Started
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733393050178):Number of requests made: 2; Number of events received: 319; Number of duplicated events filtered out: 0; Number of events generated and sent: 319; Average of events per second: 347.746.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393050178):Number of requests made: 2; Number of events received: 319; Number of duplicated events filtered out: 0; Number of events generated and sent: 319; Average of events per second: 347.411.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> The data is up to date!
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Data collection completed. Elapsed time: 0.933 seconds. Waiting for 43199.067 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,scans#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393050178):Number of requests made: 2; Number of events received: 319; Number of duplicated events filtered out: 0; Number of events generated and sent: 319; Average of events per second: 347.411.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

### Agents service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> TenableIoStatelessPuller(tenable_io#13245,agents#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Reading persisted data
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Data retrieved from the persistence: None
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Running the persistence corrections steps
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> The persistence version value is <ZERO>, so no persistence will be allocated
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> TenableIoStatelessPuller(tenable_io#13245,agents#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Pull Started
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 44.53764 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(72.25MiB -> 72.25MiB), VMS(602.25MiB -> 602.25MiB)
INFO OutputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(67.56MiB -> 67.56MiB), VMS(1.08GiB -> 1.08GiB)
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 1.19257 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> The data is up to date!
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Data collection completed. Elapsed time: 65.009 seconds. Waiting for 43134.991 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,agents#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

### Audit log service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Reading persisted data
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Data retrieved from the persistence: None
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Running the persistence corrections steps
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> The persistence version value is <ZERO>, so no persistence will be allocated
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Pull Started
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 44.53764 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(72.25MiB -> 72.25MiB), VMS(602.25MiB -> 602.25MiB)
INFO OutputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(67.56MiB -> 67.56MiB), VMS(1.08GiB -> 1.08GiB)
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 1.19257 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> The data is up to date!
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Data collection completed. Elapsed time: 65.009 seconds. Waiting for 43134.991 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,audit_log#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

<details>
<summary>Restart the persistence</summary>

This collector uses persistent storage to download events in an orderly fashion and avoid duplicates. In case you want to re-ingest historical data or recreate the persistence, you can restart the persistence of this collector by following these steps:

1. Edit the configuration file.
2. Change the value of the `start_date_utc` parameter to a different one.
3. Save the changes.
4. Restart the collector.

The collector will detect this change and will restart the persistence using the parameters of the configuration file or the default configuration in case it has not been provided.

> ⚠️ Note that this action clears the persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.
</details>

### Plugins service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Reading persisted data
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Data retrieved from the persistence: None
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Running the persistence corrections steps
WARNING InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> The persistence version value is <ZERO>, so no persistence will be allocated
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Pull Started
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 44.53764 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(72.25MiB -> 72.25MiB), VMS(602.25MiB -> 602.25MiB)
INFO OutputProcess::MainThread -> [GC] global: 22.4% -> 22.4%, process: RSS(67.56MiB -> 67.56MiB), VMS(1.08GiB -> 1.08GiB)
INFO InputProcess::60/60 -> Ratelimiter -> Local rate limit is blocking new calls for 1.19257 seconds (DEBUG < 1 sec < INFO)
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> The data is up to date!
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Data collection completed. Elapsed time: 65.009 seconds. Waiting for 43134.991 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoStatelessPuller(tenable_io#13245,plugins#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733393152914):Number of requests made: 2; Number of events received: 1186; Number of duplicated events filtered out: 0; Number of events generated and sent: 1186; Average of events per second: 18.247.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

<details>
<summary>Restart the persistence</summary>

This collector uses persistent storage to download events in an orderly fashion and avoid duplicates. In case you want to re-ingest historical data or recreate the persistence, you can restart the persistence of this collector by following these steps:

1. Edit the configuration file.
2. Change the value of the `start_date_utc` parameter to a different one.
3. Save the changes.
4. Restart the collector.

The collector will detect this change and will restart the persistence using the parameters of the configuration file or the default configuration in case it has not been provided.

> ⚠️ Note that this action clears the persistence and cannot be recovered in any way. Resetting persistence could result in duplicate or lost events.
</details>

### Export assets service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Reading persisted data
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Data retrieved from the persistence: None
WARNING InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Persistence will be overridden due to the retrieved state is empty
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Running the persistence corrections steps
WARNING InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Some changes have been detected and the persistence needs to be updated. Previous content: None. New content: {'@persistence_version': 1, 'assets': {'uuid': None, 'status': None}, 'vulns': {'uuid': None, 'status': None, 'start_time_ts': None}}
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Updating the persistence
WARNING InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Persistence has been updated successfully
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Starting data collection every 28800 seconds
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Pull Started
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Loaded persisted state: {'@persistence_version': 1, 'assets': {'uuid': None, 'status': None}, 'vulns': {'uuid': None, 'status': None, 'start_time_ts': None}}
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Started processing (assets) export job with UUID 0995b5a8-773a-4e48-b9fa-ec2b89b02db6.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Started processing (vulns) export job with UUID 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) ->  Vulns `last_found` filter: (1733365576).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Processing assets job (UUID: 0995b5a8-773a-4e48-b9fa-ec2b89b02db6). Created at: 1733394376633, Status: None/None
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Processing vulns job (UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5). Created at: 1733394377174, Status: 4/6
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> assets job 0995b5a8-773a-4e48-b9fa-ec2b89b02db6 is FINISHED.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> vulns job 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5 is FINISHED.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Number of chunks available for assets: 5
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Number of chunks available for vulns: 6
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Starting download and temporary storage of vulnerabilities.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 1 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 2 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 3 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 4 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 5 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Downloading chunk 6 of vulnerabilities (Job UUID: 13a890b2-5725-4cf4-bd5c-afb5bd48c7e5).
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Completed download. Compressing vulnerabilities by asset_id.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Completed compression of all vulnerabilities by asset_id.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Starting the process of downloading and enriching assets with vulnerability information.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Processed 5 out of 5 chunks.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> All assets (4292) have been downloaded and enriched with vulnerabilities.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733394376062):Number of requests made: 20; Number of events received: 4292; Number of duplicated events filtered out: 0; Number of events generated and sent: 4292; Average of events per second: 76.177.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733394376062):Number of requests made: 20; Number of events received: 4292; Number of duplicated events filtered out: 0; Number of events generated and sent: 4292; Average of events per second: 76.176.
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> The data is up to date!
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Data collection completed. Elapsed time: 56.359 seconds. Waiting for 28743.641 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoExportAssetsPuller(tenable_io#13245,export_assets#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733394376062):Number of requests made: 20; Number of events received: 4292; Number of duplicated events filtered out: 0; Number of events generated and sent: 4292; Average of events per second: 76.176.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

### Export vulnerabilities service

<details>
<summary>Verify data collection</summary>

Once the collector has been launched, it is important to check if the ingestion is performed in a proper way. To do so, go to the collector’s logs console.

This service has the following components:

| **Component** | **Description** |
| --- | --- |
| Setup | The setup module is in charge of authenticating the service and managing the token expiration when needed. |
| Puller | The setup module is in charge of pulling the data in a organized way and delivering the events via SDK. |

#### Puller output

A successful initial run has the following output messages for the puller module:

> ℹ️ Note that the `PrePull` action is executed only one time before the first run of the `Pull` action.

```
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) Starting the execution of pre_pull()
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Reading persisted data
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Data retrieved from the persistence: {'@persistence_version': 1, 'uuid': 'f08cde36-9ef8-4b57-9510-60954b57f7e8'}
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Running the persistence upgrade steps
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Running the persistence corrections steps
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> No changes were detected in the persistence
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) Finalizing the execution of pre_pull()
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Starting data collection every 43200 seconds
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Pull Started
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Loaded persisted state: {'@persistence_version': 1, 'uuid': 'f08cde36-9ef8-4b57-9510-60954b57f7e8'}
WARNING InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> [400: POST] https://cloud.tenable.com/vulns/export/f08cde36-9ef8-4b57-9510-60954b57f7e8/cancel body=b'{"error":"Export cannot be cancelled."}'
WARNING InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> 400 Bad Request encountered during the operation: [400: POST] https://cloud.tenable.com/vulns/export/f08cde36-9ef8-4b57-9510-60954b57f7e8/cancel body=b'{"error":"Export cannot be cancelled."}'
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 4)
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 1)
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 2)
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 3)
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 5)
INFO InputProcess::ThreadPoolExecutor-1_0 -> Processing export (type - uuid - chunk_id): (vulns - 8a325f04-ca6e-4231-8f08-f10eb250101c - 6)
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> (Partial) Statistics for this pull cycle (@devo_pulling_id=1733395274557):Number of requests made: 8; Number of events received: 51560; Number of duplicated events filtered out: 0; Number of events generated and sent: 51560; Average of events per second: 105.031.
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733395274557):Number of requests made: 8; Number of events received: 51560; Number of duplicated events filtered out: 0; Number of events generated and sent: 51560; Average of events per second: 105.031.
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> The data is up to date!
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Data collection completed. Elapsed time: 490.916 seconds. Waiting for 42709.084 second(s) until the next one
```

After a successful collector’s execution (that is, no error logs found), you will see the following log message:

```
INFO InputProcess::TenableIoExportVulnerabilitiesPuller(tenable_io#13245,export_vulnerabilities#predefined) -> Statistics for this pull cycle (@devo_pulling_id=1733395274557):Number of requests made: 8; Number of events received: 51560; Number of duplicated events filtered out: 0; Number of events generated and sent: 51560; Average of events per second: 105.031.
```

> ℹ️ The value `@devo_pulling_id` is injected in each event to group all events ingested by the same pull action. You can use it to get the exact events downloaded in that `Pull` action in Devo’s search window.
</details>

## Collector operations

This section is intended to explain how to proceed with specific operations of this collector.

<details>
<summary>Verify collector operations</summary>

### Initialization

The initialization module is in charge of setup and running the input (pulling logic) and output (delivering logic) services and validating the given configuration.

A successful run has the following output messages for the initializer module:

```
INFO MainThread -> (CollectorMultithreadingQueue) standard_queue_multithreading -> max_size_in_messages: 10000, max_size_in_mb: 1024, max_wrap_size_in_items: 100
WARNING MainThread -> [INTERNAL LOGIC] DevoSender::_validate_kwargs_for_method__init__ -> The <address> does not appear to be an IP address and cannot be verified: collector-us.devo.io
WARNING MainThread -> [OUTPUT] OutputLookupSenders -> <threshold_for_using_gzip_in_transport_layer> setting has been modified from 1.1 to 1.0 due to this configuration increases the Lookup sender performance.
WARNING MainThread -> [INTERNAL LOGIC] DevoSender::_validate_kwargs_for_method__init__ -> The <address> does not appear to be an IP address and cannot be verified: collector-us.devo.io
INFO MainThread -> [OUTPUT] OutputMultithreadingController(threatquotient_collector) -> Starting thread
INFO MainThread -> [OUTPUT] DevoSender(standard_senders,devo_sender_0) -> Starting thread
INFO MainThread -> [OUTPUT] DevoSenderManagerMonitor(standard_senders,devo_1) -> Starting thread (every 600 seconds)
INFO MainThread -> [OUTPUT] DevoSenderManager(standard_senders,manager,devo_1)(devo_1) -> Starting thread
INFO MainThread -> [OUTPUT] DevoSender(lookup_senders,devo_sender_0) -> Starting thread
INFO MainThread -> [OUTPUT] DevoSenderManagerMonitor(lookup_senders,devo_1) -> Starting thread (every 600 seconds)
INFO MainThread -> [OUTPUT] DevoSenderManager(lookup_senders,manager,devo_1)(devo_1) -> Starting thread
INFO MainThread -> InitVariables Started
INFO MainThread -> start_time_value initialized
INFO MainThread -> verify_host_ssl_cert initialized
INFO MainThread -> event_fetch_limit_in_items initialized
INFO MainThread -> InitVariables Terminated
INFO MainThread -> [INPUT] InputMultithreadingController(threatquotient_collector) - Starting thread (executing_period=300s)
INFO MainThread -> [INPUT] InputThread(threatquotient_collector,threatquotient_data_puller#111) - Starting thread (execution_period=600s)
INFO MainThread -> [INPUT] ServiceThread(threatquotient_collector,threatquotient_data_puller#111,events#predefined) - Starting thread (execution_period=600s)
INFO MainThread -> [SETUP] ThreatQuotientDataPullerSetup(threatquotient_collector,threatquotient_data_puller#111,events#predefined) - Starting thread
INFO MainThread -> [INPUT] ThreatQuotientDataPuller(threatquotient_collector,threatquotient_data_puller#111,events#predefined) - Starting thread
```

### Events delivery and Devo ingestion

The event delivery module is in charge of receiving the events from the internal queues where all events are injected by the pullers and delivering them using the selected compatible delivery method.

A successful run has the following output messages for the initializer module:

```
INFO OutputProcess::SyslogSenderManagerMonitor(standard_senders,sidecar_0) -> Number of available senders: 1, sender manager internal queue size: 0
INFO OutputProcess::SyslogSenderManagerMonitor(standard_senders,sidecar_0) -> enqueued_elapsed_times_in_seconds_stats: {}
INFO OutputProcess::SyslogSenderManagerMonitor(standard_senders,sidecar_0) -> Sender: SyslogSender(standard_senders,syslog_sender_0), status: {"internal_queue_size": 0, "is_connection_open": True}
INFO OutputProcess::SyslogSenderManagerMonitor(standard_senders,sidecar_0) -> Standard - Total number of messages sent: 44, messages sent since "2022-06-28 10:39:22.511671+00:00": 44 (elapsed 0.007 seconds)
INFO OutputProcess::SyslogSenderManagerMonitor(internal_senders,sidecar_0) -> Number of available senders: 1, sender manager internal queue size: 0
INFO OutputProcess::SyslogSenderManagerMonitor(internal_senders,sidecar_0) -> enqueued_elapsed_times_in_seconds_stats: {}
INFO OutputProcess::SyslogSenderManagerMonitor(internal_senders,sidecar_0) -> Sender: SyslogSender(internal_senders,syslog_sender_0), status: {"internal_queue_size": 0, "is_connection_open": True}
INFO OutputProcess::SyslogSenderManagerMonitor(internal_senders,sidecar_0) -> Internal - Total number of messages sent: 1, messages sent since "2022-06-28 10:39:22.516313+00:00": 1 (elapsed 0.019 seconds)
```

> ℹ️ By default, these information traces will be displayed every 10 minutes.

#### Sender services

The Integrations Factory Collector SDK has 3 different senders services depending on the event type to delivery (`internal`, `standard`, and `lookup`). This collector uses the following Sender Services:

| **Sender services** | **Description** |
| --- | --- |
| `internal_senders` | In charge of delivering internal metrics to Devo such as logging traces or metrics. |
| `standard_senders` | In charge of delivering pulled events to Devo. |

#### Sender statistics

Each service displays its own performance statistics that allow checking how many events have been delivered to Devo by type:

| **Logging trace** | **Description** |
| --- | --- |
| `Number of available senders: 1` | Displays the number of concurrent senders available for the given Sender Service. |
| `sender manager internal queue size: 0` | Displays the items available in the internal sender queue.<br>> ℹ️ This value helps detect bottlenecks and needs to increase the performance of data delivery to Devo. This last can be made by increasing the concurrent senders. |
| `Total number of messages sent: 44, messages sent since "2022-06-28 10:39:22.511671+00:00": 21 (elapsed 0.007 seconds)` | Displayes the number of events from the last time and following the given example, the following conclusions can be obtained:<br>- 44 events were sent to Devo since the collector started.
- The last checkpoint timestamp was `2022-06-28 10:39:22.511671+00:00`.
- 21 events where sent to Devo between the last UTC checkpoint and now.
- Those 21 events required `0.007 seconds` to be delivered.<br>> ℹ️ By default these traces will be shown every 10 minutes. |
</details>

<details>
<summary>Check memory usage</summary>

To check the memory usage of this collector, look for the following log records in the collector which are displayed every 5 minutes by default, always after running the memory-free process.

- The used memory is displayed by running processes and the sum of both values will give the total used memory for the collector.
- The global pressure of the available memory is displayed in the `global` value.
- All metrics (Global, RSS, VMS) include the value before freeing and after `previous -> after freeing memory`

```
INFO InputProcess::MainThread -> [GC] global: 20.4% -> 20.4%, process: RSS(34.50MiB -> 34.08MiB), VMS(410.52MiB -> 410.02MiB)
INFO OutputProcess::MainThread -> [GC] global: 20.4% -> 20.4%, process: RSS(28.41MiB -> 28.41MiB), VMS(705.28MiB -> 705.28MiB)
```

> ℹ️ Differences between `RSS` and `VMS` memory usage:
> ℹ️ 
> ℹ️ - `RSS` is the Resident Set Size, which is the actual physical memory the process is using
> ℹ️ - `VMS` is the Virtual Memory Size which is the virtual memory that process is using
</details>

<details>
<summary>Enable/disable the logging debug mode</summary>

Sometimes it is necessary to activate the debug mode of the collector's logging. This debug mode increases the verbosity of the log and allows you to print execution traces that are very helpful in resolving incidents or detecting bottlenecks in heavy download processes.

- To enable this option you just need to edit the configuration file and change the debug_status parameter from false to true and restart the collector.
- To disable this option, you just need to update the configuration file and change the debug_status parameter from true to false and restart the collector.

For more information, visit the configuration and parameterization section corresponding to the chosen deployment mode.
</details>

## Change log

| **Release** | **Released on** | **Release Type** | **Details** | **Recommendations** |
| --- | --- | --- | --- | --- |
| `v2.2.0` | 6/3/2025 | [BUG FIX]  
[IMPROVEMENT] | **Improvements:**<br>- Upgraded the DCSDK to 1.16.2
- Upgraded SDK image base to 1.5.1<br>**Bug Fixing:**<br>- Fixed wrong parameter issue with was_vulnerabilities service. | `Recommended version` |
| `v2.0.0` | 11/14/2024 | [BUG FIX]  
[IMPROVEMENT] | **Improvements:**<br>- A complete refactor of the code was implemented, streamlining services and enhancing overall performance. This new architecture improves maintainability and extensibility, ensuring a smoother experience in future updates.<br>**Bug Fixing:**<br>- Significant improvements were made in memory handling, particularly in high-load services. By utilizing compression techniques, memory usage was optimized, reducing the risk of system lockups and boosting efficiency under heavy workloads. | `Update` |
| `v1.6.0` | 10/22/2024 | [NEW FEATURE]   
[IMPROVEMENT] | **New Features:**<br>- New service: `was_vulnerabilities`: Returns a list of vulnerabilities detected by Tenable Web App Scanning API v2 scans.<br>**Improvements:**<br>- Update DC SDK from v1.12.1 to v1.13.1 | `Update` |
| `v1.4.0` | 4/12/2024 | [IMPROVEMENT] | **Improvements:**<br>- Upgrade DC SDK to the latest version 1.11.1. Upgrade the Docker base image to 1.2.0. | `Update` |
| `v1.3.0` | 10/13/2023 | [NEW FEATURE]   
[IMPROVEMENT] | **New Features:**<br>- Added new data source `export_vulnerabilities` as a new service.<br>**Improvements:**<br>- Upgrade DC SDK to the latest version 1.9.2. | `Update` |
| `v1.2.5` | 4/5/2023 | [IMPROVEMENT] | **Improvements:**<br>- Upgrade DC SDK to the latest version 1.7.2.dev1 to avoid errors in the sender module. These errors made the collector restart constantly. Added a lock to enhance the sender object. | `Update` |
| `v1.2.4 ` | 4/3/2023 | [IMPROVEMENT] | **Improvements:**<br>- The `export_assets` service now cancels previous jobs (if any) and launches new jobs instead of waiting for all other jobs to finish. Upgrade DC SDK to the latest version 1.7.0. | `Update` |
| `v1.2.3` | 3/27/2023 | [BUG FIX]  
[IMPROVEMENT] | **Bug Fixing:**<br>- Resolved a fix in the `audit_log` service, as it was sending duplicate events upon change of day.<br>**Improvements:**<br>- Adjusted rate limits. | `Update` |
| `v1.2.2` | 3/26/2023 | [BUG FIX]  
[IMPROVEMENT] | **Bug Fixing:**<br>- Fixed the `audit_log` service, which wasn't paginating through the available results and didn't send anything to Devo.<br>**Improvements:**<br>- Avoided confusing ERROR logs, standardized statistic logs, and added rate limits. | `Update` |
| `v1.2.0` | 3/22/2023 | [NEW FEATURE]   
[BUG FIX] | **New Features:**<br>- A new `export_assets` service has been created to allow massive data ingestion, enriched with vulnerabilities information.<br>**Bug Fixing:**<br>- Resolved constant 500-level errors caused by the previous assets endpoint. | `Update` |