---
title: "Salesforce collector"
canonical: "https://docs.devo.com/space/latest/94657195/Salesforce%20collector"
format: markdown
---
> Macro (toc)
## Overview
[Salesforce](https://www.salesforce.com/) is a customer relationship management (CRM) solution that brings companies and customers together. It's one integrated CRM platform that gives all the departments, including marketing, sales, commerce, and service. This collector provides the possibility to integrate Salesforce with the Devo Platform making it easy to query and analyze the Salesforce data, view it in the pre-configured Activeboards, or customize them to enable Enterprise IT and Cybersecurity teams to make impactful data-driven decisions.
The Devo Salesforce Collector enables customers to retrieve data from the “Salesforce Objects API“ (a.k.a. `SObjects`). The collector processes the Salesforce API responses and sends them to the Devo platform which then categorizes all data received on tables along rows and columns in your Devo domain.
> ⚠️ This collector works only with the “Enterprise”, “Performance”, “Unlimited”, “Developer”, and “[http://Database.com](http://Database.com) ” Salesforce editions. Please ensure that you have one of the listed editions.
## Data source description
Listed in the table below are the `SObject` names, their details, and how the Devo Platform treats the data. You can see that all the `SObject` events (JSON documents) fall under the `crm.salesforceobjects` technologies, except for the “event log files” processed events, which fall under the `crm.salesforce` one:
| **Product** | **Data source** | **Devo table** | **Collector service** | **Related remote endpoint** | **Description** |
| --- | --- | --- | --- | --- | --- |
| Salesforce | Account | `crm.salesforceobjects.account` | `Account` | The main endpoint is `GET https://{instance}/services/data/v{version}/{query_method}?q={query}`, where: - `{instance}` is your Salesforce domain name.
- `{version}` is an internal variable that controls the API's versioning.
- `{query_method}` can be `query` or `queryAll`.
- `{query}` represents an SQL-like query that is composed in the following way:
- `@fields` represents the list of fields to be retrieved by each Collector Service.
- `@table` is each Collector Service.
- `@key` should be a date field and is used for filtering and ordering.
- `@date_filter` is internally controlled by the collector. Check the “Running the data collector” sections for more information. | Represents an individual account, which is an organization or person involved with your business (such as customers, competitors, and partners). |
| Case | `crm.salesforceobjects.case` | `Case` | Represents a case, which is a customer issue or problem. |
| Content Version | `crm.salesforceobjects.contentversion` | `ContentVersion` | Represents a specific version of a document in Salesforce CRM Content or Salesforce Files. |
| Dashboard | `crm.salesforceobjects.dashboard` | `Dashboard` | Represents a dashboard, which shows data from custom reports as visual components. |
| Login History | `crm.salesforceobjects.loginhistory` | `LoginHistory` | Represents the login history for all successful and failed login attempts for organizations and enabled portals. |
| Opportunity | `crm.salesforceobjects.opportunity` | `Opportunity` | Represents an opportunity, which is a sale or pending deal. |
| Report | `crm.salesforceobjects.report` | `Report` | Represents a report, a set of data that meets certain criteria, displayed in an organized way. |
| User | `crm.salesforceobjects.users` | `User` | Represents a user in your organization. |
| Event Log Files | - `crm.salesforceobjects.eventlogfile`
- `crm.salesforce.` if file processing is enabled | `EventLogFiles` | Represents event log files for event monitoring. Possibility to set `interval` to `hourly` or `daily` (default) from the user configuration. > ⚠️ **Risk of reaching rate limits**. Enabling file processing may cause a high number of requests against the API. |
| Setup Audit Trail | `crm.salesforceobjects.setupaudittrail` | `SetupAuditTrail` | Represents changes you or other admins made in your org’s Setup area for at least the last 180 days. For more information, see SetupAuditTrail in the [Object Reference for the Salesforce Platform](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_setupaudittrail.htm). |
## External links
| **References** | **Details** | **Link** |
| --- | --- | --- |
| Salesforce `SObjects` | API guidelines and documentation for Salesforce Objects. | [Salesforce Developers](https://developer.salesforce.com/docs/atlas.en-us.sfFieldRef.meta/sfFieldRef) |
| Event types for `EventLogFiles` | List of all the possible event types that are ingested in the `crm.salesforce.` tables. > ℹ️ All `event_type`s will be in lowercase and without spaces. | [Salesforce Developers](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_eventlogfile_supportedeventtypes.htm) |
## Vendor setup
The Salesforce Collector works over the Salesforce API. We need to enable API access and ensure its permissions in your Salesforce account to allow the collector to get the data correctly.
> ℹ️ The steps and images explained here have been done using the Lightning interface.
## Enable API access on the account
> 📝 In case you encounter any difficulties in this step, we suggest you visit the corresponding article maintained by Salesforce [Help And Training Community](https://help.salesforce.com/s/articleView?id=000331470&type=1)
### Steps
1. Go to [Login | Salesforce](https://login.salesforce.com/) and sign in with your account.
2. In the top-right corner, click and select
| **Steps** | **Snapshot** |
| --- | --- |
| Login to your Salesforce account. | |
| Click on the gear icon (near the user icon) and select “Service Setup“. |  |
| In the setup interface, go to the left tree and deploy “Users“ and select “Users“. |  |
| Here, find the user you want to enable the API Access for and click on its associated user profile (its corresponding value from the **Profile** column). |  |
| In the profile screen, click on “Edit“. > 📝 Take into account that you will be modifying the user’s profile settings, so any other user that has this same profile will have the same settings applied. |  |
| Scroll down to “Administrative Permissions“, and search for “API Enabled“. |  |
| Check it and save the changes. | |
## Authentication
The Salesforce Collector supports both User/Password and OAuth 2 authentication methods. Choose your preferred option.
### Getting credentials (User/Password)
This method needs the `username`, `password`, and `security_token`.
Usually, everything comes together on the same email when the user is registered in a Salesforce account, but if the security token does not come in the email or you cannot find it, follow the next steps to obtain a new one:
| **Steps** | **Snapshot** |
| --- | --- |
| Login to your Salesforce account with the user that will be used to access the API. | |
| Click on the user icon (at the top right of the screen) and select “Settings“. |  |
| In the settings left menu, find “My personal information“ and search for “Reset my security token“. |  |
| On this screen, click on “Reset Security Token“ and you will receive a mail with the new security token. | |
### Getting credentials (OAuth 2)
To enable OAuth 2 authentication we need to create a “Connected App“ and obtain the `client_id`, `client_secret`, and `redirect_uri` values to use it.
> ℹ️ OAuth 2 is implemented in two ways in this collector.
> ℹ️
> ℹ️ - The first one needs all parameters (`username`, `password`, `security_token`, `client_id`, `client_secret,` and `redirect_uri`) to automatically authorize the OAuth2 client to access the information.
> ℹ️ - The second one authorizes the OAuth client manually, so only needs `client_id`, `client_secret`, and the `access_token` file obtained with [this script](https://drive.google.com/file/d/1p-_rzE1mzAS5Gr83xZBU8RyHTScRz0Ix/view?usp=drive_link).
| **Steps** | **Snapshot** |
| --- | --- |
| Login to your Salesforce account. | |
| Click on the gear icon (near the user Icon) and select “Service Setup“. |  |
| Type “Apps“ in the left search bar, click on “App Manager“ and create a new Connected App. |  |
| Fill the mandatory fields, check the Enable “OAuth settings checkbox” and fill the new fields as shown in the sample. |  |
| Save the changes and write down the `client_id`(Consumer Key), `client_secret` (Consumer Secret), and `redirect_uri` (Callback URL). |  |
### Getting the access token file
To get the access token file, we need to execute this [script](https://drive.google.com/file/d/1p-_rzE1mzAS5Gr83xZBU8RyHTScRz0Ix/view?usp=drive_link) with the following steps:
| **Steps** |
| --- |
| Prepare your environment: > ⚠️ For executing the access token file generation script, we need the following Python 3.6 (or above) dependencies installed
> ⚠️
> ⚠️ - click 7.1.2 (or above)
> ⚠️ - requests 2.25.1 (or above)
> ⚠️ - json (normally comes by default in the python standard library)
> ⚠️ - urllib (normally comes by default in the python standard library)
> ⚠️ - webbrowser (normally comes by default in the python standard library) |
| Execute the script with the following command: ```
python salesforce_access_token_generator.py --client_id --client_secret --redirect_uri
``` This script will open our predefined browser with a Salesforce login page. Log in to Salesforce and authorize the client in your domain. Once authorized, it will redirect to another URL like in the following image:  |
| Now, we need to copy the URL and paste it into the terminal where we have executed the script, which was waiting for this URL. Press ENTER and the script will finalize the token file creation.
Once finalized the execution, we should see an `access_token.json` file in the same directory as the script. |
| Next, we need to convert the content of this file into base64 format. This can be done using an online tool such as [Base64 Encode and Decode - Online](https://www.base64encode.org/). ***NOTE:*** this is a 3rd party tool that Devo does not manage, and is provided only as an example. |
| The base64 encoded string is then used as the value of the `initial_access_token_base64` key in the collector’s user configuration. |
### Permissions
To be able to retrieve the data, we need to have some minimal permissions in the user configured for this collector. This user must be under a “Salesforce” user license and we need to provide the “standard user” profile for almost all the services/SObjects (with some exceptions) after enabling the API access.
We are now selecting from the Standard Fields [Salesforce Developers](https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_fields.htm)
For each object they have there own standard Fields. These will all be selected all of the time. **Note:** If the authentactor doesn’t have permissions to view the standard or custom columns you’ll need to add Field Level Security to Visible. [Salesforce Developers](https://developer.salesforce.com/docs/atlas.en-us.securityImplGuide.meta/securityImplGuide/admin_fls.htm)
The special required permissions by `SObjects` are specified below:
- LoginHistory and User: we can use a profile inherited from “standard user“. To add the missing permissions, we just need to enable the “Manage Users“ permission in the user’s profile.
> ⚠️ To understand why do we need the “Manage Users” permissions for these `SObjects`, check the Salesforce's Developer documentation (Special Access Rules section): [LoginHistory](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_loginhistory.htm) and [User](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_user.htm).
- EventLogFiles: we can use a profile inherited from “standard user“. To add the missing permissions, we just need to enable the “View Event Log Files“ permission in the user’s profile.
## Devo collector features
| **Feature** | **Details** |
| --- | --- |
| Allow parallel downloading (`multipod`) | `Not allowed` |
| Running Environments | `Collector Server`, `On-Premise` |
## Minimum configuration required for basic pulling
Although this collector supports advanced configuration, the fields required to download data with basic configuration are defined below.
> 📝 This minimum configuration refers exclusively to those specific parameters of this integration. There are more required parameters related to the generic behavior of the collector. Check the detail of the parameterization for more information.
| **Setting** | **Details** |
| --- | --- |
| `` | Username to authenticate the service, if using the User/Password authentication mechanism. |
| `` | Password to authenticate the service, if using the User/Password authentication mechanism. |
| `` | Security token to authenticate the service, if using the User/Password authentication mechanism. |
| `` | Client ID to authenticate the service, if using the OAuth 2 authentication mechanism. |
| `` | Client Secret to authenticate the service, if using the OAuth 2 authentication mechanism. |
| `` | The result obtained from the script described in the Getting the access token file section, if using the OAuth 2 authentication mechanism. |
> ℹ️ See the **Accepted authentication methods** section to verify what settings are required based on the desired authentication method.
## Accepted authentication methods
Depending on how did you obtain your credentials, you will have to either fill or delete the following properties on the JSON `credentials` configuration block.
| **Authentication method** | **Username** | **Password** | **Security token** | **Client ID** | **Client secret** | **Salesforce domain** | **Initial access token in Base64** |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Username/Password | [REQUIRED] | [REQUIRED] | [REQUIRED] | | | [REQUIRED] | |
| OAuth 2 with all the parameters (without `access_token`) | [REQUIRED] | [REQUIRED] | [REQUIRED] | [REQUIRED] | [REQUIRED] | [REQUIRED] | |
| OAuth 2 with the `access_token` file manually generated | | | | [REQUIRED] | [REQUIRED] | [REQUIRED] | [REQUIRED] |
## Run the collector
Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).
> Macro (rw-ui-tabs-macro)
>
> > Macro (rw-tab)
>
> We use a piece of software called Collector Server to host and manage all our available collectors.
>
> To enable the collector for a customer:
>
> 1. In the **Collector Server** **GUI**, access the **domain** in which you want this instance to be created
> 2. Click **Add Collector** and find **Salesforce Collector - Integrations Factory**.
> 3. In the **Version** field, select the latest value.
> 4. In the **Collector Name** field, set the value you prefer (this name must be unique inside the same Collector Server domain).
> 5. In the sending method select **Direct Send. Direct Send** configuration is optional for collectors that create `Table` events, but mandatory for those that create `Lookups`.
> 6. In the **Parameters** section, establish the **Collector Parameters** as follows below:
>
> ### Editing the JSON configuration
>
> ```
> {
> "global_overrides": {
> "debug": false
> },
> "inputs": {
> "salesforce": {
> "id": 12345,
> "enabled": true,
> "api_version": "62.0",
> "credentials": {
> "username": "",
> "password": "",
> "security_token": "",
> "client_id": "",
> "client_secret": "",
> "salesforce_domain": "",
> "initial_access_token_base64": ""
> },
> "services": {
> "Account": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "Case": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "ContentVersion": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "Dashboard": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "EventLogFile": {
> "request_period_in_seconds": 300,
> "start_time": "0000-00-00T00:00:00Z",
> "process_results": {
> "enabled": true
> },
> "event_log_type": [
> "*"
> ]
> },
> "LoginHistory": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "Opportunity": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "Report": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "User": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> "SetupAuditTrail": {
> "request_period_in_seconds": 600,
> "start_time": "0000-00-00T00:00:00Z"
> },
> " like custom_user, custom_opportunity": {
> "start_time": "0000-00-00T00:00:00Z",
> "key": "most of the time LastModifiedDate",
> "fields": [
> "must append __c to value",
> "Id__c",
> "LastModifiedById__c",
> "LastModifiedDate__c",
> "Name__c"
> ]
> }
> }
> }
> }
> }
>
> ```
>
> > ℹ️ All defined service entities will be executed by the collector. If you do not want to run any of them, just remove the entity from the `services` object.
>
> Please replace the placeholders with real world values following the description table below:
>
> | | | | | |
> | --- | --- | --- | --- | --- |
> | **Parameter** | **Data type** | **Type** | **Value range / Format** | **Details** |
> | `` | `int` | `Mandatory` | Minimum length: 1
> Maximum length: 5 | Use this param to give an unique id to this input service. > ⚠️ This parameter is used to build the persistence address, do not use the same value for multiple collectors. It could cause a collision. |
> | `` | `bool` | `Mandatory` | `false` / `true` | Use this param to enable or disable the given input logic when running the collector. If the value is `true`, the input will be run. If the value is `false`, it will be ignored. |
> | `api_version` | `str` | `Mandatory` | 62.0 | This parameter allows you to update the salesforce api version whenever you want. |
> | `` | `str` | `Optional`1 | Any | Username to authenticate the service. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Optional`1 | Any | Password to authenticate the service. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Optional`1 | Any | Security token to authenticate the service. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Optional`1 | Any | Client ID to authenticate the service. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Optional`1 | Any | Client Secret to authenticate the service. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Mandatory` | Any | The Salesforce **subdomain** name. If you log in to Salesforce using devo.my.salesforce.com, then this field is `devo.my`. |
> | `` | `str` | `Optional`1 | A valid Base 64 string | The result obtained from the script described in the Getting the access token file section. > 📝 This parameter should be either filled or removed, depending on your authentication mechanism. Check the Credentials' settings section. |
> | `` | `str` | `Optional` | | The amount (in seconds) in which the service’s collection is scheduled. > 📝 This parameter can be removed if not set. |
> | `` | `str` | `Optional`2 | A key of a timestamp value from that service ie LastModifiedDate | This will be used for internal comparisons to sort the events that the service obtains. Each service has this parameter set by default (usually `LastModifiedDate` or the most similar one), but you can override it here. > ℹ️ This parameter is mandatory for custom services. > 📝 This parameter can be removed if not set. |
> | `<``override_tag_value``>` | `str` | `Optional` | A [Devo tag](https://docs.devo.com/confluence/ndt/latest/devo-parsers/about-devo-tags) | You can customize the destination tag of the events received. If it is not set, the default value is `crm.salesforceobjects.{sobject_name}`. > 📝 This parameter can be removed if not set. |
> | `` | `Array` (of `str`) | `Optional` | Additional keys from the events received from that service **Only for custom services** | You can add additional fields that are not present by default in the predefined services by adding them to this array. It will have this structure: ```
> "fields": [
> "",
> "",
> [...],
> ""
> ]
> ``` Here is an example (please note this is very dependant on the queried SObject): ```
> "fields": [
> "BrowserType__c",
> "DeviceNativeUid__c",
> "DeviceType__c",
> "IsVerified__c"
> ]
> ``` > ℹ️ This fields is ignored in custom services because the `fields` parameter is used instead. > ⚠️ To know what fields can you select, visit the [Salesforce’s Standard Objects reference](https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_objects_list.htm). Ensure that the added values are correctly written; otherwise, the collector will return an error. > 📝 This parameter can be removed if not set. |
> | `` | `str` | `Optional` | A date with format `YYYY-MM-DDTHH:mm:ssZ` | This will allow you to start from a specific date in case you want to ingest historic events. If not set, it will start at the current time. > 📝 This parameter can be removed if not set. |
> | `` | `bool` | `Optional` | false / true | This will enable that the Event Log Files services actually fetches the log files and sends their content to the `crm.salesforce.` tables. > ⚠️ **Risk of reaching rate limits, if **`true`. This parameter may cause a high number of requests, as it performs a request for each EventLogFile event to get the content in those files.
> > ⚠️
> > ⚠️ The older the `start_time`, the higher the risk of reaching rate limits. > 📝 This parameter can be removed if not set. Remove the whole `process_results` block in such case. |
> | `` | `bool` | `Optional` | false / true | This will enable compression for the log files' fetching. This is usually not needed. > 📝 This parameter can be removed if not set. Remove the whole `process_results` block in such case. |
> | `` | `str` | `Optional`2 | Words, numbers and underscores. None of the predefined Collector Services, as these names are already taken. **You Must use custom_object** examples: custom_user, custom_opportunity | When you want to set up a custom service that pulls data from SObjects other than the ones available in the predefined services (listed in the Data Source Description section), you create a new configuration block with the following structure: ```
> " like custom_user, custom_opportunity": {
> "start_time": "0000-00-00T00:00:00Z",
> "key": "most of the time LastModifiedDate",
> "fields": [
> "must append __c to value",
> "Id__c",
> "LastModifiedById__c",
> "LastModifiedDate__c",
> "Name__c"
> ]
> }
> ``` Here is an example for a sample SObject: Make sure there are no duplicates please check salesforce api docs for standard columns ```
> "custom_account": {
> "sobject_name": "Account",
> "key": "LastModifiedDate",
> "fields": [
> "CreatedDate__c",
> "Id__c"
> ]
> }
> ``` > 📝 This block can be removed if not set. |
> | `override_interval_value` | `str` | `Optional` | `daily` (default) or `hourly` | This value indicates if the data is collected `daily` or `hourly` in the customer’s Salesforce. If it is collected `hourly`, this must be defined in order to be able to collect all the events available. |
> | `event_log_type` | `array` | `Mandatory` | Use a `*` for all or if removed will be all. any event log file object such as `["apexcallout", "bulkapi", "report", ... etc]` | This allows you to filter which event log files get downloaded and sent to Devo. Thus allowing you to filter out which ones you don’t want. |
>
> > 📝 1: Visit the **Accepted authentication methods** section for details.
> > 📝
> > 📝 2: These parameters are optional –or ignored– for predefined services, but mandatory for custom ones.
>
> 
>
> > Macro (rw-tab)
>
> This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.
>
> ### Structure
>
> The following directory structure should be created for being used when running the collector:
>
> ```
>
> └── devo-collectors/
> └── /
> ├── certs/
> │ ├── chain.crt
> │ ├── .key
> │ └── .crt
> ├── state/
> └── config/
> └── config.yaml
> ```
>
> > ⚠️ Replace `` with the proper value.
>
> ### Devo credentials
>
> In Devo, go to **Administration → Credentials → X.509 Certificates**, download the **Certificate**, **Private key** and **Chain CA** and save them in `/certs/`. Learn more about security credentials in Devo [here](#).
>
> 
>
> > ⚠️ Replace `` with the proper value.
>
> ### Editing the config.yaml file
>
> ```
> globals:
> debug: false
> id: not_used
> name: salesfoce_collector
> persistence:
> type: filesystem
> config:
> directory_name: state
> outputs:
> devo_1:
> type: devo_platform
> config:
> address: collector-us.devo.io
> port: 443
> type: SSL
> chain:
> cert:
> key:
> inputs:salesforce:
> id: 12345
> enabled: true
> api_version: "62.0"
> credentials:
> username:
> password:
> security_token:
> client_id:
> client_secret:
> salesforce_domain:
> initial_access_token_base64:
> services:
> Account:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> Case:
> request_period_in_seconds: 30
> start_time: "0000-00-00T00:00:00Z"
> ContentVersion:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> Dashboard:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> EventLogFile:
> request_period_in_seconds: 30
> start_time: "0000-00-00T00:00:00Z"
> process_results:
> enabled: true
> event_log_type:
> ["*"]
> LoginHistory:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> Opportunity:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> Report:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> User:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> SetupAuditTrail:
> request_period_in_seconds: 600
> start_time: "0000-00-00T00:00:00Z"
> # : must have custom_