Salesforce collector

Service description

Salesforce is a customer relationship management solution that brings companies and customers together. It's an integrated CRM platform that gives all the departments, including marketing, sales, commerce, and service. This Salesforce collector provides the possibility to integrate Salesforce with Devo in order to easily send data to the Devo platform or customizing it by, for example, making your own Activeboards.

Data source description

The Salesforce collector retrieves data from the following “Salesforce Objects“ (also known as SObjects). The collector processes the Salesforce API responses and sends them to the Devo platform, which will categorize all the information received on specific tables on your Devo domain.

Salesforce resources

Listed in the table below are the SObject names, details, and to which tables Devo sends the data:

Setup

The Salesforce collector works over the Salesforce API. You need to enable the API in your Salesforce account to allow the collector to get the data and make sure you have the required permissions.

Enable API access on the account

1. Log in to your Salesforce account.

2. Click the gear icon and select Service Setup.

   

3. Open the Users section in the left tree and select Users. In this section, search your user and click your user profile.

   

4. In the profile screen, click Edit.

   

5. Scroll down to Administrative Permissions and search for the API Enabled checkbox. Check it and save the changes.

   

Authentication

The Salesforce collector supports User/Password and OAuth2 authentication methods.

Getting credentials (User/Password)

This method needs a username, password, and a security token.

Usually, all come together in the same mail when the user creates a Salesforce account. However, if the security token does not come in the mail or you can not find it, follow the next steps to obtain a new one:

1. Log in to your Salesforce account with the user to be used.

2. Click the user icon at the top right of the screen and select Settings.

   

3. Open the Users section in the left tree and select Users.

4. In the left menu, go to the My personal information section and select Reset my security token.

   

5. On this screen, click Reset Security Token. You will receive a mail with the new security token.

Getting credentials (OAuth2)

To enable OAuth2 authentication, you need to create a Connected App and obtain the client_id, client_secret, and redirect_uri to use it.

1. Log in to your Salesforce account.

2. Click the gear icon and select Service Setup.

   

3. Type Apps in the left search bar, then click App Manager and create a new Connected App.

   

4. Fill all the mandatory fields, check the Enable OAuth settings option and fill the new fields as shown in the following capture:
   

   

5. Save the changes and note down the client_id (Consumer Key), client_secret (Consumer Secret), and redirect_uri (Callback URL)
   

   

To get the access token file, we need to execute this script and follow these steps.

1. Execute the script using the following command:

   python salesforce_access_token_generator.py --client_id <salesforce_client_id> --client_secret <salesforce_client_secret> --redirect_uri <salesforce_client_uri>

   This script will open your predefined browser to log in to Salesforce and authorize the client in your Salesforce domain, and will redirect to other URL like in the following image:

   .

2. Now, copy the URL and paste it in the terminal where you have executed the script. Press ENTER and the script will finalize the token file creation. Then, you should see an access_token.json file in the same directory as the script.

3. Finally, paste the access token file in the credentials folder of the collector structure.

Minimum permissions

In order to get the required data, your user must have some minimal permissions. The user must be under a Salesforce user license and we need to provide the standard user profile for almost all services/SObjects (with some exceptions) after enabling the API access.

The special permissions required for each SObject are specified below:

Run the collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure should be created for being used when running the Salesforce collector:

<any_directory>
└── devo-collectors/
    └── salesforce/
        ├── certs/
        │   ├── chain.crt
        │   ├── <your_domain>.key
        │   └── <your_domain>.crt
        ├── credentials/
        │ └── access_token.json
        └── config/ 
            └── config-salesforce.yaml

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in <any directory>/devo-collectors/salesforce/certs. Learn more about security credentials in Devo here.

Editing the config-salesforce.yaml file

In the config-salesforce.yaml file, replace the <username>, <password>, and <security_token> values and enter the ones that you got in the previous steps. In the <short_unique_identifier> placeholder, enter the value that you choose.

globals:
  debug: false
  id: not_used
  name: salesforce
  persistence:
    type: filesystem                                               # File system persistence ON
    config:
      directory_name: state                                        # Directory where the persistence will be saved in case of using filesystem
outputs:
  devo_1:                                                          # Cloud Devo config EU (for US use us.elb.relay.logtrust.net)
    type: devo_platform
    config: 
      address: eu.elb.relay.logtrust.net
      port: 443
      type: SSL
      chain: chain.crt
      cert: <your_domain>.crt
      key: <your_domain>.key
inputs:
  salesforce:
    id: <short_unique_identifier>                                 # The value of this field will be used internally for having independent persistence areas
    enabled: true
    requests_per_second: 5                                        # Setup how many request API por second
    credentials:                                                  # Check the credentials section in the documentation to know which fields are necessary for your authentication method
      username: <salesforce_user_value>                           # Salesforce User
      password: <salesforce_password_value>                       # Salesforce Password
      security_token: <salesforce_security_token_value>           # Salesforce user's security token
      client_id: <oauth_client_id>                                # Salesforce OAuth client id
      client_secret: <oauth_client_secret>                        # Salesforce OAuth client secret
      redirect_uri: <oauth_redirect_uri>                          # Salesforce OAuth client redirect uri
      access_token_filename: access_token.json                    # Salesforce OAuth access token file. Default value "access_token.json". If not present, uses access_token.json too
      salesforce_domain: login                                    # OPTIONAL. Salesforce domain field. Default value "login". If not present, uses login too
    services:                                                     # Services available for this collector
      custom_1:
        request_period_in_seconds: 86400                          # Setting up time interval between API requests. 300 sec is recommended. With snapshot mode enabled is recommended a high request period, as in the sample.
        sobject_name: User # Samples: Account, User, Case, ...
        key: LastModifiedDate                                     # Field used to filter data, must be datetime field
        fields:                                                   # SObject fields to retrieve. All default fields will be retrieved in supported SObjects
          - Id
          - LastModifiedById
          - LastModifiedDate
          - Name
        tag: my.app.salesforce.sample                             # Custom tag if want to retrieve data to not common tables
        snapshot_mode: true                                       # This field has priority over start_time. Retrieve ALL existing data in each iteration
      Account:
        request_period_in_seconds: 300
        key: LastModifiedDate
        custom_fields:
          - Id
          - LastModifiedById
          - LastModifiedDate
          - Name
        start_time: '2020-07-14T00:00:00Z'                        # As str Year-Month-DayTHour-Min-Sec. Unnecessary field if snapshot_mode is true
      EventLogFile:                                               # Special SObject, if used as others, only retrieve generig data from the EventLogFile SObjects
        process_results:                                          # OPTIONAL. Field used to process all received files in the crm.salesforce.<event_types> tables
          enabled: true
          compress: false                                         # OPTIONAL. Request compressed files to reduce the bandwith usage. Frequently this field is not needed
        start_time: '2020-07-14T00:00:00Z'

Download the Docker image

The collector should be deployed as a Docker container. Click here to download the Docker image of the collector as a .tgz file.

Use the following command to add the Docker image to the system:

gunzip -c collector-salesforce-docker-image-<version>.tgz | docker load

The Docker image can be deployed on the following services:

• Docker

• Docker Compose

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/salesforce/

docker run \
--name collector-salesforce \
--volume $PWD/certs:/devo-collector/certs \
--volume $PWD/config:/devo-collector/config \
--volume $PWD/state:/devo-collector/state \
--env CONFIG_FILE=config-salesforce.yaml \
--rm -it docker.devo.internal/collector/salesforce:<version>

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/salesforce/ directory.

version: '3'
services:
  collector-salesforce:
    build:
      context: .
      dockerfile: Dockerfile
    image: docker.devo.internal/collector/salesforce:${IMAGE_VERSION:-latest}
    container_name: collector-salesforce
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config-salesforce.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/salesforce/ directory:

IMAGE_VERSION=<version> docker-compose up -d