--- title: "Microsoft Defender ATP for Endpoint collector" canonical: "https://docs.devo.com/space/latest/178618369/Microsoft%20Defender%20ATP%20for%20Endpoint%20collector" format: markdown --- > Macro (excerpt-include) > Macro (toc) ## Purpose An analyst wants to detect malicious behavior in endpoints. Using the [Microsoft Defender ATP](https://www.microsoft.com/en-us/security/business/endpoint-security/microsoft-defender-endpoint) collector to send endpoint security alerts to Devo, the analyst will find unauthorized software installations on company devices. As a result, the analyst will isolate compromised endpoints, preventing lateral movement by attackers. ## Example tables | **Table** | **Description** | | --- | --- | | `edr.microsoft_defender.alerts.events` | Security alerts from Microsoft Defender | | `edr.microsoft_defender.endpoint.alerts` | Endpoint security alerts | | `edr.microsoft_defender.endpoint.machines` | Information about monitored endpoints | | `edr.microsoft_defender.endpoint.investigations` | Automated investigation data | | `edr.microsoft_defender.endpoint.assesment_secure_configuration` | Security configuration assessment data | | `edr.microsoft_defender.endpoint.assesment_software_inventory` | Software inventory from endpoints | | `edr.microsoft_defender.endpoint.assesment_software_vulnerabilities` | Software vulnerability assessment data | | `edr.microsoft_defender.endpoint.recommendations` | Security recommendations | | `edr.microsoft_defender.endpoint.software` | Software details from endpoints | | `edr.microsoft_defender.endpoint.vulnerabilities` | Vulnerability details | | `edr.microsoft_defender.advanced_hunting.device_process_events` | Process execution events from devices | ## Authorize it 1. Log in to the [Microsoft Azure portal.](https://azure.microsoft.com/es-es/get-started/azure-portal/) 2. Search for **App registrations **and click on it. 3. Click on **“New registration” **button. 4. Give a name to the application and select the appropriate permissions. 5. Click on **“Register”** button. 6. From the **Overview** section, save **Application (client) ID** as `client_id`. 7. Save **Directory Tenant ID** as `tenant_id`. 8. Click on API permissions on the left-menu. 9. Click on **“Add permissions”**, and select **APIs my organization uses**. 10. Search for *WindowsDefenderATP* and select it. 11. Grant the following** "Read" permissions** for Application permissions: 1. *Alert.Read.All * 2. *File.Read.All* 3. *Ip.Read.All * 4. *Machine.Read.All* 5. *User.Read.All* 6. *Vulnerability.Read.All* 7. *Software.Read.All* 8. *SecurityRecommendation.Read.All* 9. *AdvancedQuery.Read.All* 10. AdvancedHunting.Read.All 11. URL.Read.All 12. Click on **Grant admin consent for your accoun**t. 13. Click on **“Certificates and Secrets”**. 14. Click on **“New client secret”** and name it. 15. Save the secret carefully, as it is going to be shown only once. 16. Go to [Microsoft Defender portal](https://www.microsoft.com/en-us/security/business/microsoft-defender). 17. Open **“Partners and APIs”** on the side panel. 18. Select **Connected applications**, and verify that the registered application is listed. ## Run it In the Cloud Collector App, [create](https://devodocs.atlassian.net/wiki/spaces/latest/pages/409305131) a **Microsoft Defender ATP** collector instance. Insert this parameters template, replacing the values enclosed in `< >`. The `id` must be a unique [five digit number](https://www.random.org/integers/?num=1&min=10000&max=99999&col=5&base=10&format=html&rnd=new). ``` "inputs": { "defender_atp": { "id": "", "services": { "advanced_hunting": { "endpoint": "", "queries": { "devo_tag": "", "query_json": "" }, "credentials": { "client_id": "", "client_secret": "", "tenant_id": "" } } } } } ``` ## Monitor it Create an [inactivity alert](https://devodocs.atlassian.net/wiki/spaces/latest/pages/95126785) to detect interruptions of transfer of data from the source using the query ``` from edr.microsoft_defender.endpoint.alerts where toktains(hostchain,"collector-") select split(hostchain,"-",1) as collector_id ``` Set the inactivity alert to keep track of the `collector_id`. ## Parameters ### Example ``` { "global_overrides": { "debug": }, "inputs": { "defender_atp": { "id": "", "enabled": true, "requests_per_second": , "override_api_base_url": "", "override_token_url": "", "credentials": { "client_id": "", "client_secret": "", "tenant_id": "" }, "services": { "alerts": { "request_period_in_seconds": , "historical_poll_datetime": "", "flatten_alert_related_evidences": , "request_alert_related_domains": , "request_alert_related_files": , "request_alert_related_ips": , "request_alert_related_devices": , "request_alert_related_users": }, "machines": { "request_period_in_seconds": , # minimum value 3600 for this service "historical_poll_datetime": "", "request_machine_logon_users": , "request_machine_related_alerts": , "request_machine_installed_software": , "request_machine_vulnerabilities": , "request_machine_security_recommendations": }, "software": { "request_period_in_seconds": , "request_software_version_distributions": , "request_machines_by_software": , "request_vulnerabilities_by_software": , "request_software_missing_kbs": }, "vulnerabilities": { "request_period_in_seconds": , "historical_poll_datetime": "", "request_machine_by_vulnerabilities": }, "recommendations": { "request_period_in_seconds": , "request_recommendations_by_software": , "request_machines_by_recommendations": , "request_vulnerability_by_recommendations": }, "investigations": { "request_period_in_seconds": , "historical_poll_datetime": "" }, "assessments": { "request_period_in_seconds": "", "include_software_vulnerabilities_by_machine": "", "historical_poll_datetime": "" }, "advanced_hunting": { "request_period_in_seconds": , "endpoint": "", "queries": [ { "devo_tag": "", "query_json": "" } ] } } } } } ``` ### Details | **Parameter** | **Data type** | **Necessity** | **Details** | | --- | --- | --- | --- | | `debug_status` | `boolean` | `Mandatory` | If the value is `true`, the debug logging traces will be enabled when running the collector. If the value is `false`, only the `info`, `warning` and `error` logging levels will be printed. | | `id` | `string` | `Mandatory` | **Unique** id to this input service. | | `requests_per_second` | `integer` | `Optional` | Customize the maximum number of API requests per second. If not used, the default setting will be used
**Default:** `60` requests/sec.
This parameter should be removed if it is not used. | | `override_api_base_url` | `string` | `Optional` | Use this param to override the default base URL used by the collector to pull data.
**Default:** `https://api.securitycenter.microsoft.com`
URL format: `(^https:\/\/)([\da-z\.-]+)\.([a-z\.])([\/\w \.-]*)*([a-z])(:\d{1,5})?$` | | `override_token_url` | `string` | `Optional` | Use this param to override the default URL used by the collector to get the authentication token.
**Default:** `https://login.microsoftonline.com`
This parameter should be removed if it is not used. | | `client_id` | `string` | `Mandatory` | Set up here your Application (client) ID of the registered application in the Azure console. | | `client_secret` | `string` | `Mandatory` | Set up here your Client Secret created in the Azure console. | | `tenant_id` | `string` | `Mandatory` | Set up here your Tenant ID of the registered application in the Azure console. | | `request_period_in_seconds` | `integer` | `Optional` | Period in seconds used between each data pulling, this value will overwrite the default value.
**Default:** `300`
This parameter should be removed if it is not used. | | `historical_poll_datetime` | `string` | `Optional` | This configuration allows you to set a custom date as the beginning of the period to download. This allows downloading historical data (1 month back for example) before downloading new events.
If not used, it will start collecting data since 10 days ago.
Note: for `assessments` service if `include_software_vulnerabilities_by_machine `parameter is `true` `historical_poll_datetime_value` can’t be more than 14 days in the past.
Format: `YYYY-mm-ddTHH:MM:SS.sssZ` | | `request_enrichment` | `boolean` | `Optional` | Use these parameters to config the enrichment behavior. You can disable entities that are not interesting to you and speed up the ingestion. When the ingestion is not selected, the field will be displayed as `Null` in Loxcope.
**Default:** `true`
This parameter should be removed if it is not used. | | `flatten_evidences` | `boolean` | `Optional` | Use this parameter to do not flatten the alert evidences. This data will be ingested into Devo in the `evidences` field as raw content. This field is populated only when when the data is not flattened.
**Default:** `true`
This parameter should be removed if it is not used. | | `endpoint` | `string` | `Mandatory` only for Advanced Hunting service. | Use this parameter if you are going to use Advanced Hunting service. It completes the endpoint to which the query is going to be performed. A usual value for this is `/api/advancedqueries/run`.
URL format: `(^https:\/\/)([\da-z\.-]+)\.([a-z\.])([\/\w \.-]*)*([a-z])(:\d{1,5})?$` | | `devo_tag` | `string` | `Mandatory` only for Advanced Hunting service. | Use this parameter if you are going to use Advanced Hunting service. It is the tag bound to the data collected using Advanced Hunting. Format: `my.app.{table_name}` | | `query_json` | `string` | `Mandatory` only for Advanced Hunting service. | Use this parameter if you are going to use Advanced Hunting service. It is the query that is going to be requested to the Microsoft Defender API. **Note**: Tables which can be included in the query except the “Graph” Tables: [https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-schema-tables#learn-the-schema-tables](https://learn.microsoft.com/en-us/defender-xdr/advanced-hunting-schema-tables#learn-the-schema-tables) Sample query example: `query_json: "{\"Query\": \"DeviceFileEvents | project Timestamp, DeviceId, DeviceName, FileName, FolderPath, InitiatingProcessFileName | limit 1000\"}"` **Note**: Data is filtered , for all the Tables, using `Timestamp` field it will start filtering the data adding `Timestamp`> `historical_poll_datetime` in the query. Also, we can search upto 30 days of security data. | | `include_software_vulnerabilities_by_machine` | `boolean` | `Optional` | If this parameter to get the data from this endpoint `https://api.securitycenter.microsoft.com/SoftwareVulnerabilityChangesByMachine` to get the software vulnerabilities assessment per device data. If assigned **true** then it is **mandatory** to define `historical_poll_datetime_value`.
**Default:** `false` | ## API Endpoints | **Endpoint** | **Service** | **Table** | | --- | --- | --- | | - List alerts:
`https://api.securitycenter.microsoft.com/api/alerts`
- Alert details:
`https://api.securitycenter.microsoft.com/api/alerts/{{alert_id}}`
- Alerts related to other `{{topic}}:`
`https://api.securitycenter.microsoft.com/api/alerts/{{alert_id}}/{{topic}}` | `alerts` | `edr.microsoft_defender.endpoint.alerts`
The events ingested by this table are **Microsoft Defender Identity Events**. | | - List vulnerabilities:
`https://api.securitycenter.microsoft.com/api/vulnerabilities`
- Vulnerability details:
`https://api.securitycenter.microsoft.com/api/vulnerabilities/{{vulnerability_id}}`
- Alerts related to machine:
`https://api.securitycenter.microsoft.com/api/vulnerabilities/{{vulnerability_id}}/machineReferences` | `vulnerabiities` | `edr.microsoft_defender.endpoint.vulnerabilities` | | - List machines:
`https://api.securitycenter.microsoft.com/api/machines`
- Machine details:
`https://api.securitycenter.microsoft.com/api/machines/{{machine_id}}`
- Machines related to other `{{topic}}`:
`https://api.securitycenter.microsoft.com/api/machines/{{machine_id}}/{{topic}}` | `machines` | `edr.microsoft_defender.endpoint.machines` | | - List software:
`https://api.securitycenter.microsoft.com/api/software`
- Software details:
`https://api.securitycenter.microsoft.com/api/software/{{software_id}}`
- Software related to other `{{topic}}`:
`https://api.securitycenter.microsoft.com/api/software/{{software_id}}/{{topic}}` | `softwares` | `edr.microsoft_defender.endpoint.software` | | - List recommendations:
`https://api.securitycenter.microsoft.com/api/recommendations`
- Recommendation details:
`https://api.securitycenter.microsoft.com/api/recommendations/{{recommendation_id}}`
- Recommendations related to other `{{topic}}`:
`https://api.securitycenter.microsoft.com/api/recommendations/{{recommendation_id}}/{{topic}}` | `recommendations` | `edr.microsoft_defender.endpoint.recommendations` | | `https://api.securitycenter.microsoft.com/api/investigations` | `investigations` | `edr.microsoft_defender.endpoint.investigations` | | `https://api.securitycenter.microsoft.com/SecureConfigurationsAssessmentExport`
`https://api.securitycenter.microsoft.com/SoftwareInventoryExport`
`https://api.securitycenter.microsoft.com//SoftwareVulnerabilitiesExport` | `assessments` | `edr.microsoft_defender.endpoint.assessment_secure_configuration`
`edr.microsoft_defender.endpoint.assessment_software_inventory`
`edr.microsoft_defender.endpoint.assessment_software_vulnerabilities` | ## Flattening preprocessing In order to improve the data exploitation and enrichment, this collector is able to apply some flattering actions to the collected data before delivering it to Devo. ### Flattening over objects When flattening over objects, it creates new keys whose names are the combination of the external/internal keys. | **Original structure** | **Result** | | --- | --- | | ``` { 'machine': { 'hostname': 'machine1', 'os': 'linux', 'ip': '1.2.3.4' } } ``` | ``` { 'machine_hostname': 'machine1', 'machine_os': 'linux', 'machine_ip': '1.2.3.4' } ``` | ### Flattening over arrays When flattening over arrays it replicates the event using one element of the array in each replicated event: Example: Take into account that this flattening method generates more events than those originally collected. In the example below 1 single event, 3 events are generated. In the case of an event with multiple fields that contains an array, the number of events generated per original event is obtained by multiplying the number of elements in each array. For instance, given an event with 3 fields that contains arrays with 2, 3 and 4 elements each, if we flatten this event, were going to generate 24 events from this single event (2*3*4 = 24). | **Original structure** | **Result** | | --- | --- | | ``` { 'machine_hostname': 'machine1', 'machine_os': 'linux', 'ips':['1.2.3.4', '4.3.2.1', '1.1.1.1'] } # Flattened data structure (3 events generated from the original one) ``` | ``` [ { 'machine_hostname': 'machine1', 'machine_os': 'linux', 'ip':'1.2.3.4' }, { 'machine_hostname': 'machine1', 'machine_os': 'linux', 'ip':'4.3.2.1' }, { 'machine_hostname': 'machine1', 'machine_os': 'linux', 'ip':'1.1.1.1' } ] ``` | ## Devo collector features | **Feature** | **Details** | | --- | --- | | Allow parallel downloading (`multipod`) | - `Not allowed` | | Running environments | - `Collector server` | | Populated Devo events | - `Table` | | Flattening preprocessing | - `Yes (optional)` | ## Release Notes - **Version 3.1.0** (2025-09-01) - **Fixed** - Fixed the issue with GCC high environment. - Fixed missing issue with alerts service - **Version 3.0.0** (2025-08-20) - **Improvements** - Added a migration guide because of persistence incompatibility. - Fixed the issue with advance hunting service. - **Version 2.3.0** (2025-08-07) - **Improvement** - Refactored the code to use the SDK `Template1CollectorPuller` - Refactored all the puller logics. - Updated the persistence logic, added new param in the persistence to avoid duplicates - Added unit-tests - updated docker base image to 1.6.0 - **Version 2.2.1** (2025-06-20) - Fixed the issue related to 429 error for services - **Version 2.2.0** (2025-06-18) - Upgraded DCSDK `1.15.0` to `1.16.2` - Upgraded docker base image to `1.5.1`. - Fixed the issue related to the assessments service caused by looping. - **Version 2.1.0 **(2025-04-10) - Improved the pull logic of the alerts service. - Fixed the bug of ``api/alerts/[id]/user`` endpoint. - **Version 2.0.0** (2025-03-21) - Added an new endpoint for the service assessments - Updated Docker image to 1.4.1 - Updated `DevoCollectorSDK` from `v1.12.4` to `v1.15.0` - **Version 1.4.0** (2024-09-18) - Updated `DevoCollectorSDK` from `v1.11.1` to `v1.12.4` - Updated the docker base image to 1.3.0 - **Version 1.1.1** (2023-12-04) - Devo Collector SDK upgraded from `1.8.0` to `1.10.2`. - Fix a use case that allowed sending duplicate events to Devo for the Alerts service, as the last IDs sent were deleted before required. - Fix a use case that avoided sending some events to Devo for the Alerts service, as the time of the latest sent event wasnt appropriately persisted. - **Version 1.0.0** (2022-09-20) - Alerts created in MS Defender are collected and enriched. The enrichment includes information related to evidences, domains, files, IPs, devices and users. - Machine information enriched with information related to logged on users, alerts, installed software, vulnerabilities and security recommendations. - Snapshot of software installed and enriched with data related to version distributions, machine, vulnerabilities and missing software updates. - Vulnerabilities information enriched with machines affected by the vulnerability. - Recommendations information snapshot, including information related to specific software, machines and vulnerabilities. - Investigations performed. - Advanced hunting, which allows making custom queries in Kusto Query Language.