- The Devo data analytics platform
- Getting started
- Domain administration
-
Sending data to Devo
-
The Devo In-House Relay
- Installing the Devo Relay
- Configuring the In-House Relay
- Relay migration
- Sending SSL/TLS encrypted events to the Devo relay
- Relay troubleshooting tips (v1.4.2)
-
Event sources
- Unix-like machines
- Windows
- MacOS X
- Cloud services
- Commercial products
- Custom apps
-
Universal Agent
- Pre-integrated query packs
- Data querying in Devo
-
Universal Agent Manager deployment
- Universal Agent Manager - CentOS 7 Deployment
- Universal Agent Manager - CentOS 8 Deployment
- Universal Agent Manager - Debian 9 Deployment
- Universal Agent Manager - Debian 10 Deployment
- Universal Agent Manager - RHEL 7 Deployment
- Universal Agent Manager - RHEL 8 Deployment
- Universal Agent Manager - Ubuntu 18 Deployment
- Universal Agent deployment
- Performance considerations
- Other data collection methods
- Uploading log files
- Devo software
-
The Devo In-House Relay
-
Parsers and collectors
- About Devo tags
- Special Devo tags and data tables
-
List of Devo parsers
- Business & Consumer
- Cloud technologies
- Databases
- Host and Operating Systems
-
Network and application security
- auth.secureauth
- auth.securenvoy
- av.mcafee
- av.sophos
- box.iptables
- edr.cylance
- edr.fireeye.alerts
- edr.minervalabs.events
- edr.paloalto
- endpoint.symantec
- firewall.checkpoint
- firewall.cisco firepower and vpn.cisco
- firewall.fortinet
- firewall.huawei
- firewall.juniper
- firewall.paloalto
- firewall.pfsense
- firewall.sonicwall
- firewall.sophos
- firewall.sophos.xgfirewall
- firewall.stonegate
- firewall.windows
- ids.extrahop
- mail.proofpoint
- nac.aruba
- network.meraki
- network.versa
- network.vmware
- proxy.bluecoat
- proxy.forcepoint
- proxy.squid
- uba.varonis
- vuln.beyondtrust
- vpn.pulsesecure.sa
- Network connectivity
- Web servers
- Technologies supported in CEF syslog format
- Collectors
-
Searching data
- Accessing data tables
-
Building a query
- Data types in Devo
- Build a query in the search window
- Build a query using LINQ
- Working with JSON objects in data tables
- Subqueries
-
Operations reference
-
Aggregation operations
- Average (avg)
- Count (count)
- First (first)
- First not null (nnfirst)
- HyperLogLog++ (hllpp)
- HyperLogLog++ Count Estimation (hllppcount)
- Last (last)
- Last not null (nnlast)
- Maximum (max)
- Median / 2nd quartile / Percentile 50 (median)
- Minimum (min)
- Non-null average (nnavg)
- Non-null standard deviation (biased) (nnstddev)
- Non-null standard deviation (unbiased) (nnustddev)
- Non-null variance (biased) (nnvar)
- Non-null variance (unbiased) (nnuvar)
- Percentile 10 (percentile10)
- Percentile 25 / 1st quartile (percentile25)
- Percentile 5 (percentile5)
- Percentile 75 / 3rd quartile (percentile75)
- Percentile 90 (percentile90)
- Percentile 95 (percentile95)
- Standard deviation (biased) (stddev)
- Standard deviation (unbiased) (ustddev)
- Sum (sum)
- Sum Square (sum2)
- Variance (biased) (var)
- Variance (unbiased) (uvar)
-
Arithmetic group
- Absolute value (abs)
- Addition, sum, plus / Concatenation (add, +)
- Ceiling (ceil)
- Cube root (cbrt)
- Division (div, \)
- Division remainder (rem, %)
- Floor (floor)
- Modulo (mod, %%)
- Multiplication, product (mul, *)
- Power (pow)
- Real division (rdiv, /)
- Rounding (round)
- Sign (signum)
- Square root (sqrt)
- Subtraction, minus / Additive inverse (sub, -)
-
Conversion group
- Duration (duration)
- Format date (formatdate)
- From base16, b16, hex (from16)
- From base64, b64 (from64)
- From UTF8 (fromutf8)
- From Z85, base85 (fromz85)
- Human size (humanSize)
- Make byte array (mkboxar)
- Parse date (parsedate)
- Regular expression, regexp (re)
- Template (template)
- Timestamp (timestamp)
- To base16, b16, hex (to16)
- To base64, b64, hex (to64)
- To BigInt (bigint)
- To boolean (bool)
- To Float (float)
- To image (image)
- To Int (int)
- To IPv4 (ip4)
- To IPv4 net (net4)
- To IPv6 (ip6)
- To IPv6 compatible (compatible)
- To IPv6 mapped (mapped)
- To IPv6 net (net6)
- To IPv6 translated (translated)
- To MAC address (mac)
- To string (str)
- To string (stringify)
- To UTF8 (toutf8)
- To Z85, base85 (toz85)
- Cryptography group
- Date group
- Flow group
- General group
-
Geolocation group
- Coordinates distance (distance)
- Geocoord (geocoord)
- Geographic coordinate system (coordsystem)
- Geohash (geohash)
- Geohash string (geohashstr)
- Geolocated Accuracy Radius with MaxMind GeoIP2 (mm2accuracyradius)
- Geolocated ASN (mmasn)
- Geolocated ASN with MaxMind GeoIP2 (mm2asn)
- Geolocated AS Organization Name with MaxMind GeoIP2 (mm2asorg)
- Geolocated AS owner (mmasowner)
- Geolocated City (mmcity)
- Geolocated City with MaxMind GeoIP2 (mm2city)
- Geolocated Connection Speed (mmspeed)
- Geolocated connection type with MaxMind GeoIP2 (mm2con)
- Geolocated Coordinates (mmcoordinates)
- Geolocated coordinates with MaxMind GeoIP2 (mm2coordinates)
- Geolocated Country (mmcountry)
- Geolocated Country with MaxMind GeoIP2 (mm2country)
- Geolocated ISP (mmisp)
- Geolocated ISP name with MaxMind GeoIP2 (mm2isp)
- Geolocated Latitude (mmlatitude)
- Geolocated Latitude with MaxMind GeoIP2 (mm2latitude)
- Geolocated Level 1 Subdivision with MaxMind GeoIP2 (mm2subdivision1)
- Geolocated Level 2 Subdivision with MaxMind GeoIP2 (mm2subdivision2)
- Geolocated Longitude (mmlongitude)
- Geolocated Longitude with MaxMind GeoIP2 (mm2longitude)
- Geolocated Organization (mmorg)
- Geolocated organization name with MaxMind GeoIP2 (mm2org)
- Geolocated Postal Code (mmpostalcode)
- Geolocated Postal Code with MaxMind GeoIP2 (mm2postalcode)
- Geolocated Region (mmregion)
- Geolocated Region Name (mmregionname)
- ISO-3166-1 Continent Alpha-2 Code (continentalpha2)
- ISO-3166-1 Continent Name (continentname)
- ISO-3166-1 Country Alpha-2 Code (countryalpha2)
- ISO-3166-1 Country Alpha-2 Continent (countrycontinent)
- ISO-3166-1 Country Alpha-3 Code (countryalpha3)
- ISO-3166-1 Country Latitude (countrylatitude)
- ISO-3166-1 Country Longitude (countrylongitude)
- ISO-3166-1 Country Name (countryname)
- Latitude (latitude)
- Latitude and longitude coordinates (latlon)
- Longitude (longitude)
- Parse geocoord format (parsegeo)
- Represent geocoord format (reprgeo)
- Round coordinates (gridlatlon)
- JSON group
- Logic group
-
Mathematical group
- Arc cosine (acos)
- Arc sine (asin)
- Arc tangent (atan)
- Bitwise AND (band, &)
- Bitwise left shift (lshift, <<)
- Bitwise NOT (bnot, ~)
- Bitwise OR (bor, |)
- Bitwise right shift (rshift, >>)
- Bitwise unsigned right shift (urshift, >>>)
- Bitwise XOR (bxor, ^)
- Cosine (cos)
- e (mathematical constant) (e)
- Exponential: base e (exp)
- Hyperbolic cosine (cosh)
- Hyperbolic sine (sinh)
- Hyperbolic tangent (tanh)
- Logarithm: base 2 (log2)
- Logarithm: base 10 (log10)
- Logarithm: natural / arbitrary base (log)
- Pi (mathematical constant) (pi)
- Sine (sin)
- Tangent (tan)
- Meta Analysis group
- Name group
-
Network group
- HTTP Status Description (httpstatusdescription)
- HTTP Status Type (httpstatustype)
- IP Protocol (ipprotocol)
- IP Reputation Score (reputationscore)
- IP Reputation Tags (reputation)
- IPv4 legal use (purpose)
- IPv6 host number (host)
- IPv6 routing number (routing)
- Is IPv4 (ipip4)
- Is Private IPv4 (isprivate)
- Is Public IPv4 (ispublic)
- Squid Black Lists Flags (sbl)
- Order group
-
Packet group
- Ethernet destination MAC address (etherdst)
- Ethernet payload (etherpayload)
- Ethernet source MAC address (ethersrc)
- Ethernet status (etherstatus)
- Ethernet tag (ethertag)
- EtherType (ethertype)
- Has Ethernet frame (hasether)
- Has IPv4 datagram (hasip4)
- Has TCP segment (hastcp)
- Has UDP datagram (hasudp)
- IPv4 destination address (ip4dst)
- IPv4 differentiated services (ip4ds)
- IPv4 explicit congestion notification (ip4ecn)
- IPv4 flags (ip4flags)
- IPv4 fragment offset (ip4fragment)
- IPv4 header checksum (ip4cs)
- IPv4 header length (ip4hl)
- IPv4 identification (ip4ident)
- IPv4 payload (ip4payload)
- IPv4 protocol (ip4proto)
- IPv4 source address (ip4src)
- IPv4 status (ip4status)
- IPv4 time to live (ip4ttl)
- IPv4 total length (ip4len)
- IPv4 type of service (ip4tos)
- TCP ACK (tcpack)
- TCP checksum (tcpcs)
- TCP destination port (tcpdst)
- TCP flags (tcpflags)
- TCP header length (tcphl)
- TCP payload (tcppayload)
- TCP sequence number (tcpseq)
- TCP source port (tcpsrc)
- TCP status (tcpstatus)
- TCP urgent pointer (tcpurg)
- TCP window size (tcpwin)
- UDP checksum (udpcs)
- UDP destination port (udpdst)
- UDP length (udplen)
- UDP payload (udppayload)
- UDP source port (udpsrc)
- UDP status (udpstatus)
- Statistical group
-
String group
- Contains (has, ->)
- Contains - case insensitive (weakhas)
- Contains tokens (toktains)
- Contains tokens - case insensitive (weaktoktains)
- Edit distance: Damerau (damerau)
- Edit distance: Hamming (hamming)
- Edit distance: Levenshtein (levenshtein)
- Edit distance: OSA (osa)
- Ends with (endswith)
- Format number (formatnumber)
- Hostname public suffix (publicsuffix)
- Hostname root domain (rootdomain)
- Hostname root prefix (rootprefix)
- Hostname root suffix (rootsuffix)
- Hostname subdomains (subdomain)
- Hostname top level domain (topleveldomain)
- Is empty (isempty)
- Is in (`in`, <-)
- Is in - case insensitive (weakin)
- Length (length)
- Locate (locate)
- Lower case (lower)
- Matches (matches, ~)
- Peek (peek)
- Replace all (replaceall)
- Replace first (replace)
- Shannon entropy (shannonentropy)
- Split (split)
- Split regexp (splitre)
- Starts with (startswith)
- Substitute (subs)
- Substitute all (subsall)
- Substring (substring)
- Trim both sides (trim)
- Trim the left side (ltrim)
- Trim the right side (rtrim)
- Upper case (upper)
-
Web group
- Absolute URI (absoluteuri)
- Opaque URI (opaqueuri)
- URI authority (uriauthority)
- URI fragment (urifragment)
- URI host (urihost)
- URI path (uripath)
- URI port (uriport)
- URI query (uriquery)
- URI scheme (urischeme)
- URI ssp (urissp)
- URI user (uriuser)
- URL decode (urldecode)
- User Agent Company (uacompany)
- User Agent Company URL (uacompanyurl)
- User Agent Device Icon (uadeviceicon)
- User Agent Device Information URL (uadeviceinfourl)
- User Agent Device Type (uadevicetype)
- User Agent Family (uafamily)
- User Agent Icon (uaicon)
- User Agent Information URL (uainfourl)
- User Agent is Robot (uaisrobot)
- User Agent Name (uaname)
- User Agent OS Company (uaoscompany)
- User Agent OS Company URL (uaoscompanyurl)
- User Agent OS Family (uaosfamily)
- User Agent OS Icon (uaosicon)
- User Agent OS Name (uaosname)
- User Agent OS URL (uaosurl)
- User Agent Type (uatype)
- User Agent URL (uaurl)
- User Agent Version (uaversion)
-
Aggregation operations
-
Working in the search window
-
Generate charts
- Affinity chord diagram
- Availability timeline
- Bipartite chord diagram
- Bubble chart
- Chart aggregation
- Custom date chart aggregation
- Flame graph
- Flat world map by coordinates
- Flat world map by country
- Google animated heat map
- Google area map
- Google heat map
- Graph diagram
- Histogram
- Pew Pew map
- Pie chart
- Pie layered chart
- Punch card
- Robust Random Cut Forest chart
- Sankey diagram
- Scatter plot
- Time heatmap
- Triple exponential chart
- Voronoi treemap
- Data enrichment
- Setting up a data table
- Advanced data operations
- Use case: eCommerce behavior analysis
-
Generate charts
- Managing your queries
- Best practices for data search
- Monitoring tables
- Activeboards
-
Dashboards
- Create a new dashboard
-
Working with dashboard widgets
- Availability timeline widget
- Chord diagram widget
- Circle world map widget
- Color key value widget
- Color world map widget
- Column chart widget
- Comparative chart widget
- Funnel widget
- Gauge meter widget
- Google heatmap widget
- Heat calendar widget
- Line chart widget
- Monitoring widget
- Pie chart widget
- Punch card widget
- Sectored pie chart widget
- Table widget
- Time heatmap widget
- Tree diagram widget
- Voronoi tree widget
- Configuring and sharing dashboards
- Alerts and notifications
- Panels
- Applications
- Tools
- Flow
- Social Intelligence
- API reference
- Release notes
Configuring Security Insights
During the installation of Security Insights or after it's up and running, there are a few things you can do to customize the application for your domain.
Tune alert priorities
Each of the security alerts has been assigned a default priority. While our security experts have carefully considered the priorities for each alert, you can customize the priorities if you choose.
The valid priority values are:
Priority | Description |
---|---|
1 | Very High |
2 | High |
3 | Medium |
4 | Low |
5 | Info |
To do so, you’ll need to edit the AlertDescriptionLocal.csv lookup file or upload it if it doesn't yet exist in the Devo domain where Security Insights is installed. Simply add one line for each alert for which you want to change the priority. Although there are other fields, to change the priority of an existing alert you only need to specify the alertName and the priority values in this file.
This lookup file is also used when you want to create custom security alerts so the alertCategory, alertTemplate, alertDescription and alertFields fields are necessary in that case. In the case of changing the priority of an existing alert, you can leave these undefined.
Click here to download this example file and use it as a template. Once you have edited and saved this file, follow the instructions for uploading a lookup table. Be sure to set the data type for alertCategory and priority to integer and the rest to alphanumeric.
If the lookup file already exists, follow the instructions for editing an uploaded lookup. If the alert already appears in the existing lookup, you only need to edit the priority value. If it doesn’t appear in the lookup, add a new row where you enter the alertName and the new priority.
Create custom security alerts
Define the alert
While our security alerts address a wide range of security events, you can create your own alerts to meet the specific needs of your environment.
First, create the new alert as you would with any other alert with just two additional requirements:
- The Alert Name must start with SecInt. For example, SecIntMyCustomAlert.
- The Subcategory must be threats.
Optionally, if you want to use your custom alert to create an alert chain, you must include the fields you want to use to join it with the rest of alerts of the chain in the Message. Add them at the end of the message, preceded by ||. For example:
IP $sourceIP is creating connections to multiple IP addresses in the last 5 minutes||sourceIP;destIP
The fields added to use as joins for an alert chain must be named in a specific way in order to match the names of the fields in the default alerts defined by Devo.
This table lists the names accepted by Devo for the following default alert fields:
Value in Devo default alerts | Values allowed |
---|---|
SOURCE_IP | sourceIP, sourceip, srcip |
DESTINATION_IP | destinationIP, destinationip, destip, dstip |
SOURCE_PORT | sourcePort, sourceport, srcport |
DESTINATION_PORT | destinationPort, destinationport, destport, dstport |
HOST | host, hostname |
USER | user, username, userfield |
In the query below, we have changed the names of two fields (srcPort and dstPort) and given them one of the accepted names:
|
---|
Fields containing IP addresses in ip data type must be converted to strings if you want to use them in a custom alert. The ip format is valid during the query creation, but you must change it to string after grouping your data, using the To string (str) operation. For example, see the following query
from firewall.all.traffic
where ispublic(dstIp)
select `lu/Threat-Fraud-by-IP/threat`(dstIp) as ThreatFraud
where isnotnull(ThreatFraud)
group every 5m by srcIp, dstIp, ThreatFraud, dstPort
every 5m
select count() as count
where
not ThreatFraud = "fraud",
not ThreatFraud = ""
select str(srcIp) as sourceIP
select str(dstIp) as destinationIP
select ThreatFraud as Threat
where count > 10
Add the alert to Security Insights
Now you need to add the alert to Security Insights by including it in the AlertDescriptionLocal lookup file.
- If this lookup already exists, simply edit the lookup and add a new line for the alert. To see the list of lookups, go to Data Search and open the Lookup Management tab.
- If it doesn’t exist, you can download our template and enter the values that describe your new alert. Then when you’ve saved the CSV file containing your new alert, upload it to your Devo domain.
- It is important to set the correct field types when uploading the template as a lookup file. Devo is able to identify the template as a CSV file and separate the fields properly, but you must change the default value Integer number to Alphanumeric, as shown in the picture:
Here’s an example of a hypothetical custom alert called SecIntMyCustomAlert. The 6 fields you can see in the template file are mandatory to set. Priority and alertCategory are integer, the rest are alphanumeric. In alertFields you need to set the fields you added to the grouping to make them appear in the Security Insights user interface.
|
---|
The possible values for the alertCategory field are:
alertCategory | Description |
---|---|
2 | Network: alerts from firewall, web, proxy. |
3 | Threats: alerts from firewall, web or proxy considered as a threat. |
4 | Access: alerts from auth.all, box.win, or referred to user authentication. |
6 | Alert Chain: alerts from siem.alert.info (alerts over alerts, which are a combination of two or more alerts) |
7 | Availability: low alerts or not availability alerts. |
Create an alert chain
You can combine two or more of the alerts defined for the application to create an alert chain. An alert chain combines the criteria of multiple selected alerts. An alert chain is triggered when its component alerts all meet the established criteria. The queries that define the alerts in the chain are joined using a common field; that is to say, a field that is present in all of them. For example, you might create an alert chain to notify you when the same source IP address has triggered two separate alert types within a specified period of time.
See the instructions in the previous section to learn how to define custom alerts to be used to create an alert chain.
To create an alert chain:
- Click the lightning icon in the top right.
- You will see a list of all the alerts that are defined for the application. Select New alert chain to start the process.
- In step 1, you select the alerts you want to combine. Use the search to find specific alerts by the contents of their name or description. Click Next step when you're done.
In step 2, you set the join field, time window, alert chain details. These are the fields to complete:
Join field Select the common table field you want to use to join the alerts. If the alerts selected have no fields in common, the join field will be eventdate, which is present in every single data table. Time window Set the threshold of the alert chain. For example, choose 1h to check if the conditions of all the alerts added to the chain are met every 60 minutes. Priority Choose a priority for the alert chain between Info, Low, Medium, High and Critical. Alert name Enter a name for the alert chain. Alert chain names must always start with SecIntChain. Alert message A short message used to identify the alert condition. Alert description The full description of the alert chain conditions. You may use variables in this field. To reference a field value, write it preceded by a dollar sign, like this: $sourceIP. Click Next step when you're done.
- Step 3 is simply a summary of the alert chain. Review and confirm the alert chain definition. You can select Show query to check the definition of the alert. Click Create alert to confirm the alert chain and create it.
Close the dialog and open it again to see the newly defined alert chain. You can create a new alert chain using this newly alert—there is no limitation in creating new alert chains using other alert chains or common alerts.
Now you can go to Administration → Alerts Configuration and locate the new alert chain. Alert chains are automatically activated after you create them. However, note that they can be disabled, but not edited nor deleted. Following versions of the Security Insights application will make these options available.
Learn more about alerts and how to configure them reading the articles in the Configuring alerts section.
Create an IP address white list
You might have a list of IP addresses that generate activity on your network but that you know for sure do not pose threats. In this case, you can create a white list so that any alerts generated by events involving these IPs as the Source ID are immediately disregarded.
To implement the white list, we create a post-processing filter in the Alerts area of the web application. These filters must be created from a triggered alert that appears in your Alert Dashboard, and will affect all the occurrences of that alert definition.
For instance, we have an alert that detects port scan attacks (SecIntTestPortScan) and we want to discard the alerts that are triggered by two specific IPs: 77.72.85.17 and 188.22.33.44. These two IP addresses are causing a lot of noise and belong to internal servers so we want to discard them.
We must filter out these IPs and add them to a white list as Watched. To do it, we must click one instance of this alert in the Alert Dashboard and select New Filter.
When defining the filter:
- Add conditions only in the Extra Data field, not in Basic Data (only for Devo predefined alerts).
- Select srcIpStr under Extra Data and set it to the IP address you want on the white list (in this case 77.72.85.17). Click the + icon to add it. Note that you can set only one condition per filter.
- Once the condition is added, select the Action to be applied when the condition fits. We will select Mark as read to watch the alerts meeting the condition as Watched. Click Save to end the process.
In the window that appears, click Add to enter the second IP to the filter (188.22.33.44). Repeat the steps above and click Save again.
Once saved, the filter will set to Watched all the triggered alerts where the Source IP is one of the IPs specified in the filter. In the following image, you can see the effect of the sample filters. Two SecIntTestPortScan alerts are automatically set to Watched in the Alerts Dashboard because the IP matches with the filter. The other alert has a different IP so it is triggered normally.
You can apply other filters as required and the status of the alert will be updated in the Alert Dashboard and in the siem.logtrust.alert.info
table. If you select the Delete action, the alert will obviously not appear.
Each action has a different status code that you can check in the status column of the siem.logtrust.alert.info
table. In the following example, we have changed the alert state from New (code 4) to False Positive (code 2).
To modify the white list, it’s necessary to create a new filter with the updated list of Source IP addresses, then delete the existing filter. Post-filters cannot be edited.
Disable unused tabs (or re-enable them)
By default, the application contains nine different tabs; Overview, Threats, Network, DNS, Firewall, Proxy, Access, Web and IDS but Admin users can disable (or hide) any tabs that are not currently used in their deployment. For example, if your domain doesn’t have IDS data sources, it makes sense to disable the IDS tab.
To do this, we again use a special lookup file called Tabs.csv. This file has just two fields; tab is set to the tab name and enabled is set to either true or false (in lowercase).
|
---|
Simply set a tab to false to hide it in the application. You can edit your lookup at any time to enable or disable tabs. Download the template that you can modify, then follow the instructions to upload it to your domain.
Note that this custom lookup must be called Tabs.