- The Devo data analytics platform
- Getting started
- Domain administration
-
Sending data to Devo
-
The Devo In-House Relay
- Installing the Devo Relay
- Configuring the In-House Relay
- Relay migration
- Sending SSL/TLS encrypted events to the Devo relay
- Relay troubleshooting tips (v1.4.2)
- Event sources
- Other data collection methods
- Uploading log files
- Devo software
-
The Devo In-House Relay
-
Searching data
- Accessing data tables
-
Building a query
- Data types in Devo
- Build a query in the search window
- Build a query using LINQ
- Working with JSON objects in data tables
- Subqueries
-
Operations reference
-
Aggregation operations
- Average (avg)
- Count (count)
- First (first)
- First not null (nnfirst)
- HyperLogLog++ (hllpp)
- HyperLogLog++ Count Estimation (hllppcount)
- Last (last)
- Last not null (nnlast)
- Maximum (max)
- Median / 2nd quartile / Percentile 50 (median)
- Minimum (min)
- Non-null average (nnavg)
- Non-null standard deviation (biased) (nnstddev)
- Non-null standard deviation (unbiased) (nnustddev)
- Non-null variance (biased) (nnvar)
- Non-null variance (unbiased) (nnuvar)
- Percentile 10 (percentile10)
- Percentile 25 / 1st quartile (percentile25)
- Percentile 5 (percentile5)
- Percentile 75 / 3rd quartile (percentile75)
- Percentile 90 (percentile90)
- Percentile 95 (percentile95)
- Standard deviation (biased) (stddev)
- Standard deviation (unbiased) (ustddev)
- Sum (sum)
- Sum Square (sum2)
- Variance (biased) (var)
- Variance (unbiased) (uvar)
-
Arithmetic group
- Absolute value (abs)
- Addition, sum, plus / Concatenation (add, +)
- Ceiling (ceil)
- Cube root (cbrt)
- Division (div, \)
- Division remainder (rem, %)
- Floor (floor)
- Modulo (mod, %%)
- Multiplication, product (mul, *)
- Power (pow)
- Real division (rdiv, /)
- Rounding (round)
- Sign (signum)
- Square root (sqrt)
- Subtraction, minus / Additive inverse (sub, -)
-
Conversion group
- Duration (duration)
- Format date (formatdate)
- From base16, b16, hex (from16)
- From base64, b64 (from64)
- From UTF8 (fromutf8)
- From Z85, base85 (fromz85)
- Human size (humanSize)
- Make byte array (mkboxar)
- Parse date (parsedate)
- Regular expression, regexp (re)
- Template (template)
- Timestamp (timestamp)
- To base16, b16, hex (to16)
- To base64, b64, hex (to64)
- To BigInt (bigint)
- To boolean (bool)
- To Float (float)
- To image (image)
- To Int (int)
- To IPv4 (ip4)
- To IPv4 net (net4)
- To IPv6 (ip6)
- To IPv6 compatible (compatible)
- To IPv6 mapped (mapped)
- To IPv6 net (net6)
- To IPv6 translated (translated)
- To MAC address (mac)
- To string (str)
- To string (stringify)
- To UTF8 (toutf8)
- To Z85, base85 (toz85)
- Cryptography group
- Date group
- Flow group
- General group
-
Geolocation group
- Coordinates distance (distance)
- Geocoord (geocoord)
- Geographic coordinate system (coordsystem)
- Geohash (geohash)
- Geohash string (geohashstr)
- Geolocated Accuracy Radius with MaxMind GeoIP2 (mm2accuracyradius)
- Geolocated ASN (mmasn)
- Geolocated ASN with MaxMind GeoIP2 (mm2asn)
- Geolocated AS Organization Name with MaxMind GeoIP2 (mm2asorg)
- Geolocated AS owner (mmasowner)
- Geolocated City (mmcity)
- Geolocated City with MaxMind GeoIP2 (mm2city)
- Geolocated Connection Speed (mmspeed)
- Geolocated connection type with MaxMind GeoIP2 (mm2con)
- Geolocated Coordinates (mmcoordinates)
- Geolocated coordinates with MaxMind GeoIP2 (mm2coordinates)
- Geolocated Country (mmcountry)
- Geolocated Country with MaxMind GeoIP2 (mm2country)
- Geolocated ISP (mmisp)
- Geolocated ISP name with MaxMind GeoIP2 (mm2isp)
- Geolocated Latitude (mmlatitude)
- Geolocated Latitude with MaxMind GeoIP2 (mm2latitude)
- Geolocated Level 1 Subdivision with MaxMind GeoIP2 (mm2subdivision1)
- Geolocated Level 2 Subdivision with MaxMind GeoIP2 (mm2subdivision2)
- Geolocated Longitude (mmlongitude)
- Geolocated Longitude with MaxMind GeoIP2 (mm2longitude)
- Geolocated Organization (mmorg)
- Geolocated organization name with MaxMind GeoIP2 (mm2org)
- Geolocated Postal Code (mmpostalcode)
- Geolocated Postal Code with MaxMind GeoIP2 (mm2postalcode)
- Geolocated Region (mmregion)
- Geolocated Region Name (mmregionname)
- ISO-3166-1 Continent Alpha-2 Code (continentalpha2)
- ISO-3166-1 Continent Name (continentname)
- ISO-3166-1 Country Alpha-2 Code (countryalpha2)
- ISO-3166-1 Country Alpha-2 Continent (countrycontinent)
- ISO-3166-1 Country Alpha-3 Code (countryalpha3)
- ISO-3166-1 Country Latitude (countrylatitude)
- ISO-3166-1 Country Longitude (countrylongitude)
- ISO-3166-1 Country Name (countryname)
- Latitude (latitude)
- Latitude and longitude coordinates (latlon)
- Longitude (longitude)
- Parse geocoord format (parsegeo)
- Represent geocoord format (reprgeo)
- Round coordinates (gridlatlon)
- JSON group
- Logic group
-
Mathematical group
- Arc cosine (acos)
- Arc sine (asin)
- Arc tangent (atan)
- Bitwise AND (band, &)
- Bitwise left shift (lshift, <<)
- Bitwise NOT (bnot, ~)
- Bitwise OR (bor, |)
- Bitwise right shift (rshift, >>)
- Bitwise unsigned right shift (urshift, >>>)
- Bitwise XOR (bxor, ^)
- Cosine (cos)
- e (mathematical constant) (e)
- Exponential: base e (exp)
- Hyperbolic cosine (cosh)
- Hyperbolic sine (sinh)
- Hyperbolic tangent (tanh)
- Logarithm: base 2 (log2)
- Logarithm: base 10 (log10)
- Logarithm: natural / arbitrary base (log)
- Pi (mathematical constant) (pi)
- Sine (sin)
- Tangent (tan)
- Meta Analysis group
- Name group
-
Network group
- HTTP Status Description (httpstatusdescription)
- HTTP Status Type (httpstatustype)
- IP Protocol (ipprotocol)
- IP Reputation Score (reputationscore)
- IP Reputation Tags (reputation)
- IPv4 legal use (purpose)
- IPv6 host number (host)
- IPv6 routing number (routing)
- Is IPv4 (ipip4)
- Is Private IPv4 (isprivate)
- Is Public IPv4 (ispublic)
- Squid Black Lists Flags (sbl)
- Order group
-
Packet group
- Ethernet destination MAC address (etherdst)
- Ethernet payload (etherpayload)
- Ethernet source MAC address (ethersrc)
- Ethernet status (etherstatus)
- Ethernet tag (ethertag)
- EtherType (ethertype)
- Has Ethernet frame (hasether)
- Has IPv4 datagram (hasip4)
- Has TCP segment (hastcp)
- Has UDP datagram (hasudp)
- IPv4 destination address (ip4dst)
- IPv4 differentiated services (ip4ds)
- IPv4 explicit congestion notification (ip4ecn)
- IPv4 flags (ip4flags)
- IPv4 fragment offset (ip4fragment)
- IPv4 header checksum (ip4cs)
- IPv4 header length (ip4hl)
- IPv4 identification (ip4ident)
- IPv4 payload (ip4payload)
- IPv4 protocol (ip4proto)
- IPv4 source address (ip4src)
- IPv4 status (ip4status)
- IPv4 time to live (ip4ttl)
- IPv4 total length (ip4len)
- IPv4 type of service (ip4tos)
- TCP ACK (tcpack)
- TCP checksum (tcpcs)
- TCP destination port (tcpdst)
- TCP flags (tcpflags)
- TCP header length (tcphl)
- TCP payload (tcppayload)
- TCP sequence number (tcpseq)
- TCP source port (tcpsrc)
- TCP status (tcpstatus)
- TCP urgent pointer (tcpurg)
- TCP window size (tcpwin)
- UDP checksum (udpcs)
- UDP destination port (udpdst)
- UDP length (udplen)
- UDP payload (udppayload)
- UDP source port (udpsrc)
- UDP status (udpstatus)
- Statistical group
-
String group
- Contains (has, ->)
- Contains - case insensitive (weakhas)
- Contains tokens (toktains)
- Contains tokens - case insensitive (weaktoktains)
- Edit distance: Damerau (damerau)
- Edit distance: Hamming (hamming)
- Edit distance: Levenshtein (levenshtein)
- Edit distance: OSA (osa)
- Ends with (endswith)
- Format number (formatnumber)
- Hostname public suffix (publicsuffix)
- Hostname root domain (rootdomain)
- Hostname root prefix (rootprefix)
- Hostname root suffix (rootsuffix)
- Hostname subdomains (subdomain)
- Hostname top level domain (topleveldomain)
- Is empty (isempty)
- Is in (`in`, <-)
- Is in - case insensitive (weakin)
- Length (length)
- Locate (locate)
- Lower case (lower)
- Matches (matches, ~)
- Peek (peek)
- Replace all (replaceall)
- Replace first (replace)
- Shannon entropy (shannonentropy)
- Split (split)
- Split regexp (splitre)
- Starts with (startswith)
- Substitute (subs)
- Substitute all (subsall)
- Substring (substring)
- Trim both sides (trim)
- Trim the left side (ltrim)
- Trim the right side (rtrim)
- Upper case (upper)
-
Web group
- Absolute URI (absoluteuri)
- Opaque URI (opaqueuri)
- URI authority (uriauthority)
- URI fragment (urifragment)
- URI host (urihost)
- URI path (uripath)
- URI port (uriport)
- URI query (uriquery)
- URI scheme (urischeme)
- URI ssp (urissp)
- URI user (uriuser)
- URL decode (urldecode)
- User Agent Company (uacompany)
- User Agent Company URL (uacompanyurl)
- User Agent Device Icon (uadeviceicon)
- User Agent Device Information URL (uadeviceinfourl)
- User Agent Device Type (uadevicetype)
- User Agent Family (uafamily)
- User Agent Icon (uaicon)
- User Agent Information URL (uainfourl)
- User Agent is Robot (uaisrobot)
- User Agent Name (uaname)
- User Agent OS Company (uaoscompany)
- User Agent OS Company URL (uaoscompanyurl)
- User Agent OS Family (uaosfamily)
- User Agent OS Icon (uaosicon)
- User Agent OS Name (uaosname)
- User Agent OS URL (uaosurl)
- User Agent Type (uatype)
- User Agent URL (uaurl)
- User Agent Version (uaversion)
-
Aggregation operations
-
Working in the search window
-
Generate charts
- Affinity chord diagram
- Availability timeline
- Bipartite chord diagram
- Bubble chart
- Chart aggregation
- Custom date chart aggregation
- Flame graph
- Flat world map by coordinates
- Flat world map by country
- Google animated heat map
- Google area map
- Google heat map
- Graph diagram
- Histogram
- Pew Pew map
- Pie chart
- Pie layered chart
- Punch card
- Robust Random Cut Forest chart
- Sankey diagram
- Scatter plot
- Time heatmap
- Triple exponential chart
- Voronoi treemap
- Data enrichment
- Setting up a data table
- Advanced data operations
- Use case: eCommerce behavior analysis
-
Generate charts
- Managing your queries
- Best practices for data search
- Monitoring tables
-
Parsers and collectors
- About Devo tags
- Special Devo tags and data tables
-
List of Devo parsers
- Business & Consumer
- Cloud technologies
- Databases
- Host and Operating Systems
-
Network and application security
- auth.secureauth
- auth.securenvoy
- av.mcafee
- av.sophos
- box.iptables
- edr.cylance
- edr.fireeye.alerts
- edr.minervalabs.events
- endpoint.symantec
- firewall.checkpoint
- firewall.cisco firepower and vpn.cisco
- firewall.fortinet
- firewall.huawei
- firewall.juniper
- firewall.paloalto
- firewall.pfsense
- firewall.sonicwall
- firewall.sophos
- firewall.sophos.xgfirewall
- firewall.stonegate
- firewall.windows
- mail.proofpoint
- nac.aruba
- network.meraki
- network.versa
- proxy.bluecoat
- proxy.forcepoint
- proxy.squid
- uba.varonis
- vuln.beyondtrust
- vpn.pulsesecure.sa
- Network connectivity
- Web servers
- Technologies supported in CEF syslog format
- Collectors
- Activeboards
-
Dashboards
- Create a new dashboard
-
Working with dashboard widgets
- Availability timeline widget
- Chord diagram widget
- Circle world map widget
- Color key value widget
- Color world map widget
- Column chart widget
- Comparative chart widget
- Funnel widget
- Gauge meter widget
- Google heatmap widget
- Heat calendar widget
- Line chart widget
- Monitoring widget
- Pie chart widget
- Punch card widget
- Sectored pie chart widget
- Table widget
- Time heatmap widget
- Tree diagram widget
- Voronoi tree widget
- Configuring and sharing dashboards
- Alerts and notifications
- Panels
- Applications
- Tools
- Social Intelligence
- API reference
- Release notes
Technologies configuration
Introduction
In the context of Service Operations, a technology consists of a data source, understood as an internal data structure or table, alongside a number of metadata records and links to other resources in the platform. For example, a technology database would include in its definition the main entry point of its data when ingested in Devo (e.g., my.app.database.mydatabase) and linkages with existing models or activeboards, and links to internal or external resources such as information repositories, etc. Similar to the case of models, Devo provides off-the-shelf technologies support with Service Operations, leaving end-users of the application with the possibility to create their own ones.
Available technologies
To list all available technologies in a Service Operations instance, access the administration section in the application clicking on the cogwheel icon in the upper-right side of the user interface. Once in there, click on Technologies > Supported technologies on the left menu.
This section does not allow the creation or modification of existing technologies. However, it allows users to list, access related documentation, or open the raw data structure in the Devo core platform. Using the upper part of the UI it is possible to see all available technologies categories for quick access or filtering. You can use the search boxes in both subsections to filter categories or technologies by their respective name.
Creating / editing technologies
To list all available technologies in a Service Operations instance, access the administration section in the application by clicking on the cogwheel icon in the upper-right side of the user interface. Once in there, click on Technologies > Supported technologies on the left menu.
Clicking on the Create new technology button will display the technologies creation and editing form:
Parameter | Description | Value type | Example |
---|---|---|---|
Technology name | Overall behavior of the entity in terms of its status calculation. When auto-mode = off, the value of the entity will be retrieved from the result of the associated query defined for it. If auto-mode = on, the result of the value will be ignored and overridden with the warning and critical status definition as specified below. | Text string | XYZ database |
Technology logo | URL to the image file (.png, .jpg, .gif) used as logo for the technology | Text string | https://cdn.pixabay.com/photo/2016/08/25/07/30/red-1618916_1280.png |
Publisher | Person, organization or company that created the technology | Text string | Devo |
Category | Type of technology | Selectable or new text string | Cloud |
Subcategory | Subtype, area or purpose of technology | Selectable or new text string | Performance metrics |
Main data structure | Data table that holds the information related to the technology | Devo data structure identifier | my.app.databasexyz.metrics |
Data sample file | Technology-related sample files, to be used as inputs for the synthetic data injector | Uploaded file (max 20MB) | databasexyz_performance_metrics.log |
List of associated maps | Models that utilize or rely on the technology in its definition, or show information related to it | Selectable map name | Databases health |
List of associated activeboards | Activeboards that utilize or rely on the technology, or show information related to it | Name, URL text strings | Databases status - https://devo.com/39393993 |
Setup documentation URL | Link to internal or external documentation related to the configuration of the technology as a data source (collectors page, etc.) | Text string | https://docs.devo.com/confluence/ndt/supported-technologies/list-of-supported-technologies |
Reference or troubleshooting guidelines URL | Link to internal or external documentation sites related to the technology itself: official manuals, administration guides, etc. | Text string |
Data Sample File Structure
Uploaded data sample files must comply with the following specification per line in the file. Each line will be treated as a single event:
An example of a valid data sample file would be the following:
1598227200000 GR-1,cluster0001,node0001,svm0001,vol0001,agg0001,disk03rn97cn,eth0001.1,9.1,7.7,6.5,87,3910,8,51,US,CA,Pasadena
1598227201000 GR-2,cluster0002,node0002,svm0002,vol0002,agg0002,diskiktcz5uc,eth0001.2,9.1,3,3.3,48,4664,8,30,US,UT,Salt Lake City
1598227202000 GR-3,cluster0003,node0003,svm0003,vol0003,agg0003,diskjczmhxhx,eth0001.3,9.1,2.2,4.9,51,5574,12,45,US,TX,El Paso
1598227203000 GR-4,cluster0004,node0004,svm0004,vol0004,agg0004,diskxbhe4w45,eth0001.4,9.1,9.5,8.2,80,9934,5,58,US,VT,Montpelier
1598227204000 GR-1,cluster0005,node0005,svm0005,vol0005,agg0005,disklgzph91g,eth0001.5,9.1,7.9,1.4,35,6909,15,31,US,PA,Mc Keesport
The synthetic data injection engine does not pay attention to the exact timestamp to perform the data ingestion operation at that specific time. Instead, the timestamp is used as a reference to set the cadence of events between them. As an example, if the previous file was utilized as source data for a job, the result would be the following:
The job would ingest the first event in the file (line #1) immediately, i.e., ignoring the timestamp.
The second event would be ingested after 1000 milliseconds, which is the difference between the second and the first timestamp (1000 milliseconds).
The third and following events would be processed in exactly the same way, that is, introducing a delay between them which is the delta of milliseconds between their respective timestamps.
Once the last line of the file is processed, the job would stop if its type is ‘once’, or it would continue ingesting again the first event in the file.
Injecting synthetic data
Service Operations implements a built-in data injector server that allows end-users to create their own realtime data injection processes.
The data injector utilizes the definition of the entry technology entry point (internal data table) and the uploaded sample files to initiate back-end jobs in charge of sending the data contained in the sample files to the targeted table. This way, two objectives can be met:
Simulate the ingestion of the technology data into Devo so that the defined models can utilize it as baseline information to populate them.
Simulate the ingestion of the technology data for other purposes across the entire Devo platform (to feed Activeboards, for example).
Simulate specific scenarios (service errors, for example) by injecting raw data that contains the conditions to generate them.
Access the data injector by selecting Synthetic data injector on the left menu. The following UI will be shown:
To start a new injection job, follow the next steps:
Click on the start new job button corresponding to the technology whose data you want to be ingested into Devo.
Choose from the available data sample files the one you want to use as the source.
Select the ingestion mode: "continuous" for a cyclic ingestion—the content of the file will be ingested recurrently from beginning to end—, or "once" for a one-time pass to the same file.
Click on apply.
The new job will be displayed in the upper list of active jobs, along with its status. "Continuous" jobs are meant to remain in running status until they are stopped manually. "Once"-typed ones will, on their side, eventually reach stopped status as soon as they finish processing the sample file and ingesting it into the targeted data structure.
Deleting technologies
Click on the corresponding button under the manage technologies section.
Note: all associated resources (i.e., sample files) will be deleted as well. As a result, active injection jobs will be stopped.