• Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
  • The Devo data analytics platform
    • How Devo indexes data
    • How Devo works
    • Key concepts
  • Getting started
    • Sign up and log in
    • Navigating the Devo app
    • User preferences
    • Devo video tutorials
  • Domain administration
    • Users and roles
      • Managing users
      • Monitoring user activity
      • Managing roles
        • Assign resources to a role
      • Role permissions
      • Role mapping
    • Security credentials
      • Access keys
      • X.509 certificates
      • Authentication tokens
    • Applications gallery
    • Domain preferences
    • User authentication
      • Password
      • SAML
        • Google as an identity provider
        • Okta as an identity provider
        • OneLogin as an identity provider
        • O365/Azure AD as an identity provider
      • OpenID
    • Data processes and feeds
      • Aggregation tasks
      • Injections
      • Permalinks
      • API & OData feeds
  • Sending data to Devo
    • The Devo In-House Relay
      • Installing the Devo Relay
        • Install the Relay on an Ubuntu box (v1.4.2)
        • Install the Relay on a Red Hat or CentOS box (v1.4.2)
        • Install with Docker
      • Configuring the In-House Relay
        • Relay rules
          • Defining a relay rule
          • The 4 predefined relay rules
          • 5 common relay rule scenarios
            • Scenario 1: Apply a fixed tag to all events
            • Scenario 2: Apply a Devo tag based on data found in the inbound event
            • Scenario 3: Filter out unwanted events
            • Scenario 4: Assign dynamic Devo tag using inbound source data
            • Scenario 5: Appending the inbound syslog tag to the outbound Devo tag
          • Using regex in relay rules
        • Customizing In-House Relay settings
        • Managing the relay on the command line
        • Setting up High-Availability with Keepalived (v1.4.2)
        • Relay buffers
      • Relay migration
      • Sending SSL/TLS encrypted events to the Devo relay
      • Relay troubleshooting tips (v1.4.2)
        • Relay troubleshooting tips (v1.4.0)
        • Relay troubleshooting tips (for versions prior to 1.4.0)
    • Event sources
      • Unix-like machines
        • Installing Devo packages for *nix
        • Third-party syslog tools configuration
          • rsyslog
            • Simple sending using rsyslog
            • Secure sending using rsyslog
            • Monitoring files using rsyslog
            • Obsolete legacy format
              • Simple sending using rsyslog (Obsolete legacy format)
              • Secure sending using rsyslog (Obsolete legacy format)
              • Monitoring files using rsyslog (Obsolete legacy format)
          • syslog-ng
            • Simple sending using syslog-ng
            • Secure sending using syslog-ng
            • Monitoring files using syslog-ng
          • syslog/syslogd
        • SELinux configuration conflicts
      • Windows
        • Devo Agent for Windows
        • Configuring WMI for Devo file monitoring
        • NXLog for Windows event collection
      • MacOS X
      • Cloud services
        • AWS S3 Buckets
        • Microsoft Azure
      • Commercial products
      • Custom apps
        • Java apps
          • JDK java.util.logging
          • Scoja client library
          • Sample code
        • Node.js apps
        • Python apps
    • Other data collection methods
      • HTTP endpoint
      • Logstash
      • Fluentd
      • NXLog
    • Uploading log files
    • Devo software
  • Searching data
    • Accessing data tables
      • Run a search using a finder
        • Use a custom finder
          • Create a custom finder
          • Assign a custom finder to a role
          • Edit a custom finder
        • Use the aliased finder
          • Add a query to your aliased finder
      • Run a global search
      • Run a LINQ free text query
      • Run a search with selected columns only
        • Selecting specific columns in LINQ
        • Selecting specific columns with the Finder
        • Selecting unrevealed columns
    • Building a query
      • Data types in Devo
      • Build a query in the search window
        • Filter data
        • Filter column data using the OR selector
        • Create columns
        • Group data
        • Aggregate data
      • Build a query using LINQ
        • LINQ query examples
      • Working with JSON objects in data tables
      • Subqueries
      • Operations reference
        • Aggregation operations
          • Average (avg)
          • Count (count)
          • First (first)
          • First not null (nnfirst)
          • HyperLogLog++ (hllpp)
          • HyperLogLog++ Count Estimation (hllppcount)
          • Last (last)
          • Last not null (nnlast)
          • Maximum (max)
          • Median / 2nd quartile / Percentile 50 (median)
          • Minimum (min)
          • Non-null average (nnavg)
          • Non-null standard deviation (biased) (nnstddev)
          • Non-null standard deviation (unbiased) (nnustddev)
          • Non-null variance (biased) (nnvar)
          • Non-null variance (unbiased) (nnuvar)
          • Percentile 10 (percentile10)
          • Percentile 25 / 1st quartile (percentile25)
          • Percentile 5 (percentile5)
          • Percentile 75 / 3rd quartile (percentile75)
          • Percentile 90 (percentile90)
          • Percentile 95 (percentile95)
          • Standard deviation (biased) (stddev)
          • Standard deviation (unbiased) (ustddev)
          • Sum (sum)
          • Sum Square (sum2)
          • Variance (biased) (var)
          • Variance (unbiased) (uvar)
        • Arithmetic group
          • Absolute value (abs)
          • Addition, sum, plus / Concatenation (add, +)
          • Ceiling (ceil)
          • Cube root (cbrt)
          • Division (div, \)
          • Division remainder (rem, %)
          • Floor (floor)
          • Modulo (mod, %%)
          • Multiplication, product (mul, *)
          • Power (pow)
          • Real division (rdiv, /)
          • Rounding (round)
          • Sign (signum)
          • Square root (sqrt)
          • Subtraction, minus / Additive inverse (sub, -)
        • Conversion group
          • Duration (duration)
          • Format date (formatdate)
          • From base16, b16, hex (from16)
          • From base64, b64 (from64)
          • From UTF8 (fromutf8)
          • From Z85, base85 (fromz85)
          • Human size (humanSize)
          • Make byte array (mkboxar)
          • Parse date (parsedate)
          • Regular expression, regexp (re)
          • Template (template)
          • Timestamp (timestamp)
          • To base16, b16, hex (to16)
          • To base64, b64, hex (to64)
          • To BigInt (bigint)
          • To boolean (bool)
          • To Float (float)
          • To image (image)
          • To Int (int)
          • To IPv4 (ip4)
          • To IPv4 net (net4)
          • To IPv6 (ip6)
          • To IPv6 compatible (compatible)
          • To IPv6 mapped (mapped)
          • To IPv6 net (net6)
          • To IPv6 translated (translated)
          • To MAC address (mac)
          • To string (str)
          • To string (stringify)
          • To UTF8 (toutf8)
          • To Z85, base85 (toz85)
        • Cryptography group
          • MD5 hash function (md5)
          • SHA1 hash function (sha1)
          • SHA256 hash function (sha256)
          • SHA512 hash function (sha512)
        • Date group
          • Day / Day of the month (day)
          • Day of the week (dayofweek)
          • Day of the year (dayofyear)
          • Epoch milliseconds (epoch)
          • Hour (hour)
          • Millisecond (millisecond)
          • Minute (minute)
          • Month (month)
          • Period (period)
          • Second (second)
          • Today (today)
          • Tomorrow (tomorrow)
          • Year (year)
          • Yesterday (yesterday)
        • Flow group
          • Conditional (ifthenelse)
          • Decode, switch (decode)
          • Null value locator (nvl)
        • General group
          • Is not null (isnotnull)
          • Is null (isnull)
        • Geolocation group
          • Coordinates distance (distance)
          • Geocoord (geocoord)
          • Geographic coordinate system (coordsystem)
          • Geohash (geohash)
          • Geohash string (geohashstr)
          • Geolocated Accuracy Radius with MaxMind GeoIP2 (mm2accuracyradius)
          • Geolocated ASN (mmasn)
          • Geolocated ASN with MaxMind GeoIP2 (mm2asn)
          • Geolocated AS Organization Name with MaxMind GeoIP2 (mm2asorg)
          • Geolocated AS owner (mmasowner)
          • Geolocated City (mmcity)
          • Geolocated City with MaxMind GeoIP2 (mm2city)
          • Geolocated Connection Speed (mmspeed)
          • Geolocated connection type with MaxMind GeoIP2 (mm2con)
          • Geolocated Coordinates (mmcoordinates)
          • Geolocated coordinates with MaxMind GeoIP2 (mm2coordinates)
          • Geolocated Country (mmcountry)
          • Geolocated Country with MaxMind GeoIP2 (mm2country)
          • Geolocated ISP (mmisp)
          • Geolocated ISP name with MaxMind GeoIP2 (mm2isp)
          • Geolocated Latitude (mmlatitude)
          • Geolocated Latitude with MaxMind GeoIP2 (mm2latitude)
          • Geolocated Level 1 Subdivision with MaxMind GeoIP2 (mm2subdivision1)
          • Geolocated Level 2 Subdivision with MaxMind GeoIP2 (mm2subdivision2)
          • Geolocated Longitude (mmlongitude)
          • Geolocated Longitude with MaxMind GeoIP2 (mm2longitude)
          • Geolocated Organization (mmorg)
          • Geolocated organization name with MaxMind GeoIP2 (mm2org)
          • Geolocated Postal Code (mmpostalcode)
          • Geolocated Postal Code with MaxMind GeoIP2 (mm2postalcode)
          • Geolocated Region (mmregion)
          • Geolocated Region Name (mmregionname)
          • ISO-3166-1 Continent Alpha-2 Code (continentalpha2)
          • ISO-3166-1 Continent Name (continentname)
          • ISO-3166-1 Country Alpha-2 Code (countryalpha2)
          • ISO-3166-1 Country Alpha-2 Continent (countrycontinent)
          • ISO-3166-1 Country Alpha-3 Code (countryalpha3)
          • ISO-3166-1 Country Latitude (countrylatitude)
          • ISO-3166-1 Country Longitude (countrylongitude)
          • ISO-3166-1 Country Name (countryname)
          • Latitude (latitude)
          • Latitude and longitude coordinates (latlon)
          • Longitude (longitude)
          • Parse geocoord format (parsegeo)
          • Represent geocoord format (reprgeo)
          • Round coordinates (gridlatlon)
        • JSON group
          • Jq evaluation (jqeval)
          • Jq filter compilation (jqcompile)
          • Json value type (label)
          • To json (jsonparse)
        • Logic group
          • And (and)
          • Not (not)
          • Or (or)
        • Mathematical group
          • Arc cosine (acos)
          • Arc sine (asin)
          • Arc tangent (atan)
          • Bitwise AND (band, &)
          • Bitwise left shift (lshift, <<)
          • Bitwise NOT (bnot, ~)
          • Bitwise OR (bor, |)
          • Bitwise right shift (rshift, >>)
          • Bitwise unsigned right shift (urshift, >>>)
          • Bitwise XOR (bxor, ^)
          • Cosine (cos)
          • e (mathematical constant) (e)
          • Exponential: base e (exp)
          • Hyperbolic cosine (cosh)
          • Hyperbolic sine (sinh)
          • Hyperbolic tangent (tanh)
          • Logarithm: base 2 (log2)
          • Logarithm: base 10 (log10)
          • Logarithm: natural / arbitrary base (log)
          • Pi (mathematical constant) (pi)
          • Sine (sin)
          • Tangent (tan)
        • Meta Analysis group
          • Pragma value (pragmavalue)
          • Table name (tablename)
        • Name group
          • Any name matches (anymatches)
          • Glob pattern on names (nameglob)
        • Network group
          • HTTP Status Description (httpstatusdescription)
          • HTTP Status Type (httpstatustype)
          • IP Protocol (ipprotocol)
          • IP Reputation Score (reputationscore)
          • IP Reputation Tags (reputation)
          • IPv4 legal use (purpose)
          • IPv6 host number (host)
          • IPv6 routing number (routing)
          • Is IPv4 (ipip4)
          • Is Private IPv4 (isprivate)
          • Is Public IPv4 (ispublic)
          • Squid Black Lists Flags (sbl)
        • Order group
          • Equal (eq, =)
          • Equal - case insensitive (eqic)
          • Greater or equal (ge, >=)
          • Greater than (gt, >)
          • Less or equal (le, <=)
          • Less than (lt, <)
          • Not equal (ne, /=)
        • Packet group
          • Ethernet destination MAC address (etherdst)
          • Ethernet payload (etherpayload)
          • Ethernet source MAC address (ethersrc)
          • Ethernet status (etherstatus)
          • Ethernet tag (ethertag)
          • EtherType (ethertype)
          • Has Ethernet frame (hasether)
          • Has IPv4 datagram (hasip4)
          • Has TCP segment (hastcp)
          • Has UDP datagram (hasudp)
          • IPv4 destination address (ip4dst)
          • IPv4 differentiated services (ip4ds)
          • IPv4 explicit congestion notification (ip4ecn)
          • IPv4 flags (ip4flags)
          • IPv4 fragment offset (ip4fragment)
          • IPv4 header checksum (ip4cs)
          • IPv4 header length (ip4hl)
          • IPv4 identification (ip4ident)
          • IPv4 payload (ip4payload)
          • IPv4 protocol (ip4proto)
          • IPv4 source address (ip4src)
          • IPv4 status (ip4status)
          • IPv4 time to live (ip4ttl)
          • IPv4 total length (ip4len)
          • IPv4 type of service (ip4tos)
          • TCP ACK (tcpack)
          • TCP checksum (tcpcs)
          • TCP destination port (tcpdst)
          • TCP flags (tcpflags)
          • TCP header length (tcphl)
          • TCP payload (tcppayload)
          • TCP sequence number (tcpseq)
          • TCP source port (tcpsrc)
          • TCP status (tcpstatus)
          • TCP urgent pointer (tcpurg)
          • TCP window size (tcpwin)
          • UDP checksum (udpcs)
          • UDP destination port (udpdst)
          • UDP length (udplen)
          • UDP payload (udppayload)
          • UDP source port (udpsrc)
          • UDP status (udpstatus)
        • Statistical group
          • Approximated estimation (estimation)
          • HyperLogLog++ pack (pack)
          • HyperLogLog++ unpack (unpackhllpp)
        • String group
          • Contains (has, ->)
          • Contains - case insensitive (weakhas)
          • Contains tokens (toktains)
          • Contains tokens - case insensitive (weaktoktains)
          • Edit distance: Damerau (damerau)
          • Edit distance: Hamming (hamming)
          • Edit distance: Levenshtein (levenshtein)
          • Edit distance: OSA (osa)
          • Ends with (endswith)
          • Format number (formatnumber)
          • Hostname public suffix (publicsuffix)
          • Hostname root domain (rootdomain)
          • Hostname root prefix (rootprefix)
          • Hostname root suffix (rootsuffix)
          • Hostname subdomains (subdomain)
          • Hostname top level domain (topleveldomain)
          • Is empty (isempty)
          • Is in (`in`, <-)
          • Is in - case insensitive (weakin)
          • Length (length)
          • Locate (locate)
          • Lower case (lower)
          • Matches (matches, ~)
          • Peek (peek)
          • Replace all (replaceall)
          • Replace first (replace)
          • Shannon entropy (shannonentropy)
          • Split (split)
          • Split regexp (splitre)
          • Starts with (startswith)
          • Substitute (subs)
          • Substitute all (subsall)
          • Substring (substring)
          • Trim both sides (trim)
          • Trim the left side (ltrim)
          • Trim the right side (rtrim)
          • Upper case (upper)
        • Web group
          • Absolute URI (absoluteuri)
          • Opaque URI (opaqueuri)
          • URI authority (uriauthority)
          • URI fragment (urifragment)
          • URI host (urihost)
          • URI path (uripath)
          • URI port (uriport)
          • URI query (uriquery)
          • URI scheme (urischeme)
          • URI ssp (urissp)
          • URI user (uriuser)
          • URL decode (urldecode)
          • User Agent Company (uacompany)
          • User Agent Company URL (uacompanyurl)
          • User Agent Device Icon (uadeviceicon)
          • User Agent Device Information URL (uadeviceinfourl)
          • User Agent Device Type (uadevicetype)
          • User Agent Family (uafamily)
          • User Agent Icon (uaicon)
          • User Agent Information URL (uainfourl)
          • User Agent is Robot (uaisrobot)
          • User Agent Name (uaname)
          • User Agent OS Company (uaoscompany)
          • User Agent OS Company URL (uaoscompanyurl)
          • User Agent OS Family (uaosfamily)
          • User Agent OS Icon (uaosicon)
          • User Agent OS Name (uaosname)
          • User Agent OS URL (uaosurl)
          • User Agent Type (uatype)
          • User Agent URL (uaurl)
          • User Agent Version (uaversion)
    • Working in the search window
      • Generate charts
        • Affinity chord diagram
        • Availability timeline
        • Bipartite chord diagram
        • Bubble chart
        • Chart aggregation
        • Custom date chart aggregation
        • Flame graph
        • Flat world map by coordinates
        • Flat world map by country
        • Google animated heat map
        • Google area map
        • Google heat map
        • Graph diagram
          • Creating a graph diagram
          • Working in the graph diagram
          • Monitor intranet traffic to dangerous websites
        • Histogram
        • Pew Pew map
        • Pie chart
        • Pie layered chart
        • Punch card
        • Robust Random Cut Forest chart
        • Sankey diagram
        • Scatter plot
        • Time heatmap
        • Triple exponential chart
        • Voronoi treemap
      • Data enrichment
        • Upload a lookup table
        • Create a lookup table from a query
        • Add lookup values to your query
        • Manage and edit lookup tables
        • Threat lookups
      • Setting up a data table
        • Modifying the column layout
          • Arrange and resize columns
          • Hide and show columns
          • Change the position of column headers
          • Sort data
          • Setting a default table layout
        • Add a description to a data table
        • Autoparser
          • Autoparse a JSON object
      • Advanced data operations
        • Graphical correlation
          • Cross-Search Graph Diagram
          • Cross-Search Table Join
          • Cross-Search Sankey Diagram
          • Cross-Search Line Chart
        • Custom and union tables
          • Create a custom table
          • Create a union table
          • Manage custom and union tables
        • Set up a data source
        • Inject data to a new table
        • Drill downs
        • Manipulate your data using CyberChef
        • Time series report
      • Use case: eCommerce behavior analysis
        • Step 1. Error analysis using status codes
          • Specific analysis for 404 codes
          • Custom alerts for 404 errors
        • Step 2. Operating system ranking
          • Get the usage share of operating systems
          • Visualize the usage share of operating systems
        • Step 3. Country distribution
          • Build a histogram displaying country distribution
          • Geolocate IP addresses
    • Managing your queries
      • Rename a query
      • Favorite queries
      • Query history
      • Check currently running queries
      • Add a description to your query
      • Block a query
      • Share a query
      • Download query data
      • Close a query
      • Load query data into Excel using the Devo Connector add-in
      • Query priority
    • Best practices for data search
    • Monitoring tables
      • Web application monitoring
      • Alerts monitoring
  • Parsers and collectors
    • About Devo tags
    • Special Devo tags and data tables
    • List of Devo parsers
      • Business & Consumer
      • Cloud technologies
        • cdn.akamai
        • cloud.aws.cloudtrail.events
          • Forwarding the events using Node.js
          • Forwarding the events using Python
        • cloud.aws.cloudwatch.events
        • cloud.office365.siem
      • Databases
        • db.mysql
      • Host and Operating Systems
        • box.unix
        • box.vmware
        • box.win
        • box.win_nxlog
        • box.win_snare
      • Network and application security
        • auth.secureauth
        • auth.securenvoy
        • av.mcafee
        • av.sophos
        • box.iptables
        • edr.cylance
        • edr.fireeye.alerts
        • edr.minervalabs.events
        • endpoint.symantec
        • firewall.checkpoint
        • firewall.cisco firepower and vpn.cisco
        • firewall.fortinet
        • firewall.huawei
        • firewall.juniper
        • firewall.paloalto
          • Sending Palo Alto events to Devo relay using SSL
        • firewall.pfsense
        • firewall.sonicwall
        • firewall.sophos
        • firewall.sophos.xgfirewall
        • firewall.stonegate
        • firewall.windows
        • mail.proofpoint
        • nac.aruba
        • network.meraki
        • network.versa
        • proxy.bluecoat
        • proxy.forcepoint
        • proxy.squid
        • uba.varonis
        • vuln.beyondtrust
        • vpn.pulsesecure.sa
      • Network connectivity
        • netstat.netflow
        • dns.bind
        • dns.windows
        • ftp.iis
      • Web servers
        • web.apache
        • web.apache.mod-security
        • web.iis
        • web.jboss
        • web.nginx
        • web.tomcat
      • Technologies supported in CEF syslog format
    • Collectors
      • AWS collector
      • Google Cloud Platform collector
      • G Suite collectors
        • G Suite Alerts collector
        • G Suite Reports collector
      • Microsoft Azure collector
      • Microsoft Graph collector
      • Okta collectors
        • Okta collector
        • Okta Advanced Server Access collector
      • OneLogin collector
      • Rapid7 InsightVM collector
      • Salesforce collector
  • Activeboards
    • Creating an Activeboard
    • Working with Activeboard widgets
      • Area chart widgets
      • Column chart widgets
      • Donut chart widgets
      • Heatmap widgets
      • Line chart widgets
      • Pie chart widgets
      • Simple value widgets
      • Table widgets
      • Voronoi diagram widgets
    • Working with Activeboard inputs
    • LINQ syntax differences between Activeboards and the search window
    • Setting a time range in Activeboards
    • Working with Activeboards
      • Active Refresh
    • Sharing Activeboards
  • Dashboards
    • Create a new dashboard
    • Working with dashboard widgets
      • Availability timeline widget
      • Chord diagram widget
      • Circle world map widget
      • Color key value widget
      • Color world map widget
      • Column chart widget
      • Comparative chart widget
      • Funnel widget
      • Gauge meter widget
      • Google heatmap widget
      • Heat calendar widget
      • Line chart widget
        • Customize your Line chart
      • Monitoring widget
      • Pie chart widget
      • Punch card widget
      • Sectored pie chart widget
      • Table widget
      • Time heatmap widget
      • Tree diagram widget
      • Voronoi tree widget
    • Configuring and sharing dashboards
  • Alerts and notifications
    • Creating new alerts
      • Alert trigger methods
        • Each alert type
        • Several alert type
        • Low alert type
          • Inactivity alert
        • Rolling alert type
        • Deviation alert type
        • Gradient alert type
      • Create an alert based on triggered alerts
    • Configuring alerts
      • Manage defined alerts
      • Manage sending policies
      • Manage delivery methods
        • Email delivery methods
        • HTTP-JSON delivery methods
        • Service Desk delivery methods
        • Jira delivery methods
        • Pushover delivery methods
        • PagerDuty delivery methods
        • Slack delivery methods
      • Manage anti-flooding policies
      • Make an alert available for panels
      • Pre-installed alert reference
    • Managing triggered alerts
      • Add a comment to a triggered alert
      • Apply a filter for post-processing
  • Panels
    • Create and customize a panel
    • Adding an alert to a panel
    • Adding a query to a panel
    • Using panels
  • Applications
    • Devo Security Operations
      • Overview Dashboard
      • Triage
        • Triaging alerts
        • Triaging investigations
      • Investigations
      • Threat Hunting
      • Use cases
        • Phishing attack
        • Command & Control
        • Alerting system status
    • Devo Stats
      • Working in the Devo Stats application
      • Application tabs and widgets
        • User tab
        • Volume tab
        • Query tab
          • User Query Info
          • CPU Query Info
          • CPU Query Info Multidomain
        • Status tab
    • Security Insights
      • Installing Security Insights
        • Security Insights lookups
      • Configuring Security Insights
      • Navigating Security Insights
        • Overview tab
        • Threats tab
        • Network tab
        • DNS tab
        • Firewall tab
        • Proxy tab
        • Access tab
        • Web tab
        • IDS tab
    • Service operations
      • Basic concepts
      • Installation
      • Global models
      • Technologies configuration
      • Models configuration
      • Service overview
      • Incidents viewer
      • Monitors
      • User experience management
      • Use case for service operations
        • Initial analysis
        • Model configuration
        • Running the model
        • Incidents
    • Systems Monitoring
      • Basic monitoring
      • Advanced monitoring
  • Tools
    • Data Explorer
    • Query App
    • Time Series Analytics
  • Social Intelligence
  • API reference
    • REST API
      • Authorizing REST API requests
      • Running queries with the REST API
        • Forward response to HDFS
        • Forward response to Amazon S3
        • Forward response to SNMP
        • Forward response to email
        • Send requests with Postman
      • Job requests
    • Provisioning API
      • Authorizing Provisioning API requests
      • Common operations
        • User operations
        • Domain operations
        • Certificates
        • Role management
        • Domain resources management
      • Reseller operations
        • Price plans
      • Role specification and examples
    • Alerting API
      • Working with alert definitions
    • Using and managing OData feeds
      • Connecting with Excel
      • Connecting with Tableau
      • Connecting with Power BI
  • Release notes
    • 6.0.0
    • 6.0.1
    • 6.1.0
    • 6.1.1
    • 6.1.2
    • 6.1.3
    • 6.1.4
    • 6.2.0
    • 6.3.0
    • 6.3.1
    • 6.3.2
    • 6.3.3-1
    • 6.4.0
    • 6.4.1
    • 6.4.3
    • 6.5.0
    • 6.5.1
    • 6.5.2
    • 6.6.0
    • 6.6.1
    • 6.6.2
    • 6.7.0
PREVIOUS
G Suite Alerts collector
NEXT
Microsoft Azure collector

Parsers and collectors / Collectors / G Suite collectors / G Suite Reports collector

Download as PDF

G Suite Reports collector

Service description

The G Suite Reports API is used to gain insights on content management with Google activity, audit administrator actions, and generate customer and user usage reports.

Data source description

The G Suite API generates account activities for these applications and sources. The collector process the Google API responses and send them to Devo platform that will categorize all information received on tables along rows and columns on your Devo domain.

G Suite Reports

Listed in the table below are some application names, details and how Devo platform treats the data.

Application name

Details

Devo data tables

Access Transparency

Activity events from a G Suite resource accessed by Google.

cloud.gsuite.reports.access_transparency

Admin

Report returns information on the Admin console activities of all of your account's administrators.

cloud.gsuite.reports.admin

Calendar

Report returns information about how your account's users manage and modify their Google Calendar events.

cloud.gsuite.reports.calendar

Google Drive

Report returns information about how your account's users manage, modify, and share their Google Drive documents.

cloud.gsuite.reports.drive

Google Cloud Platform

Activity events Interaction with the Cloud OS Login API.

cloud.gsuite.reports.gcp

Groups

Activity report returns information about how your account's users manage and modify their groups.

cloud.gsuite.reports.groups

Google+

Activity report returns information about the Google+ activity of all of your account's users.

cloud.gsuite.reports.gplus

Enterprise Groups

Audit activity events from actions performed by the moderator.

cloud.gsuite.reports.cloud.gsuite.reports.groups_enterprise

Jamboard

Activity of interactive whiteboard.

cloud.gsuite.reports.jamboard

Meet

Hangouts Meet Audit activity events describing a single Hangouts endpoint.

cloud.gsuite.reports.meet

Logins

Activity report returns information about the login activity of all of your account's users.

cloud.gsuite.reports.login

Mobile Audit

Activity report returns information on all activities in a mobile device with a Work account, managed by Google Mobile Management.

cloud.gsuite.reports.mobile

SAML

Audit activity events from login event type. 

cloud.gsuite.reports.saml

Authorization Tokens

Activity report returns information about third-party websites and applications your users have granted access to.

cloud.gsuite.reports.token

Rules

Activity report returns information about how the rules (that have been set up in Admin console) are performing.

cloud.gsuite.reports.rules

Users Account

User Accounts Audit activity events.

cloud.gsuite.reports.users_account

Each report uses the basic endpoint request with report-specific parameters or event type. The maximum time period for each report is the last 180 days. For more references about G Suite Reports, visit the Google API Reference documentation.

Setup

G Suite Reports credentials

G Suite Reports requires a script to generate a token (token.pickle). Google uses a third-party service (OAuth) to authenticate this service. To create the required credentials, you must access the Google API Console and get the credentials-gsuite-reports.json file.

  1. Create a project in the Google API console.


  2. Go to Credentials → + Create Credentials and select OAuth Client ID:


  3. Select Desktop App, enter the name you prefer and click the Create button.


    Click OK in the popup window that appears.
  4. Download the credentials in JSON format by clicking the black arrow in the right corner.


  5. Rename the file as credentials-gsuite-reports.json and move it to <any_directory>/devo-collectors/gsuite-reports/credentials directory.

Assigning roles to G Suite Reports 

Follow these steps to grant domain access to G Suite Reports:

  1. Go to the G Suite Admin Console.

  2. Click the Security option and select Advanced Settings.

  3. From the Authentication section, click Manage API client access.

  4. Grant the user who will delegate the permissions access to the G Suite Reports.

    1. Click Admin Roles and select Create a new role.

    2. Search for Reports API privileges list and drop-down to view the View Access checkbox. Check it and save the new role.

    3. Search the user and assign the new role.

Access token

To run these applications, you'll need:

  • Python 3.6 or greater

  • A G Suite domain with API access enabled.

  • A Google account in that domain with administrator privileges.

If you're a G Suite admin, go to Turn on the reports API, click on Enable the Reports API and download the credentials.json file to generate the G Suite token later on. If you don't have an admin account, ask your G Suite admin to enable this option. Learn more in G Suite Docs.

Follow these steps to generate the required token:

  1. Get the script (quickstart.py) to generate the G Suite Reports access token. Click here to download it.
  2. Move the script to <any_directory>/devo-collectors/gsuite-reports/credentials.
  3. Check if quickstart.py and credentials-gsuite-reports.json are available on the same directory (<any_directory>/devo-collectors/gsuite-reports/credentials).
  4. Run the command below to install the Google Auth API library on <any_directory>/devo-collectors/gsuite-reports/credentials

    $ pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
  5. Run the command below on <any_directory>/devo-collectors/gsuite-reports/credentials to create the token.pickle file. A Google consent window will prompt asking for permissions. Follow the instructions and allow the application.

    $ python quickstart.py

Note that quickstart.py is a third-party script from Google that is no supported by Devo.

Run the collector

This data collector can be run in any machine that has the Docker service available because it should be executed as a docker container. The following sections explain how to prepare all the required setup for having the data collector running.

Structure

The following directory structure will be required as part of the setup procedure (it can be created under any directory):

<any_directory>
└── devo-collectors/
    └── gsuite-reports/
          ├── certs/
          │   ├── chain.crt
          │   ├── <your_domain>.key
          │   └── <your_domain>.crt
          ├── credentials/
          │   └── credentials-gsuite-reports.json
          │   └── token.pickle
          └── config/
                  └── config-gsuite-reports.yaml

Devo credentials

In Devo, go to Administration → Credentials → X.509 Certificates, download the Certificate, Private key and Chain CA and save them in /devo-collectors/certs. Learn more about security credentials in Devo here. 

Editing the config-gsuite-reports.yaml file

In the config-gsuite-reports.yaml file, replace the <short_unique_identifier> placeholder by the required value.

config-gsuite.yaml

globals:
  debug: false                                                     # <- Setup as True or False for debugging mode
  id: not_used
  name: gsuite
  persistence:                                                     # <- Persistence setup filesystem
    type: filesystem
    config:
      directory_name: state                                        # <- Persistence directory
outputs:
  devo_1:
    type: devo_platform
    config:
      address: eu.elb.relay.logtrust.net                           # <- Devo platform address EU/US
      port: 443
      type: SSL
      chain: chain.crt
      cert: <your_domain>.crt                                      # <- Please, replace with the certificate from your Devo domain (Administration>Credentials>x.509)
      key: <your_domain>.key                                       # <- Please, replace with the certificate from your Devo domain (Administration>Credentials>x.509)
inputs:
  gsuite_reports:
    id: <short_unique_identifier>                                  # The value of this field will be used internally for having independent persistence areas
    enabled: false                                                 # <- As the G Suite reports shares the same config as G Suite alerts, in order to use the config only for alerts. Reports is disabled.
    requests_per_second: 5                                         # <- Setting up requests per second. 5 recommended.
    autoconfig:
      enabled: false                                              
      refresh_interval_in_seconds: 600                             # <- Time wait in second between requests - 600s recommended.
    credentials:
      token_pickle_filename: token.pickle                          # <- token.pickle file name
    services:                                                      # <- List with the Alerts that you want to collect
      # Request frequency values are in seconds
      # Default values recommended. 
      access_transparency:
        request_period_in_seconds: 60                              # <- Controls waiting time for to the next request
        start_time: "9999-12-31T23:59:59.999Z"
      admin:
        request_period_in_seconds: 60
        start_time: "9999-12-31T23:59:59.999Z"                     # <- Setting a start time for data collection. Format sample supportted "YYYY-mm-ddTHH:MM:SS.sssZ"
      calendar:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      login:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      chat:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      drive:
        request_period_in_seconds: 60
        start_time: "9999-12-31T23:59:59.999Z"
      gcp:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      gplus:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      groups:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      groups_enterprise:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      jamboard:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      meet:
        request_period_in_seconds: 60
        start_time: "9999-12-31T23:59:59.999Z"
      mobile:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      rules:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      saml:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      token:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"
      user_accounts:
        request_period_in_seconds: 3600
        start_time: "9999-12-31T23:59:59.999Z"

The start_time fields are optional. If you would like to establish any value, the required format is 0000-00-00T00:00:00.000Z

Download the Docker image

The collector should be deployed as a Docker container. Click here to download the Docker image of the collector as a .tgz file.

Use the following command to add the Docker image to the system:

gunzip -c collector-gsuite-docker-image.tgz | docker load

Once the Docker image is imported, it will show the real name of the Docker image (including version info).

The Docker image can be deployed on the following services:

  • Docker

  • Docker Compose

Docker

Execute the following command on the root directory <any_directory>/devo-collectors/gsuite-reports/

docker run \
--name collector-gsuite-reports \
--volume $PWD/certs:/devo-collector/certs \
--volume $PWD/config:/devo-collector/config \
--volume $PWD/state:/devo-collector/state \
--env CONFIG_FILE=config-gsuite-reports.yaml \
--rm -it docker.devo.internal/collector/gsuite:<version>

Replace <version> with the proper version.

Docker Compose

The following Docker Compose file can be used to execute the Docker container. It must be created in the <any_directory>/devo-collectors/gsuite-reports/ directory.

docker-compose.yaml

version: '3'
services:
  collector-gsuite-reports:
    build:
      context: .
      dockerfile: Dockerfile
    image: docker.devo.internal/collector/gsuite:${IMAGE_VERSION:-latest}
    container_name: collector-gsuite-reports
    volumes:
      - ./certs:/devo-collector/certs
      - ./config:/devo-collector/config
      - ./state:/devo-collector/state
    environment:
      - CONFIG_FILE=${CONFIG_FILE:-config-gsuite-reports.yaml}

To run the container using docker-compose, execute the following command from the <any_directory>/devo-collectors/gsuite-reports/ directory:

IMAGE_VERSION=<version> docker-compose up -d

Replace <version> with the proper version.

G Suite Report lag times

The lag times in this table show how long it can take before data for specific Admin console reports and audit logs are available.

Item name

Report name

Lag time

Highlights

Gmail

Gmail report

1-3 days

Drive

Drive report

1-3 days

Hangouts

Hangouts report

1-3 days

Google+

Google+ report

1-3 days

Calendar

Calendar report

1-3 days

Document Link Shared Status

Drive report

1-3 days

Security

External Link Shared Files

Drive report

1-3 days

External Link Shared Files

Security report

1-3 days

Less Secure Apps Access

Security report

1-3 days

2-Step Verification Enrollment

2SV report

1-3 days

Aggregate reports

Accounts

Accounts report

1-3 days

Gmail

Gmail report

1-3 days

Drive

Drive report

1-3 days

Google+

Google+ report

1-3 days

Mobile

Mobile report

1-3 days

Apps usage activity

Files added

Drive report

1-3 days

Total Emails

Gmail report

1-3 days

Total Storage Used (MB)

Quota report

1-3 days

Audit

Admin

Admin audit

almost real-time (couple of minutes)

Login

Login audit

1-2 days

Drive

Drive audit

almost real-time (a couple of minutes)

Calendar

Calendar audit

tens of minutes (can also go up to a couple of hours)

Jamboard

Jamboard audit

1-3 days

Google+

Google+ audit

1-3 days

Chat

Chat audit

1-3 days

Meet

Meet audit

almost real-time (a couple of minutes)

Voice

Voice audit

1-3 days

Mobile devices

Devices audit

up to a few hours

SAML

SAML audit

up to a few hours

LDAP

LDAP audit

1-3 days

Token

Token audit

a couple of hours

Groups

Groups audit

tens of minutes (can also go up to a couple of hours)

User accounts

User accounts audit

tens of minutes

Access Transparency

Access Transparency audit

almost real-time (a couple of minutes)

Email log search

Email audit

1-3 days

Retrieving report or audit log data for older dates or a wide time range might take so long that, by the time results are available, the most recent log data might no longer be fresh. For tools that require real-time monitoring, use a short time range. Many products listed above (such as Gmail and Google Drive) are relevant for G Suite only, and not for other Google services, such as Cloud Identity.

Learn more about limits and quotas and lag times.

Activeboards

Click here to download a preconfigured Activeboard that makes use of this collector and try in your Devo domain.

To start working with it, follow these instructions:

  1. Create a new Activeboard in your domain. Learn how to do it here.

  2. In Edit mode, click the ellipsis button and select Edit raw configuration.

  3. Open the downloaded file, select all the text, and copy it into the clipboard.

  4. Paste the contents of the file in the raw editor. Make sure you replace the existing configuration completely.

  5. Click Save changes. The Activeboard should show up immediately.

Disclaimer

The API limits the number of requests for your APIs Console project. The API project's maximum number of requests per second (project QPS) is 5 QPS and the maximum number of requests per day (project QPD) is 150,000 QPD across the account. If these limits are exceeded, the server returns an HTTP 503 status code.

You might find that your Admin console reports and audit logs don’t show the latest data, because reports don’t reflect real-time data. The lag times in the table below show how long it can take before data for specific Admin console reports and audit logs is available. Some reports might take longer to display updated information. 

There's a small chance that reports and audit logs for some events will be delayed beyond the specified times below. In very rare cases, events may not be reported.

Download as PDF

Did you find what you were looking for?

If not, please let us know what you need. Your feedback will help us to improve.

PREVIOUS
G Suite Alerts collector
NEXT
Microsoft Azure collector

Export

See what Devo can do for you. Request a demo!
Discover what's new (Release notes)
  • Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
  • +1 888 6830910 (USA)
  • +34 900 838 880 (Spain)
Copyright © 2019 Legal Terms Privacy Policy Cookies Policy

Powered by Confluence and Scroll Viewport