Working in the search window
Once you open a data table, you are redirected to the search window, where all the events corresponding to the tag you have selected appear arranged in rows and columns, forming the data table. This is where you can start to query your data, apply operations using the set of tools in the toolbar and customize the aspect of the data table, rearranging columns, filtering the data, etc.
The search window contains the following elements:
- Event timeline
- Time range selector
- Search window toolbar
- Applied query operations
- Data table
This useful graph shows a count of the queried events over the period of time set in the From and To fields of the time range selector.
The data counts represented in the timeline are plotted before the actual events are loaded in the browser. To avoid overloading the browser's memory, not all the events in the data table are downloaded to the browser. Instead, Devo download events in interval blocks within the time range selected. This is important to understand, especially with respect to using the Get Server Counts and Event Loading Indicator described below.
The timeline is a dynamic graph and gives you the ability to:
- Hover over the timeline to show the count of events at a specific time.
- Click and drag the mouse across a segment of the timeline to display only the event count and data for that period, narrowing the range of analysis. Use the Back button to go back to previously selected periods.
- Click on the timeline to jump to events from that date/time in the data table. In this way, you can use the timeline to navigate the events in the table. If the events from the selected date/time have not yet been downloaded to the browser, this will download them. When this occurs, a blue band appears in the timeline indicating those events that are being downloaded to the browser. Alternatively, you can use the table scroll bar to download events to the data table.
The following table describes the settings above the timeline:
This determines the intervals at which event counts are totaled and plotted on the timeline. When you hover over the timeline, tooltips appear reporting event totals. These point along the timeline are determined by the Events per setting. Auto sets the interval based on the query's current time range. Use this setting if you want to plot event counts at a specific interval.
This applies a logarithmic scale to the y-axis of the timeline chart instead of the default scale, which uses uniform intervals of units. This can be especially helpful when outlying data is causing significant spikes or dives, distorting your ability to visualize the detail of the timeline.
This toggle appears after applying a filter to your data. When your data is filtered, the green line automatically adjusts to represent the number of events with the filter applied. Activate this toggle to display a comparison between the count of filtered events (green line) and the full count of events (yellow line).
Get server counts
This button appears after applying a filter to your data. Select it to plot the real count of events in the timeline after applying a filter.
When you apply a filter, segments of the timeline may appear as dotted lines, indicating that the counts are actually extrapolated values for those subintervals that have not been downloaded to the browser. Click this button to obtain the actual counts for the dotted segments. The line will change from dotted to continuous.
Note that this doesn’t mean the actual events are downloaded to the browser, just that the real event count is reflected in the timeline.
The timeline appears embedded at the top of the search window by default. You can pop it out of the window so you can place it freely in your screen, or you can close it. After closing it, you can open it again selecting Additional Tools → Query Info → Toggle Chart in the search window toolbar.
Event loading indicator
Devo automatically controls how events are loaded in order to maintain optimal browser performance while at the same time fulfilling user requests for viewing and working with their data.
This reports what percentage of the query's time range has been loaded in the browser so far. For example, if the time range is set to 24 hours and 12 hours of data has been loaded, the progress indicator will report 50%.
Click the indicator to open the Event Loading Preferences. This shows you a more detailed summary of the event loading status and gives you access to some preferences that give you greater control over how events are loaded.
Exercise caution when modifying these preferences. By forcing Devo to load and maintain large amounts of data in the browser, you are likely to experience performance degradation and even browser failure.
Here's a description of these preferences:
|Smart event loading||This is the default behavior. When on, it loads and manages a subset of the query's events to maintain browser performance and satisfy the user's requests for data. Turn this off to stop loading the query's remaining events into the browser.|
|Load all events||Turn ON to load all of the query's events in the selected date range. Exercise caution with this setting because when turned on, there is a risk of overloading the browser and causing it to crash.|
|Load all only when sorting||To sort a column, the data needs to be downloaded to the browser in order to take into account all of the column's values. Turn this setting on to load all events only when you sort the contents of a column.|
|Load all only when chart-building||To build a chart that plots data from individual events, all of the query's events need to be downloaded to the browser. Turn this setting on to load all events only when you build one of these types of charts. Examples of charts that are built using individual event data (not grouped events with aggregate functions) are scatter charts and some world maps.|
|Retain all events||By default, Devo employs a memory management process that can remove events from the browser's memory in order to make room for events that are more relevant. Turn this setting on to prevent loaded events from being removed from the browser's memory.|
|Set thresholds. Event count, Browser memory (MB)||This becomes available when you turn Retain all events on. Turn on Set thresholds to enforce an upper limit to the amount of data to load. This amount can be expressed in number of events, Event count, or in MB used by the query in the Browser memory. If you define upper limits in both fields, event loading will stop when either one is met.|
The Event Loading Status details include:
|Progress||This reports what percentage of the query's time range has been loaded in the browser so far. For example, if the time range is set to 24 hours and 12 hours of data has been loaded, the progress indicator will report 50%.|
|Events loaded||This reports the number of events loaded and the corresponding use of memory.|
|Gaps remaining||Devo loads a query's events to the browser in blocks. This leaves gaps in the event timeline that contain the missing events. This tells you how many gaps exist in the current query.|
Time range selector
These tools allow you to apply filters by time. To narrow your search, you can select a specific time range. Use extended periods to analyze long-term patterns like an advanced persistent threat. You can perform the following actions:
Set a new time interval
Select the new interval in the From and To fields, then click Apply Interval to update the data table. Click the Back button to return to the previous time setting.
Activate or deactivate real-time data flow
Click the spinning clock icon to suspend or reestablish the flow of real-time data. In some cases of extremely large volumes of data, real-time data flow will stop automatically and a warning message will be shown above the table. This is done to prevent the browser from crashing.
Users with the necessary permissions can determine if real-time data flow is active or inactive by default when users run searches. Go to Preferences → Account Preferences → Global to access this setting. For more information, see Domain preferences.
Apply previously used time intervals
Use the Back button to apply previously selected time intervals in your query.
Additionally, the Time Interval History tool allows you to easily apply previously selected time periods in the current or other data tables, to facilitate the analysis of data over time. The results can be used in reports or to create dashboard data sources from different time intervals.
Select the required interval in the Available Time Intervals area. When there are multiple active queries, checkboxes will be available to let you apply the interval to more than one query. The current query is selected by default.
Search window toolbar
This toolbar offers a rich set of tools to work with the table data including grouping, aggregation, data download, and more. Hover over each icon to see its tooltip. These are the default tools displayed in the toolbar:
|Time interval history||Apply time intervals previously set in any active queries. Learn more about this in the previous section.|
|Search column layout||Hide or show columns in the data table to work only with the necessary ones. Check out Hide and show columns to learn how to do it.|
|View selected events||Check information about specific events in the data table.|
|Column operations||Access a set of operations to edit and arrange the table columns. Learn more about these operations in the articles in Modifying the column layout.|
|Toggle query editor||Open a query editor where you can build or modify the current query using LINQ.|
|Toggle search tree||Display a treemap representing all the operations applied to the original data table. See the below section to learn more.|
|New alert definition||Define alerts to monitor active queries and receive notifications when certain conditions occur. Check the instructions in Configuring alerts.|
|Aggregation||Perform aggregation operations on table data that has been already grouped by time interval.|
|Group||Group data to get all the different row value combinations of the grouped columns.|
|OR filter||You can use an OR filter to get records that have any of the values for a given property.|
|Filter||Filter data to retrieve certain values or exclude them from the table.|
|Create column||Create columns in your data tables based on other table data.|
|CyberChef||Use this tool to analyze and decode your data before building your query. Learn more in Manipulate your data using CyberChef.|
|Download||Download your query in different formats. Go to Download a query for further information.|
|Server mode||Check this box to activate server mode in your searches. The default search mode is recommended for small queries, while server mode is recommended for queries that process a large amount of data. Learn more in Best practices for data search. You can set server mode as default in your User preferences.|
|Additional tools||Access a set of additional operations that do not appear in the default toolbar.|
|Close search||Close the current search.|
You can customize the default toolbar configuration to provide quick access to the tools that you use most frequently. You can perform the following actions to customize your toolbar as needed.
Add new tools to the toolbar and manage them
Select Additional tools and navigate to the required tool in the list. Point its icon and when the cursor becomes a move pointer, drag the tool to the table toolbar. The icon will appear as the first tool in the toolbar.
To change the order of the tools, select the tool you want to move and drag it to the new position in the toolbar. To remove an icon from the table toolbar, select, hold and move it to the dynamic trash bin that appears, as seen above.
Save and restore the toolbar configuration
After adding the tools you use the most to the toolbar and move them as required, select Additional tools → Panel → Save Panel. The custom toolbar will appear the next time you access the search window, no matter the data table you open. Select Additional tools → Panel → Reset Panel to restore the original toolbar configuration.
Applied query operations
Any operations you apply on the table when building your query will appear listed above the data table. This way, you can easily consult the operations affecting the data, modify them, or undo operations. You can go back to any of the operations applied and start a new path of actions from there.
Select the Toggle Search Tree tool to display a visual record of all the modifications applied to the original data table. The actions and their sequences are displayed in a tree, known as the search tree. Select any point in the tree to display the query results at that point in the sequence of modifications. If you select an operation in a branch different from the current one, that path will be shown in the applied operations bar.
Each tab in the bar/node in the tree appears in a color that denotes the type of action taken. The color code is described here:
See Build a query using the search window tools to learn more about how to transform and work with query data.
In the data table, each row represents an event and each column represents a data value correctly recognized by Devo. If the data is not separated by several columns or is shown in the unknown tag structure of the search view, it is normally due to missing or incorrect tags. Learn more about tags here.