Searching data / Working in the search window / Data enrichment

Data enrichment

About lookup tables

Lookup tables are used to enrich the information in raw data tables by correlating values in the data table with corresponding values in the lookup table. For example, a lookup table containing IP addresses with their geographical addresses may be used to add geographical addresses to a data table containing IP addresses during a query.

Lookup values are added to the virtual data table at query time, as new columns. The original data tables are never modified.
A key value must be selected. This is the column in the lookup table that has values that correspond to values in the data table. In our example, the key column will be the column containing the IP addresses, which exists both in the lookup table and original data table.
Lookup tables can be edited to add, change or delete information.

Use cases

Here are some common use cases that demonstrate how lookup tables can be used.

Converting codes into names

Convert an IP address into a machine name.
Convert an IP address into a geo-localization.

Add values to classify or filter events

Associate an IP to known threats.
Group IP by types of devices: servers, portable computers, printers.

For example, lookup tables can be used to enrich a data table containing information about a manufacturing company's robots.

They can associate robot IDs to factory locations.
They can categorize types of robots but their functions.
They can assign rankings to robots based on measurements in the data table.

Types of lookup tables

In Devo, lookup tables are grouped into four different categories and can be created by uploading a .csv file or using query data.

Source	Lookup table type	Description
External file	Upload	External lookup tables uploaded as a .csv file. External sources may include lists of values, geo-localizations, or extracts from a database. Learn here how to upload external data as a lookup table.
Query data	Static query	These lookup tables are created using query data from a specified period of time. See Create a lookup table from a query to learn more.
	Dynamic query	These lookup tables are fed with new data events every 5 minutes. Rows with duplicated key values will be overwritten. See Create a lookup table from a query to learn more.
	Time range lookup	Both static and dynamic query lookups can be created as a time range lookup. To create these lookups, you must choose a timestamp type column that will dictate the lookup values to be inserted. That is to say, the same entry of your key column must be matched with different results depending on the specified date. Learn more here.

Apart from user-created lookup tables, Devo is pre-installed with a set of threat lookups you can use to detect IP addresses or domains related to potential fraud or malware threats. See threat lookups for more information.

Download as PDF