Monitor intranet traffic to dangerous websites

In this guided tutorial, you will generate a Graph diagram using firewall log data in order to visualize and analyze access to dangerous sites from within your company's Intranet.

There are two phases explained below:

• Build and enhance the query
• Generate the Graph diagram

This video takes you through the same tutorial if you want a quick overview of the process.

Build and enhance the query

This is the query used in the tutorial video. You can go to Data Search → Free Text Query and paste it, or follow the steps below to build it.

from firewall.paloalto.traffic
where serial = "012001000758",
ispublic(dstIp)
select mmcoordinates(dstIp) as dstServerCoordinates,
`lu/Threat-Malware-by-IP/threat`(dstIp) as Threat
where isnotnull(Threat)
select `lu/IPNames/Full Name`(srcIp) as UserName
group every 30m by UserName, dstIp, dstServerCoordinates, Threat, dstCountry
every 1h
select count() as count,
avg(bytes) as bytes

1. Go to Data Search and open the firewall.paloalto.traffic data table.
2. First, we'll filter the data to show only the events associated with the firewall whose traffic we want to analyze. Select Filter in the toolbar, then select the Equal operation. Add the Serial column and 012001000758 as arguments, then click the Filter button. Remember that you have to click the pencil icon to enter the Serial value as free text.
   
   
3. Now we'll apply a filter to show only destination IP addresses that are public. Select Filter in the toolbar, then select the Is Public IPv4 operation. Add the dstIp column as the argument, then click the Filter button.
   
   
4. In order to enhance the geographic information for the events (so that later we can view sites on a map), we'll use a built-in operation that returns the geographic coordinates based on the destination IP address. Select Create Column in the toolbar, then select Geolocated Coordinates (mmcoordinates) as the operation. Select the dstIp column as the argument and name the column dstServerCoordinates. Click the Create Column button.
   
   
5. Now we'll use another built-in operation that uses an internal lookup table that lists IP addresses that are known threats along with the type of threat they pose. Select Create Column in the toolbar, then select Search Threat-Malware-by-IP for threat as the operation. Select the dstIp column as the argument and name the column Threat. Click the Create Column button.
   
   
6. The newly created Threat column reports null for any IP address not identified as a threat. Since we only want to analyze accesses to dangerous sites, let's filter those events out of our query. Select Filter in the toolbar, then select Is not null as the operation. Add the Threat column as an argument, then click the Filter button.
   
   
7. Even though Intranets usually assign dynamic IP addresses, for the purposes of this example we're going to assume the source IP address are fixed. Download the CSV file previously mentioned and upload it as a lookup table.
   Now select Create Column in the toolbar, then under Operation click custom or all and select the Full Name lookup column as the operation. Select the srcIp column as the argument and name the new column UserName.
   
   
8. Now that we have generated new columns of data and applied the relevant filters, we can proceed to group our data by time and choose the more relevant fields to use in our graph. Select Group in the toolbar and select 1 hour as the grouping period. Now add UserName, dstIP, dstServerCoordinates, Threat and dstCountry as the arguments. Click the Group By button.
   
   
9. We want to include a couple of calculated metrics in our graph; the number of connections in each time period, and the average number of bytes per connection. Select Aggregation in the toolbar, and select Count as the Aggregation and name the new column count. Select Aggregation in the toolbar again, and this time select Average as the Aggregation and name the new column bytes.
   
   
10. At this point, confirm that your data table looks like this. If it does, we can proceed to generate the Graph diagram.
    
    

Generate the Graph diagram

1. From the toolbar, click the gear icon and select Charts → Diagrams → Graph diagram. This opens the graph builder. Drag the UserName, dstIp and Threat column headers to the builder canvas, then add the count and bytes columns as metrics. 
   
   
2. Add the dstServerCoordinates and dstCountry columns to the position and color attributes of dstIp to geolocate the servers in the map and assign them colors based on the corresponding destination countries.
   
   
3. Let's customize the colors and icons of the elements in the graph. To do this, select the element's node type, then make your changes in the properties bar that appears in the builder. For example, let's make the UserName nodes use a person icon. Select the settings node type above the UserName node and set the Icon to user_profile_avatar_man_male2.
   
   
4. Click Apply. Activate the Map mode option in the graph menu to geolocate your nodes in the map.
   
   

For more details on how use the settings to view the information in different ways, see Working in the graph diagram.