PREVIOUS proxy.bluecoat

NEXT Web

proxy.squid

The logs generated by proxy Squid are marked with the  proxy.squid.type.server tag.

Squid generates the following log types:

  • Access log
  • Cache log
  • Storage log
  • ICAP log

Tag structure

The element server is free, but mandatory. It takes the value chosen by the you to identify the proxy server and it must always be used.

The element type is fixed and identifies the type and format of the sent event. It takes one of the following values (that correspond to the different types of logs): 

  • access-clf
  • access-combined
  • access-lt
  • access-squid
  • access-squidmime
  • store
  • cache

For more information on how tags work, please check the Introduction to tags article.

Access log

There is an event in the access log for each request processed by the server. Both the event and the destination can be configured with directives. 

  • logformat gives name to the format
  • acesslog indicates the destination and format you want to use for each event

Both policies can be invoked multiple times. This way you can save the same event at multiple sites, each with a different format.

The access log has many similarities with the web.apache log. The event content, both format and fields, can be controlled in detail.

Devo support 5 access event formats, 4 standard formats and an additional one defined by Devo for users who need more details. Each format is identified with a type value.

  • access-clf for the Common Log Format (CLF). The directive to define this (default) format is:

    CLF

    logformat common %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st %Ss:%Sh
  • access-combined for the NCSA exended/combined Log Format. The directive to define this format is:

    Combined

    logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
  • access-squid for the default format used by Squid. The directive to define this format is:

    squid

    logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt
  • access-squidmime is the access-quid format including the request and response HTTP headers. The directive to define this format is:

    squid mime

    logformat squidmime %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %un %Sh/%<A %mt [%>h] [%<h]
  • access-lt for the format defined by Devo adding more details to the standard formats listed above. The directive to define this format is:

    Devo Access

    logformat access-lt %{%F:%T%z}tl %>a:%>p %ui %un %<A "%rm %ru HTTP/%rv" "%{Referer}>h" "%{User-Agent}>h" "%{Cookie}>h" %>Hs %>st %tr %<st %mt %Ss:%Sh

Cache log

  • It saves the error and debug messages generated by the proxy server.
  • It is enabled by default.
  • This log destination file is indicated in the Squid definition with cache_log.
  • The log has a fixed format that cannot be configured. The events must be sent with the type element as storage

ICAP log

  • Squid allows the integration with a ICAP sever (e.g. for content filtering, antivirus, etc.). This log contains a summary of the activity exchanged through ICAP.
  • The log format can be configured with directives.
  • By default it comes with the type set as icap_squid:

icap_squid log format

logformat icap_squid %ts.%03tu %6icap::tr %>a %icap::to/%03icap::Hs %icap::<size %icap::rm %icap::ru% %un -/%icap::<A -

Sending to Devo via files

You can configure Squid to write the logs to file and rely on other tools (like rsyslog or syslog-ng) to send the events.

See below example of a log configuration for Squid:

/etc/squid/squid.conf file extract

cache_log /var/log/squid3/cache.log
cache_store_log /var/log/squid3/store.log
 
#Choose one or more formats for the access log
access_log /var/log/squid3/accessSquid.log squid
access_log /var/log/squid3/accessMime.log squidmime
access_log /var/log/squid3/accessClf.log common
access_log /var/log/squid3/accessCombined.log combined
access_log /var/log/squid3/access-lt.log access-lt

You can use the following rsyslog configuration file:

/etc/rsyslog.d/45-squid.conf file

$template squid,"<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%"
 
# File access
$InputFileName /var/log/squid3/cache.log
$InputFileTag proxy.squid.cache.myProxyName:  
$InputFileStateFile stat-file1-SquidCache
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# File access
$InputFileName /var/log/squid3/access-lt.log
$InputFileTag proxy.squid.access-lt.myProxyName:  
$InputFileStateFile stat-file1-SquidAccessLt
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# File access
$InputFileName /var/log/squid3/accessClf.log
$InputFileTag proxy.squid.access-clf.myProxyName:  
$InputFileStateFile stat-file1-SquidAccessClf
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# File access
$InputFileName /var/log/squid3/accessCombined.log
$InputFileTag proxy.squid.access-combined.myProxyName:  
$InputFileStateFile stat-file1-SquidAccessCombined
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# File access
$InputFileName /var/log/squid3/accessSquid.log
$InputFileTag proxy.squid.access-squid.myProxyName:  
$InputFileStateFile stat-file1-SquidAccessSquid
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# File access
$InputFileName /var/log/squid3/accessMime.log
$InputFileTag proxy.squid.access-squidmime.myProxyName:  
$InputFileStateFile stat-file1-SquidAccessSquidMime
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
 
# SSL config for devo secure relay
#$DefaultNetstreamDriver gtls # use gtls netstream driver
#$DefaultNetstreamDriverCAFile /etc/rsyslog.d/ca.crt
#$DefaultNetstreamDriverCertFile /etc/rsyslog.d/user.crt
#$DefaultNetstreamDriverKeyFile /etc/rsyslog.d/user.key
#$ActionSendStreamDriverMode 1 # require TLS for the connection
#$ActionSendStreamDriverAuthMode x509/name
#$ActionSendStreamDriverPermittedPeer collector
 
if $syslogtag contains 'proxy.squid.' and $syslogfacility-text == 'local7' then @@DEVO-RELAY:PORT;squid
:syslogtag, contains, "proxy.squid." ~
  • Replace DEVO-RELAY and PORT with the server and the port of your Devo In-house Relay. 
  • If you are going to send to a secure relay, uncomment the SSL section of the configuration file.

Other sending mechanisms

  • A log destination, besides being a file, can also be an external process, such as local syslog machine or a ydp/tcp server. To send to an external process, you should include program name and full path. Squid uses a minimal protocol in all the messages sent to the process and controls the logs processing by using each event first byte. The external protocol should be capable of sending to a remote syslog server (e.g. a script using the logger command).

Forwarding to an external process

logfile_daemon /path/to/helper_process
access_log daemon:/var/log/squid/access.log access-lt
  • Another mechanism is to send to the machine local syslog, by specifying the destination as syslog:facility.priority. In order for this mechanism to function, you should apply a rsyslog or syslog-ng filter for this specific facility.priority to label all the events coming through it as proxy.squid.access-lt.myProxyName and send them to Devo remote server.

Forwarding to local syslog

access_log syslog:local4.info access-lt
  • Squid also allows sending the logs to a remote server via TCP/UDC as unformatted plain text. Since it doesn't use syslog protocol, this option is not supported by Devo. 

Have we answered your question?

If not, please contact our technical support team via email by clicking the button below.

CONTACT US

PREVIOUS proxy.bluecoat

NEXT Web