Manual Chapter : Using the Request Logging Profile

Applies To:

Show Versions Show Versions

BIG-IP AAM

  • 14.1.3, 14.1.2, 14.1.0, 14.0.1, 14.0.0, 13.1.5, 13.1.4, 13.1.3, 13.1.1, 13.1.0, 13.0.1, 13.0.0
Manual Chapter

Overview: Configuring a Request Logging profile

The Request Logging profile gives you the ability to configure data within a log file for HTTP requests and responses, in accordance with specified parameters.

Task summary

Perform these tasks to log HTTP request and response data.

About the Request Logging profile

Many sites perform traffic analysis against the HTTP log files that their web servers generate. With the Request Logging profile, you can specify the data and the format for HTTP requests and responses that you want to include in a log file. If you prefer, you can tailor the information that appears in the logs so that the logs work seamlessly with whatever analysis tools you use for your origin web server's HTTP log files.

Standard log formats

Log headers appear in the lines at the top of a log file. You can use log headers to identify the type and order of the information written to each line in the log file. Some log analysis software also uses log headers to determine how to parse a log file.

There are three common conventions for log headers shown here.

Convention Description
No header line Apache™ web servers use this option. By default, Apache web servers write access logs in a format that is identical to the NCSA Common format.
NCSA Common or Combined headers Netscape® servers, and their descendants (such as the iPlanet™ Enterprise Server) write a log header line that is unique to this family of servers. These servers generally use either the NCSA Common or Combined log format, and the log header lines are composed of keywords. For example:
#format=%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] ....
W3C headers Most Microsoft® Internet Information Services (IIS) web servers write log files in the extended log file format, which is defined by a W3C working draft.

The logging information that is commonly used by origin web servers consists of the following conventions:

  • NCSA Common (no log header)
  • NCSA Common (Netscape log header)
  • NCSA Combined (no log header)
  • NCSA Combined (Netscape log header)
  • W3C Extended

NCSA Common log format example

This is the NCSA Common log format syntax:

host rfc931 username [date:time UTC_offset] 
"method URI?query_parameters protocol" status bytes

Here is an example that uses this syntax:

125.125.125.2 - - [03/Apr/2011:23:44:03 -0600] 
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045

NCSA Combined log format example

This is the NCSA Combined log format syntax:

host rfc931 username [date:time UTC_offset] 
"method URI?query_parameters protocol" status bytes 
"referrer" "user_agent" "cookie"

Here is an example that uses this syntax:

125.125.125.2 - - [03/Apr/2011:23:44:03 -0600] 
"GET /apps/example.jsp?sessionID=34h76 HTTP/1.1" 200 3045
"http://www.siterequest.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" 
"UserID=ssarette;Category=PDA;Selection=Various"

W3C Extended log format example

This is the W3C extended log format syntax:

date time rfc931 username host method URI query_parameters 
status bytes request_length time_taken protocol user_agent cookie referrer 

Following is an example that uses this syntax:

2011-04-03 23:44:03 205.47.62.112 - 125.125.125.2 
GET /apps/example.jsp sessionID=34h76 200 3045 124 138 
HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0 
UserID=ssarette;Category=PDA;Selection=Various http://www.siterequest.com

Request Logging profile settings

With the Request Logging profile, you can specify the data and the format for HTTP requests and responses that you want to include in a log file.

General Properties

Setting Value Description
Name No default Specifies the name of the profile.
Parent Profile Selected predefined or user-defined profile Specifies the selected predefined or user-defined profile.

Request Settings

Setting Value Description
Request Logging Disabled Enables logging for requests.
Template   Specifies the directives and entries to be logged.
HSL Protocol UDP Specifies the protocol to be used for high-speed logging of requests.
Pool Name None Defines the pool associated with the virtual server that is logged.
Respond On Error Disabled Enables the ability to respond when an error occurs.
Error Response None Specifies the response text to be used when an error occurs.
For example, the following response text provides content for a 503 error.
<html>
 <head>
  <title>ERROR</title>
 </head>
 <body>
  <p>503 ERROR-Service Unavailable</p>
 </body>
</html>
            
Close On Error Disabled When enabled, and logging fails, drops the request and closes the connection.
Log Logging Errors Disabled Enables the ability to log any errors when logging requests.
Error Template None Defines the format for requests in an error log.
HSL Error Protocol UDP Defines the protocol to be used for high-speed logging of request errors.
Error Pool Name None Specifies the name of the error logging pool for requests.

Response Settings

Setting Value Description
Response Logging Disabled Enables logging for responses.
Log By Default Enabled Defines whether to log the specified settings for responses by default.
Template None Specifies the directives and entries to be logged.
HSL Protocol UDP Specifies the protocol to be used for high-speed logging of responses.
Pool Name None Defines the pool name associated with the virtual server that is logged.
Log Logging Errors Disabled Enables the ability to log any errors when logging responses.
Error Template None Defines the format for responses in an error log.
HSL Error Protocol UDP Defines the protocol to be used for high-speed logging of response errors.
Error Pool Name None Specifies the name of the error logging pool for responses.

Request Logging parameters

This table lists all available parameters from which you can create a custom HTTP Request Logging profile. These are used to specify entries for the Template and Error Template settings For each parameter, the system writes to the log the information described in the right column.

Table 1. Request logging parameters
Parameter Communication method Log file entry description
BIGIP_BLADE_ID Request and Response An entry for the slot number of the blade that handled the request.
BIGIP_CACHED Response An entry of Cached status: true, if the response came from BIG-IP® cache, or Cached status: false, if the response came from the server.
BIGIP_HOSTNAME Request and Response An entry for the configured host name of the unit or chassis.
CLIENT_IP Request and Response An entry for the IP address of a client, for example, 192.168.74.164.
CLIENT_PORT Request and Response An entry for the port of a client, for example, 80.
DATE_D Request and Response A two-character entry for the day of the month, ranging from 1 (note the leading space) through 31.
DATE_DAY Request and Response An entry that spells out the name of the day.
DATE_DD Request and Response A two-digit entry for the day of the month, ranging from 01 through 31.
DATE_DY Request and Response A three-letter entry for the day, for example, Mon.
DATE_HTTP Request and Response A date and time entry in an HTTP format, for example, Tue, 5 Apr 2011 02:15:31 GMT.
DATE_MM Request and Response A two-digit month entry, ranging from 01 through 12.
DATE_MON Request and Response A three-letter abbreviation for a month entry, for example, APR.
DATE_MONTH Request and Response An entry that spells out the name of the month.
DATE_NCSA Request and Response A date and time entry in an NCSA format, for example, dd/mm/yy:hh:mm:ss ZNE.
DATE_YY Request and Response A two-digit year entry, ranging from 00 through 99.
DATE_YYYY Request and Response A four-digit year entry.
HTTP_CLASS Neither Request nor Response: Deprecated The name of the httpclass profile that matched the request, or an empty entry if a profile name is not associated with the request.
HTTP_KEEPALIVE Request and Response A flag summarizing the HTTP1.1 keep-alive status for the request:: aY if the HTTP1.1 keep-alive header was sent, or an empty entry if not.
HTTP_METHOD Request and Response An entry that defines the HTTP method, for example, GET, PUT, HEAD, POST, DELETE, TRACE, or CONNECT.
HTTP_PATH Request and Response An entry that defines the HTTP path.
HTTP_QUERY Request and Response The text following the first ? in the URI.
HTTP_REQUEST Request and Response The complete text of the request, for example, $METHOD $URI $VERSION.
HTTP_STATCODE Response The numerical response status code, that is, the status response code excluding subsequent text.
HTTP_STATUS Response The complete status response, that is, the number appended with any subsequent text.
HTTP_URI Request and Response An entry for the URI of the request.
HTTP_VERSION Request and Response An entry that defines the HTTP version.
NCSA_COMBINED Response An NCSA Combined formatted log string, for example, $NCSA_COMMON $Referer ${User-agent} $Cookie.
NCSA_COMMON Response An NCSA Common formatted log string, for example, $CLIENT_IP - - $DATE_NCSA $HTTP_REQUEST $HTTP_STATCODE $RESPONSE_SIZE.
RESPONSE_MSECS Response The elapsed time in milliseconds (ms) between receiving the request and sending the response.
RESPONSE_SIZE Response An entry for the size of response in bytes.
RESPONSE_USECS Response The elapsed time in microseconds (µs) between receiving the request and sending the response.
SERVER_IP Response The IP address of the pool member to which the HTTP request was sent, for example, 10.10.0.1.
SERVER_PORT Response The port of the pool member to which the HTTP request was sent, for example, 80.
SNAT_IP Response An entry for the self IP address of the BIG-IP-originated connection to the server when SNAT is enabled, or an entry for the client IP address when SNAT is not enabled.
SNAT_PORT Response An entry for the port of the BIG-IP-originated connection to the server when SNAT is enabled, or an entry for the client port when SNAT is not enabled.
TIME_AMPM Request and Response A twelve-hour request-time qualifier, for example, AM or PM.
TIME_H12 Request and Response A compact twelve-hour time entry for request-time hours, ranging from 1 through 12.
TIME_HRS Request and Response A twelve-hour time entry for hours, for example, 12 AM.
TIME_HH12 Request and Response A twelve hour entry for request-time hours, ranging from 01 through 12.
TIME_HMS Request and Response An entry for a compact request time of H:M:S, for example, 12:10:49.
TIME_HH24 Request and Response A twenty-four hour entry for request-time hours, ranging from 00 through 23.
TIME_MM Request and Response A two-digit entry for minutes, ranging from 00 through 59.
TIME_MSECS Request and Response An entry for the request-time fraction in milliseconds (ms).
TIME_OFFSET Request and Response An entry for the time zone, offset in hours from GMT, for example, -11.
TIME_SS Request and Response A two-digit entry for seconds, ranging from 00 through 59.
TIME_UNIX Request and Response A UNIX time entry for the number of seconds since the UNIX epoch, for example, 00:00:00 UTC, January 1st, 1970.
TIME_USECS Request and Response An entry for the request-time fraction in microseconds (µs).
TIME_ZONE Request and Response An entry for the current Olson database or tz database three-character time zone, for example, PDT.
VIRTUAL_IP Request and Response An entry for the IP address of a virtual server, for example, 192.168.10.1.
VIRTUAL_NAME Request and Response An entry for the name of a virtual server.
VIRTUAL_POOL_NAME Request and Response An entry for the name of the pool containing the responding server.
VIRTUAL_PORT Request and Response An entry for the port of a virtual server, for example, 80.
VIRTUAL_SNATPOOL_NAME Request and Response The name of the Secure Network Address Translation pool associated with the virtual server.
WAM_APPLICATION_NAM Response An entry that defines the name of the BIG-IP® acceleration application that processed the request.
WAM_X_WA_INFO Response An entry that specifies a diagnostic string (X-WA-Info header) used by BIG-IP acceleration to process the request.
NULL   Undelineated strings return the value of the respective header.