Manual Chapter : Logging and Reporting

Applies To:

Show Versions Show Versions

BIG-IP APM

  • 14.1.5, 14.1.4, 14.1.3, 14.1.2, 14.1.0
Manual Chapter

Logging and Reporting

Overview: Configuring remote high-speed APM and SWG event logging

You can configure the BIG-IP® system to log information about Access Policy Manager (APM ) and Secure Web Gateway events and send the log messages to remote high-speed log servers.
When configuring remote high-speed logging of events, it is helpful to understand the objects you need to create and why, as described here:
Object
Reason
Pool of remote log servers
Create a pool of remote log servers to which the BIG-IP system can send log messages.
Destination (unformatted)
Create a log destination of Remote High-Speed Log type that specifies a pool of remote log servers.
Destination (formatted)
If your remote log servers are the ArcSight, Splunk, or Remote Syslog type, create an additional log destination to format the logs in the required format and forward the logs to a remote high-speed log destination.
Publisher
Create a log publisher to send logs to a set of specified log destinations.
Log Setting
Add event logging for the APM system and configure log levels for it or add logging for URL filter events, or both. Settings include the specification of up to two log publishers: one for access system logging and one for URL request logging.
Access profile
Add log settings to the access profile. The log settings for the access profile control logging for the traffic that comes through the virtual server to which the access profile is assigned.
Association of remote high-speed logging configuration objects
Associations between remote high-speed logging configuration objects

Task summary

Perform these tasks to configure remote high-speed APM and SWG event logging on the BIG-IP system.
Enabling remote high-speed logging impacts BIG-IP system performance.

Task list

About the default-log-setting

Access Policy Manager (APM) provides a default-log-setting. When you create an access profile, the default-log-setting is automatically assigned to it. The default-log-setting can be retained, removed, or replaced for the access profile. The default-log-setting is applied to user sessions only when it is assigned to an access profile.
Regardless of whether it is assigned to an access profile, the default-log-setting applies to APM processes that run outside of a user session. Specifically, on a BIG-IP® system with an SWG subscription, the default-log-setting applies to URL database updates.

Create a pool of remote logging servers

Before creating a pool of log servers, gather the IP addresses of the servers that you want to include in the pool. Ensure that the remote log servers are configured to listen to and receive log messages from the BIG-IP system.
Create a pool of remote log servers to which the BIG-IP system can send log messages.
  1. At the top of the screen, click
    Configuration
    .
  2. On the Main tab, click
    Local Traffic
    Pools
    .
    The Pool List screen opens.
  3. Click
    Create
    .
    The New Pool screen opens.
  4. In the
    Name
    field, type a unique name for the pool.
  5. Using the
    New Members
    setting, add the IP address for each remote logging server that you want to include in the pool:
    1. Type an IP address in the
      Address
      field, or select a node address from the
      Node List
      .
    2. Type a service number in the
      Service Port
      field, or select a service name from the list.
      Typical remote logging servers require port
      514
      .
    3. Click
      Add
      .
  6. Click
    Finished
    .

Create a remote high-speed log destination

Before creating a remote high-speed log destination, ensure that at least one pool of remote log servers exists on the BIG-IP system.
Create a log destination of the
Remote High-Speed Log
type to specify that log messages are sent to a pool of remote log servers.
  1. On the Main tab, click
    System
    Logs
    Configuration
    Log Destinations
    .
    The Log Destinations screen opens.
  2. Click
    Create
    .
  3. In the
    Name
    field, type a unique, identifiable name for this destination.
  4. From the
    Type
    list, select
    Remote High-Speed Log
    .
    If you use log servers such as Remote Syslog, Splunk, or ArcSight, which require data be sent to the servers in a specific format, you must create an additional log destination of the required type, and associate it with a log destination of the
    Remote High-Speed Log
    type. With this configuration, the BIG-IP system can send data to the servers in the required format.
    The BIG-IP system is configured to send an unformatted string of text to the log servers.
  5. From the
    Pool Name
    list, select the pool of remote log servers to which you want the BIG-IP system to send log messages.
  6. From the
    Protocol
    list, select the protocol used by the high-speed logging pool members.
  7. Click
    Finished
    .

Create a formatted remote high-speed log destination

Ensure that at least one remote high-speed log destination exists on the BIG-IP system.
Create a formatted logging destination to specify that log messages are sent to a pool of remote log servers, such as Remote Syslog, Splunk, or IPFIX servers.
  1. On the Main tab, click
    System
    Logs
    Configuration
    Log Destinations
    .
    The Log Destinations screen opens.
  2. Click
    Create
    .
  3. In the
    Name
    field, type a unique, identifiable name for this destination.
  4. From the
    Type
    list, select a formatted logging destination, such as
    Remote Syslog
    ,
    Splunk
    , or
    IPFIX
    .
    The Splunk format is a predefined format of key value pairs.
    The BIG-IP system is configured to send a formatted string of text to the log servers.
  5. If you selected
    Remote Syslog
    , then from the
    Syslog Format
    list select a format for the logs, and then from the
    High-Speed Log Destination
    list, select the destination that points to a pool of remote Syslog servers to which you want the BIG-IP system to send log messages.
    For logs coming from Access Policy Manager (APM), only the BSD Syslog format is supported.
  6. If you selected
    Splunk
    or
    IPFIX
    , then from the
    Forward To
    list, select the destination that points to a pool of high-speed log servers to which you want the BIG-IP system to send log messages.
  7. Click
    Finished
    .

Create a publisher

Ensure that at least one destination associated with a pool of remote log servers exists on the BIG-IP system.
Create a publisher to specify where the BIG-IP system sends log messages for specific resources.
  1. On the Main tab, click
    System
    Logs
    Configuration
    Log Publishers
    .
    The Log Publishers screen opens.
  2. Click
    Create
    .
  3. In the
    Name
    field, type a unique, identifiable name for this publisher.
  4. For the
    Destinations
    setting, select a destination from the
    Available
    list, and click
    <<
    to move the destination to the
    Selected
    list.
    If you are using a formatted destination, select the destination that matches your log servers, such as Remote Syslog, Splunk, or ArcSight.
  5. Click
    Finished
    .

Configuring log settings for access system and URL request events

Create log settings to enable event logging for access system events or URL filtering events or both. Log settings specify how to process event logs for the traffic that passes through a virtual server with a particular access profile.
  1. On the Main tab, click
    Access
    Overview
    Event Logs
    Settings
    .
    A log settings table screen opens.
  2. Select a log setting and click
    Edit
    or click
    Create
    for a new APM log setting.
    A popup screen opens with General Information selected in the left pane.
  3. For a new log setting, in the
    Name
    field, type a name.
  4. To specify logging, select one or both of these check box options:
    • Enable access system logs
      - This setting is generally applicable. It applies to access policies, per-request policies, Secure Web Gateway processes, and so on. When you select this check box,
      Access System Logs
      becomes available in the left pane.
    • Enable URL request logs
      - This setting is applicable for logging URL requests when you have set up a BIG-IP system configuration to categorize and filter URLs. When you select this check box,
      URL Request Logs
      becomes available in the left pane.
    When you clear either of these check boxes and save your change, you are not only disabling that type of logging, but any changes you made to the settings are also removed.
  5. To configure settings for access system logging, select
    Access System Logs
    from the left pane.
    Access System Logs settings display in the right panel.
  6. For access system logging, from the
    Log Publisher
    list select the log publisher of your choice.
    A log publisher specifies one or more logging destinations.
    The BIG-IP system is not a logging server and has limited capacity for storing, archiving, and analyzing logs. For this reason a dedicated logging server is recommended.
  7. For access system logging, retain the default minimum log level,
    Notice
    , for each option.
    You can change the minimum log level, but
    Notice
    is recommended.
    Access Policy
    Events that occur while an access policy runs.
    Per-Request Policy
    Events that occur while a per-request policy runs.
    ACL
    Events that occur while applying APM access control lists.
    SSO
    Events that occur during single-sign on.
    Secure Web Gateway
    Events that occur during URL categorization on a BIG-IP system with an SWG subscription.
    ECA
    Events that occur during NTLM authentication for Microsoft Exchange clients.
    OAuth
    Events that occur while APM, as an OAuth authorization server, processes requests.
    PingAccess Profile
    Events related to PingAccess authentication.
    For PingAccess authentication, only the log levels defined in default-log-settings apply.
    VDI
    Events related to connections to virtual desktop resources.
    Endpoint Management System
    Events related to connections to an endpoint management system.
  8. To configure settings for URL request logging, select
    URl Request Logs
    from the left pane.
    URL Request Settings settings display in the right panel.
  9. For URL request logging, from the
    Log Publisher
    list, select the log publisher of your choice.
    A log publisher specifies one or more logging destinations.
    The BIG-IP system is not a logging server and has limited capacity for storing, archiving, and analyzing logs. For this reason a dedicated logging server is recommended.
  10. To log URL requests, you must select at least one check box option:
    • Log Allowed Events
      - When selected, user requests for allowed URLs are logged.
    • Log Blocked Events
      - When selected, user requests for blocked URLs are logged.
    • Log Confirmed Events
      - When selected, user requests for confirmed URLs are logged.
    Whether a URL is allowed, blocked, or confirmed depends on both the URL category into which it falls, and the URL filter that is applied to the request in the per-request policy.
  11. To assign this log setting to multiple access profiles now, perform these substeps:
    Up to three log settings for access system logs can be assigned to an access profile. If you assign multiple log settings to an access profile, and this results in duplicate log destinations, logs are also duplicated.
    1. Select
      Access Profiles
      from the left pane.
    2. Move access profiles between the
      Available
      and the
      Selected
      lists.
    You can delete (and add) log settings for an access profile on the Logs page for the access profile.
    You can configure the log destinations for a log publisher from the Logs page in the System area of the product.
  12. Click
    OK
    .
    The popup screen closes. The table displays.
To put a log setting into effect, you must assign it to an access profile. Additionally, the access profile must be assigned to a virtual server.

Disabling logging

Disable event logging when you need to suspend logging for a period of time or you no longer want the BIG-IP system to log specific events.
Logging is enabled by adding log settings to the access profile.
  1. To clear log settings from access profiles, on the Main tab, click
    Access
    Profiles / Policies
    .
  2. Click the name of the access profile.
    Access profile properties display.
  3. On the menu bar, click
    Logs
    .
  4. Move log settings from the
    Selected
    list to the
    Available
    list.
  5. Click
    Update
    .
Logging is disabled for the access profile.

About event log levels

Event log levels are incremental, ranging from most severe (
Emergency
) to least severe (
Debug
). Setting an event log level to
Warning
for example, causes logging to occur for warning events, in addition to events for more severe log levels. The possible log levels, in order from highest to lowest severity are:
  • Emergency
  • Alert
  • Critical
  • Error
  • Warning
  • Notice
    (the default log level)
  • Informational
  • Debug
Logging at the
Debug
level can increase the load on the BIG-IP system.

Configuring logging for the URL database

Configure logging for the URL database so that log messages are published to the destinations, and at the minimum log level, that you specify. (Logging for the URL database occurs at the system level, not the session level, and is controlled using the default-log-setting log setting.)
A URL database is available only on a BIG-IP system with an SWG subscription.
  1. On the Main tab, click
    Access
    Overview
    Event Logs
    Settings
    .
    A log settings table screen opens.
  2. From the table, select
    default-log-setting
    and click
    Edit
    .
    A log settings popup screen displays.
  3. Verify that the
    Enable access system logs
    check box is selected.
  4. To configure settings for access system logging, select
    Access System Logs
    from the left pane.
    Access System Logs settings display in the right panel.
  5. From the
    Log Publisher
    list, select the log publisher of your choice.
    A log publisher specifies one or more logging destinations.
    The BIG-IP system is not a logging server and has limited capacity for storing, archiving, and analyzing logs. For this reason a dedicated logging server is recommended.
  6. To change the minimum log level, from the
    Secure Web Gateway
    list, select a log level.
    Setting the log level to
    Debug
    can adversely impact system performance.
    The default log level is
    Notice
    . At this level, logging occurs for messages of severity Notice and for messages at all incrementally greater levels of severity.
  7. Click
    OK
    .
    The popup screen closes. The table displays.