F5 Logo MyF5
Search
  1. MyF5 Home
  2. BIG-IQ: Monitoring and Reports
  3. Configuring Statistics Collection
Manual Chapter : Configuring Statistics Collection

Applies To:

Show Versions Show Versions

BIG-IQ Centralized Management

  • 7.1.0
Manual Chapter

Configuring Statistics Collection

BIG-IP device configuration requirements for viewing statistics in BIG-IQ

Before you can enable statistics collection for centralized management, you must ensure that the BIG-IP device has the proper configuration. The proper configuration varies depending on the version of the BIG-IP device. The minimum supported BIG-IP device is version 12.1.0. BIG-IQ has limited visibility for BIG-IP devices prior to 13.1.0.5.
For details about how to configure statistics visibility, based on the BIG-IP version, see
Enabling statistics collection during device discovery
 on
support.f5.com
.
For details on how to access statistic information, based on the BIG-IP version and service, refer to
Statistics compatibility and visibility
 on
support.f5.com
.
For details on how to configure the connection between your BIG-IP devices and your BIG-IQ data collection cluster, refer to
Connecting Devices to a Data Collection Device Cluster
 on
support.f5.com
.

Monitoring BIG-IP statistics in BIG-IQ

Visibility of statistics in BIG-IQ depends on the version of your managed BIG-IP devices. Devices running versions 13.1.X, or earlier, have limited statistics visibility support within BIG-IQ. Below outlines the compatibility and what to expect when accessing Analytics (AVR) data within BIG-IQ. For more information, see the supporting documentation found in the
BIG-IQ Centralized Management: Monitoring and Reports
 guide.

Statistics visibility of managed BIG-IP devices

The format in which statistics are presented in the BIG-IQ environment, depends on the managed version of BIG-IP and the service presented. Refer to the table to access statistics visibility, based on the managed device version. Ensure that the managed device configuration meets the requirements outlined below.
Application data is visible to SC (service cluster), Legacy, and AS3 configurations.

Minimum configuration requirements:

BIG-IP Version 13.1.x or earlier
  • Ports 22 and 443 on each BIG-IP device must be open for the BIG-IQ DCD to retrieve data.
  • There must be a Data Collection Device (DCD) configured to your BIG-IQ.
BIG-IP Version 13.1.0.5 or later
  • You must have AVR provisioned for each BIG-IP device.
  • It is strongly recommended that monitored applications and virtual servers are associated with an analytics profile (HTTP and/or TCP).
  • BIG-IQ needs to provide access on Port 443 to receive BIG-IP AVR data.
  • There must be a Data Collection Device (DCD) configured to your BIG-IQ.
    To view statistics, ensure that the licenses for your managed BIG-IP devices include root access. A BIG-IP license running in Appliance Mode, will not allow for statistics visibility in the BIG-IQ environment.

Where to view statistics

Location of service statistics per managed BIG-IP version
BIG-IP v12.1
BIG-IP v13.0
BIG-IP v13.1
BIG-IP v13.1.0.5
BIG-IP v14.0
BIG-IP v14.1
BIG-IP v15.0 or later
Device Traffic
Monitoring
DASHBOARDS
Device
Local Traffic (General)
Monitoring
DASHBOARDS
Local Traffic
Local Traffic (HTTP)
Not available to this version
Monitoring
DASHBOARDS
Local Traffic
HTTP
Local Traffic (TCP)
Not available to this version
Monitoring
DASHBOARDS
Local Traffic
TCP
DNS (General)*
Monitoring
DASHBOARDS
DNS
Network Firewall (General)
+
Monitoring
REPORTS
Security
Network Firewall
Reporting
Network Firewall information is provided by ACL, IP Reputation, and IPS.
Network Firewall (ACL)
Not applicable to this version
Monitoring
DASHBOARDS
AFM
Network Security (IP Reputation)
Not applicable to this version
Monitoring
DASHBOARDS
AFM
Network Firewall (IPS)
Not applicable to this version
Monitoring
DASHBOARDS
IPS
Web Application Security (General)
Monitoring
REPORTS
Security
Web Application Security
Reporting
Monitoring
DASHBOARDS
Web Application Security
Web Application Security (Bot)
Not available to this version
Monitoring
DASHBOARDS
Bot Traffic
DDoS (Shared Security)
Not available to this version
Monitoring
DASHBOARDS
DDoS
Behavioral DoS (Shared Security)
Not applicable to this version
Visible on the analytics tab of shared security virtual server dashboard.
Monitoring
DASHBOARDS
DDoS
:
Protected Objects
:
Selected Object Name
***
Application Summary
Applications
APPLICATIONS
(limited statistics visibility)
Applications
APPLICATIONS
Secure Web Gateway
Not available to this version
Monitoring
DASHBOARDS
SWG
SSLO**
Not available to this version
Monitoring
DASHBOARDS
SSLO
Access
Monitoring
DASHBOARDS
Access
*Top Charts are only available to BIG-IP version 13.1.0.5 or later
+
 Does not require AVR on host device for visibility.
**SSLO support is available to versions 5.4 to 7.0
***BIG-IP versions 14.1 only displays transaction outcomes/ L3 protocols (depending on virtual server configuration). Version 15.0 includes limited charts and metrics for Behavioral DoS. For more information see
Monitoring Behavioral DoS protection
.

How do I start viewing BIG-IP device statistics from BIG-IQ?

To start viewing statistics for a BIG-IP device, you must have enabled statistics collection for that device. You can do that either during or after adding the device to the BIG-IP Devices inventory list on the BIG-IQ system. You also need to install, configure, and add a data collection device before you can view statistics for your managed BIG-IP devices.

Enabling statistics collection during device discovery

Before you can enable statistics for BIG-IP devices:
  • There must be a BIG-IQ data collection device (DCD) configured for the BIG-IQ device.
  • The BIG-IP device must be located in your network and running a compatible software version. Refer to K34133507#cm6.0.1 for more information.
    • For BIG-IP devices running version 13.1.0.5 or later, you must have AVR provisioned.
  • For BIG-IP devices running versions prior to 13.1.0.5, configure Ports 22 and 443 must be open to traffic from the BIG-IQ DCD to the managed BIG-IP devices.
  • For BIG-IP devices running version 13.1.0.5 or later, BIG-IQ needs to provide access on Port 443 so that the BIG-IP AVR module can send statistics to the BIG-IQ DCD
One way to enable statistics collection for BIG-IP devices is to do it when you add those devices to the BIG-IQ system inventory. Adding devices to the inventory is referred to as
device discovery
. If the devices you want to enable have already been discovered, refer to
Enabling collection after device discovery
.
The ADC component is automatically included (first) any time you discover or import services for a device.
You do not need to discover and import a device’s configuration to collect and view statistics for it. You just need to establish trust between your BIG-IQ and the device. If you do not discover and import the device configuration, the virtual servers, pool, pool members, and iRules will be visible in the statistics dimension panes, but these objects will not appear in the configuration page for those objects. Also, you will not be able to manage these objects in BIG-IQ. If you decide you want to manage these objects, you can discover and import the BIG-IP device’s configuration later without interrupting statistics collection.
  1. At the top of the screen, click
    Devices
    .
  2. Click the
    Add Device(s)
     button.
  3. For
    IP Address
    , type the IPv4 or IPv6 address of the device.
  4. Type the
    User Name
     and
    Password
     for the device.
  5. If this device is part of a DSC group, for the
    Cluster Display Name
     setting, specify how to handle it:
    • For an existing DSC group, select
      Use Existing
       from the list, and then select the name of the DSC group from the next list.
    • To create a new DSC group, select
      Create New
       from the list, and type a name in the field.
    For BIG-IQ to properly associate the devices in the same DSC group, the
    Cluster Display Name
     must be the same for all members in a group.
    There can be up to eight members in a DSC group.
    For BIG-IP devices with ASM services, you can only add five devices at a time. If the BIG-IP device(s) provisioned with ASM is part of a DSC cluster, that device must also be a member of a sync-only device group, and ASM synchronization must be enabled for the device group. Without these DSC group settings, deploying changes to the ASM device can cause the cluster to get out of sync. For more information see K12200102, or the ASM Implementations chapter
    Automatically Synchronizing Application Security Configurations
     on
    support.f5.com
    .
  6. If this device is configured in a DSC group or you are creating a new DSC group, for the
    Cluster Properties
    , specify how to handle it:
    • Initiate BIG-IP DSC sync when deploying configuration changes (Recommended)
      : Select this option if you want this device to automatically synchronize configuration changes with other members in the DSC.
    • Allow deployment when DSC configured devices have changes pending ( Not Recommended)
      : Select this option if you want to deploy changes to this device even if there are changes pending for devices in the DSC group.
      This option is not recommended, because it can lead to unpredictable results.
    • Ignore BIG-IP DSC sync when deploying configuration changes
      : Select this option if you want to manually synchronize configurations changes between members in the DSC group.
  7. Click the
    Add
     button at the bottom of the screen.
    The BIG-IQ system opens communication to the BIG-IP device, and checks the BIG-IP device framework.
    The BIG-IQ system can properly manage a BIG-IP device only if the BIG-IP device is running a compatible version of the REST framework.
  8. If a framework upgrade is required, in the popup window, in the
    Root User Name
     and
    Root Password
     fields, type the root user name and password for the BIG-IP device, and click
    Continue
    .
  9. If in addition to basic management tasks (like software upgrades, license management, and UCS backups) you also want to centrally manage this device's configurations for licensed services, select the check box next to each service you want to discover.
    You can also select these service configuration after you add the BIG-IP device to the inventory.
  10. To enable statistics collection for this BIG-IP device, under Statistics monitoring, select the check box next to each service you want to collect statistics for, and then click
    Continue
    .
    For Network Security, enable the
    AFM Statistics Collection
    .
    If you want to enable statistics collection without managing any services, clear the check boxes for all services.
  11. Click the
    Add
     button at the bottom of the screen.

Enable statistics collection for devices

Before you can enable statistics collection for a BIG-IP device:
  • The device must already be in the BIG-IQ system inventory.
  • There must be a BIG-IQ data collection device configured for the BIG-IQ device.
  • For BIG-IP device version 13.1.0.5 or later, AVR must be provisioned.
To collect statistics for a BIG-IP device, you enable statistics collection when you discover it, however, you can enable or disable statistics collection for a device any time it is convenient for you. When enabling statistics for services and modules (including AFM), local storage is disabled on the device to prevent data duplication and extraneous resource usage.
  1. At the top of the screen, click
    Devices
    .
  2. Click the name of the device you want to enable statistics collection for.
  3. On the left, click
    Statistics Collection
    .
  4. To begin statistics collection, for
    Collect Statistics Data
    , select
    Enabled
    .
    If this option is disabled, no statistics will be collected from your device, regardless of service or module.
  5. For
    Modules/Services
    , click the check box to enable statistics collection for specific system modules and services.
  6. For
    AFM Statistics Collection
     select
    Enabled
     to collect statistics from your device's Network Security module.
  7. For
    Frequency
    , next to
    Collect every
    , select the interval at which you want to collect statistics from this device.
  8. Click
    Save & Close
    .
After you enable statistics collection for a device, data for that device begins aggregating along with any other devices for which you are collecting data. Two buttons (
View Health Statistics
, and
View Traffic Statistics
) are added to the properties page for enabled devices. Clicking either of these takes you directly to the overview page for the statistics type you clicked

Statistics retention policy overview

When you choose how much raw data to retain, you need to consider how much disk space you have available. The controls on this screen are simple to set up, but understanding how they work takes a bit of explanation.
The fields on the Statistics Retention Policy screen all work in similar fashion. One way to understand how these fields work is to think of your data storage space as a set of containers. The values you specify on this screen determine how much storage space each container consumes. Because data is saved for the time periods you specify, the longer the time period that you specify, the more space you consume. The disk storage that is consumed depends on several factors.
  • The number of BIG-IP devices you manage
  • The number of objects on the BIG-IP devices you manage (for example, virtual servers, pools, pool members, and iRules)
  • The frequency of statistics collection
  • The data retention policy
  • The data replication policy
There are three key concepts to understand about how the retention policy works.
How long is data in each container retained?
Data is retained in each container for the time period you specify. When the specified level is reached, the oldest chunk of data is deleted. For example, if you specify a raw data value of 48 hours, then when 48 hours of raw data accumulate, the next hour of incoming raw data causes the oldest hour to be deleted.
When does data from one container pass on to the next?
Data passes from one container to the next in increments that are the size of the next (larger) container. That is, every 60 minutes, the last 60 minutes of raw data is aggregated into a data set and passed to the
Hour(s)
container. Every 24 hours, the last 24 hours of hourly data is aggregated into a data set and passed to the
Day(s)
 container, and so on for the
Month(s)
 container.
What about limits?
Limit Max Storage to
 specifies the percentage of total disk space that you want data to consume on the data collection devices in your cluster.
If more disk space is consumed than the percentage you specified, BIG-IQ takes two actions:
  1. New statistical data is not accepted until the available disk space complies with the
    Limit max storage to
     setting.
  2. Statistical data not required to calculate the next higher time layer is removed (for example, you need 60 minutes of raw data to aggregate to the Hours level). Data is removed starting with the raw data container, then the hourly data container, then the daily time container. This process stops when storage consumption is below the
    Limit max storage to
     setting.
The BIG-IQ takes this action to prevent data corruption when storage is completely exhausted.

The aggregation policy for your statistics data

There is a default statistics aggregation policy for the data added to your data collection device. The aggregation policy impacts the quality of the entity data, per dimension, over time. This optimizes the disk usage, and allows for high quality data for short-term analysis and troubleshooting for raw, hour, or even day time layers of data. Long term data storage provides insights into global statistics over time, but are not recommended for troubleshooting.

Manage the retention policy for your statistics data

Before you can set the statistics retention policy, you must have added a data collection device.
You can manage the settings that determine how your statistics data is retained. The highest quality data is the raw data, (data that has not been averaged), but that consumes a lot of disk space, so you need to consider your needs in choosing your data retention settings.
  1. At the top of the screen, click
    System
    , then, on the left, click
    BIG-IQ DATA COLLECTION
     and then select
    BIG-IQ Data Collection Cluster
    .
    The BIG-IQ Data Collection Cluster screen opens. On this screen, you can either view summary status for the data collection device cluster or access the screens that you can use to configure the DCD cluster.
    • Under Summary, you can view information detailing how much data is stored, as well as how the data is stored.
    • Under Configuration, you can access the screens that control DCD cluster performance.
  2. Under the screen name, click
    Configuration
    Statistics Data Collection
    .
    The Statistics Collection Status screen opens.
  3. On the left, click
    Statistics Data Collection
    .
    The Statistics Collection Status screen displays the percentage of available disk space currently consumed by statistics data for each container.
  4. To change the retention settings for your statistics data, click
    Configure
    .
    The Statistics Retention Policy screen opens.
  5. In the
    Keep real-time (raw) data up to
     field, type the number of hours of raw data to retain.
    You must specify a minimum of 1 hour, so that there is sufficient data to average and create a data point for the
    Keep hourly data up to
     container.
  6. In the
    Keep hourly data up to
     field, type the number of hourly data points to retain.
    You must specify a minimum of 24 hours, so that there is sufficient data to average and create a data point for the
    Keep daily data up to
     container.
  7. In the
    Keep daily data up to
     field, type the number of daily data points to retain.
    You must specify a minimum of 31 days, so that there is sufficient data to average and create a data point for the
    Keep monthly data up to
     container.
  8. In the
    Keep monthly data up to
     field, type the number of monthly data points to retain.
    Once the specified number of months passes, the oldest monthly data set is deleted.
  9. In the
    Limit max storage to
     field, type the percentage of disk space that you want collected data to consume before the oldest monthly data set is deleted.
  10. Expand Advanced Settings, and then select the
    Enable Replicas
     check box.
    Replicas
     are copies of a data set that are available to the DCD cluster when one or more devices within that cluster become unavailable. By default, data replication for statistics is enabled. Disabling replication reduces the amount of disk space required for data retention. However, this provides no protection from data corruption that can occur when you remove a data collection device. You should enable replicas to provide this protection.
  11. When you are satisfied with the values specified for data retention, click
    Save & Close
    .

About Analytics profiles

An
Analytics profile
 is a set of definitions that determines the circumstances under which the managed BIG-IP system gathers, logs, notifies, and displays information regarding HTTP or TCP traffic to an application. Each monitored application is associated with an Analytics profile. You associate an Analytics profile with one or more virtual servers used by the application. Each virtual server can have one HTTP and/or one TCP Analytics profile associated with it.
In the HTTP Analytics profile, you can customize:
  • Which statistics to collect, and their collection value thresholds
  • Location of data collection (locally, remotely, or both)
  • Notifications
  • Traffic capturing specifications (HTTP only)
    See
    Troubleshoot HTTP Traffic by Reviewing Captured Traffic for more information
The system includes a default HTTP Analytics profile called
analytics
 and a TCP Analytics profile called
tcp-analytics
. These serve as the parent of all other Analytics profiles that you create on the system. You can modify the default profile, or create custom Analytics profiles for each application if you want to track different data for each one.
Certain settings, such as SMTP Configuration, Transaction Sampling, and the Subnets list, can only be set in the default HTTP Analytics profile.

Analytics profiles for AS3 applications

If you are managing AS3 applications, update analytics profiles using an AS3 template or directly add profile settings to the application declaration JSON. Information about the field attribute is listed within the customizing procedures. For the full reference index for analytics profile attributes, including traffic capturing, go to schema-reference.html.
For HTTP Analytics properties, see
Analytics_Profile
.
For Traffic Capturing settings, see
Capture_Filter
For TCP Analytics properties, see
Analytics_TCP_Profile
.

Customizing the default HTTP Analytics profile

Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
Devices
BIG-IP DEVICES
<DEVICE NAME>
STATISTICS COLLECTION
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.
To view log messages on an external server, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
 section of
BIG-IQ Centralized Management: Local Traffic and Network Implementations
 on
support.f5.com
.
An HTTP Analytics profile directs the system to store various HTTP statistics for troubleshooting application-layer issues. The system includes a default HTTP Analytics profile called
analytics
. You can edit the settings in the default profile so it uses the values you want.
Certain information can be specified only in the default HTTP Analytics profile: the SMTP configuration (a link to an SMTP server), transaction sampling (whether enabled or not), and subnets (assigning names to be used in the reports). To edit these values, you need to open and edit the default profile.
  1. Go to
    Configuration
    LOCAL TRAFFIC
    Profiles
    .
    If you would like to edit the parent profile, select
    analytics
     from the Parent Profile column and proceed to step 7.
  2. Click
    Create
    .
    The New Profile screen opens.
  3. For
    Name
    , type a name for the new profile.
  4. If required by your managed BIG-IP device, change the
    Silo
     and
    Partition
     field.
  5. From the
    Parent Profile
     list, select the profile from which you want to inherit settings.
    The default profile is often used as the parent profile.
    The new profile inherits the values from the parent profile. If the parent is changed, the inherited values in the new profile also change.
  6. To make all the fields editable, click
    Override All
    . This applied to both the Settings, Metrics Gathering Configuration, and Dimensions Gathering Configuration options.
  7. For
    Collected Statistics Internal Logging
    , select
    Enable
     to specify the system collectes a postion of the application traffic data.
    AS3 Attribute
    collectedStatsInternalLogging
    Once enabled, you can manage which Dimension Gathering Configuration settings are collected.
  8. (Optional) To send HTTP traffic data to an external server, enable
    Collected Statistics External Logging
    .
    AS3 Attribute
    collectedStatsExternalLogging
    To specify Remote Publisher:
    externalLoggingPublisher
    Selecting this option allows you to choose a
    Remote Publisher
    . You must select a remote publisher configured to BIG-IQ to view log data using BIG-IQ.
    AS3 Attribute: externalLoggingPublisher
  9. For
    Captured Traffic Internal Logging
    , select
    Enable
     to manage the Capture Filter settings.
    Once traffic capturing is enabled, you can configure the capture criteria in the
    Capture Filter
     area at the bottom of the screen. For more information about the dimension and metric options for traffic capturing, see
    Configure traffic capturing for troubleshooting
    .
    AS3 Attribute
    capturedTrafficInternalLogging
  10. (Optional) To send TCP traffic data to an external server, enable
    Collected Statistics External Logging
    .
    AS3 Attribute
    capturedTrafficExternalLogging
    To specify remote SMTP server:
    externalLoggingPublisher
    Once you enable this field, you can select a pre-configured server from the
    SMTP Configuration
     field.
  11. To send email alerts, specify an
    SMTP Configuration
    .
    You can change the SMTP configuration only in the default profile. It is used globally for the system. If no configuration is available, click
    Create
     to create one.
  12. For the
    Notification by...
     settings, enable the settings to send alerts and notifications.
    Entity
    Description
    AS3 Attribute
    Syslog
    Select
    Syslog
     if you want the system to send notification and alert messages to the local (Host BIG-IP) log system.
    notificationBySyslog
    SNMP
    Select
    SNMP
     if you want the system to send notification and alert messages as SNMP traps. You create these settings directly on the host BIG-IP device.
    notificationBySnmp
    E-mail
    Select
    E-mail
     if you want the system to send notification and alert messages to configured email addresses. Type each email address in the
    Notification E-Mails
     field, and click
    +
     to create the list. This option requires that the default analytics profile includes an SMTP configuration.
    notificationByEmail
    Specify e-mail addresses:
    notificationEmailAddresses
  13. If you want the system to perform traffic sampling, make sure that for
    Transaction Sampling
     area, the
    Sample
     check box is selected.
    You can change this setting only in the default profile.
    Sampling improves system performance. F5 recommends that you enable sampling if you generally use more than 50 percent of the BIG-IP system CPU resources, or if you have at least 100 transactions in 5 minutes for each entity.
  14. If you want the system to collect and display statistics, according to the expressions written in an iRule, enable
    Publish iRule Statistics
    .
    AS3 Attribute
    publishIruleStatistics
    When you select this option, iRule statistics are visible per analytics profile. In addition, these iRule events are displayed in near real time (delay of 10 seconds), while statistics in the Configuration utility have a delay of at least 5 minutes. You can view iRule statistics per Analytics profile on the command line by typing
    ISTATS dump
    .
    For the system to collect iRule statistics, you must also write an iRule describing which statistics the system should collect.
  15. In the Metrics Gathering Configuration area, enable additional statistics you want the system to collect from the HTTP requests:
    By default, the system collects many metrics, including TPS, throughput, server latency, response time, and network latency. You can select the metrics here, in addition to the ones already collected, once the HTTP Analytics profile is attached to one or more virtual servers.
    Entity
    Description
    AS3 Attribute
    Max TPS and Throughput
    Collects and logs statistics regarding the maximum number of transactions per second (TPS) and the amount of traffic moving through the system.
    collectMaxTpsAndThroughput
    Page Load Time
    Collects and logs statistics regarding the time it takes an application user to get a complete response from the application, including network latency and completed page processing.
    End-user response times and latency can vary significantly based on geographic location and connection types.
    collectPageLoadTime
    HTTP Timing (RTT, TTFB, Duration)
    Collects and logs statistics regarding the HTTP request and response times, including round-trip time, time to first byte and overall transaction duration time.
    N/A
    User Sessions
    Collects and logs statistics regarding the number of unique user sessions.
    collectUserSession
    Cookie Secure Attribute
    Specifies how to log secure session cookies:
    • Always
      , the secure attribute is always added to the session cookie.
    • Never
      , the secure attribute is never added to the session cookie.
    • Only SSL
      , the secure attribute is added to the session cookie only when the virtual server has a client SSL profile (the default value).
    sessionCookieSecurity
    Timeout
    Logs data by the allowed minutes of user inactivity before the system considers the session to be over.
    sessionTimeoutMinutes
  16. In the Dimensions Gathering Configuration area, enable additional entities to collect statistics for each request.
    By default, the system collects many entity statistics, including virtual servers, pool members, browser names, and operating systems You can select the ones here, in addition to the ones already collected, once the HTTP Analytics profile is attached to one or more virtual servers.
    When you select
    URLs
    ,
    Countries
    ,
    Client IP Addresses
     or
    Client Subnets
     you have additional options configure specific statistics filtering options.
    Entity
    Description
    AS3 Attribute
    URLs
    Collects all, or only specified, URLs.
    collectUrl
    To specify URLs:
    urlsForStatCollection
    Countries
    Collects all, or only specified, countries. Country information is based on where the request came from, and is based on the client IP address criteria.
    collectGeo
    To specify countries:
    countriesForStatCollection
    Client IP Addresses
    Collects all, or only specified, IP address. IP address information is based on where the request originated. The address saved also depends on whether the request has an XFF (X-forwarded-for) header and whether the HTTP profile accepts XFF headers.
    collectIp
    Client Subnets
    Collects statistics for predefined client subnets. Client subnets can be added in the Subnets area of the default HTTP Analytics profile.
    collectSubnet
    To specify subnets:
    subnetsForStatCollection
    Response Codes
    Collects HTTP response codes that the server returned in response to requests.
    collectResponseCode
    User Agents
    Collects information about browsers making the request.
    collectUserAgent
    Methods
    Collects HTTP methods in requests.
    collectMethod
  17. When you are done click
    Save & Close
    .
Virtual servers and applications configured to this profile collect and report traffic statistics according to specified settings. For more information about how to view an analyze application traffic, see
Monitoring and Managing Applications Using BIG-IQ
 on
support.f5.com
.

Customizing the default TCP Analytics profile

Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
Devices
BIG-IP DEVICES
<DEVICE NAME>
STATISTICS COLLECTION
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.
To view log messages on an external publisher, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
 section of
BIG-IQ Centralized Management: Local Traffic and Network Implementations
 on
support.f5.com
.
A TCP Analytics profile directs the system to store TCP statistics about specific entities for use in diagnosing network problems. The system includes a default TCP Analytics profile called
tcp-analytics
. You can edit the values in the default profile, or create a new one, as described here.
  1. Go to
    Configuration
    LOCAL TRAFFIC
    Profiles
    .
    If you would like to edit the parent profile, select
    tcp-analytics
     from the Parent Profile column and proceed to step 7.
  2. Click
    Create
    .
    The New Profile screen opens.
  3. For
    Name
    , type a name for the new profile.
  4. If required by your managed BIG-IP device, change the
    Silo
     and
    Partition
     field.
  5. From the
    Parent Profile
     list, select the profile from which you want to inherit settings.
    The default profile is often used as the parent profile.
    The new profile inherits the values from the parent profile. If the parent is changed, the inherited values in the new profile also change.
  6. To make all the fields editable, click
    Override All
    . This applied to both the Settings and Dimensions Gathering Configuration options.
  7. For
    Collected Statistics Internal Logging
    , select
    Enable
     to manage the Dimension Gathering Configuration settings.
    AS3 Attribute
    collectedStatsInternalLogging
  8. (Optional) To send TCP traffic data to an external server, enable
    Collected Statistics External Logging
    .
    AS3 Attribute
    collectedStatsExternalLogging
    To specify Remote Publisher:
    externalLoggingPublisher
    Selecting this option allows you to choose a
    Remote Publisher
    . You must select a remote publisher configured to BIG-IQ to view log data on an external server.
  9. For
    Collected Statistics By Server Side
     enable to specify that statistics from the server side of the TCP transaction are collected.
    AS3 Attribute
    collectedByServerSide
  10. For
    Collected Statistics By Client Side
     enable to specify that statistics from the client side of the TCP transaction are collected.
    AS3 Attribute
    collectedByClientSide
  11. From Dimensions Gathering Configuration, select the entities for which you want the system to collect information.
    The more entities you enable, the greater the impact on system performance.
    Entity
    Description
    AS3 Attribute
    City
    Collects the name of the city with which traffic was exchanged.
    collectCity
    Country
    Collects the name of the country with which traffic was exchanged.
    collectCountry
    Continent
    Collects the name of the continent with which traffic was exchanged.
    collectContinent
    Next Hop Ethernet Address
    Collects the addresses to which traffic is being routed.
    collectNexthop
    Post Code
    Collects the name of the postal code with which traffic was exchanged.
    collectPostCode
    Remote Host IP Address
    Collects the IP addresses with which traffic was exchanged
    collectRemoteHostIp
    Region
    Collects the name of the region with which traffic was exchanged.
    collectRegion
    Remote Host Subnet
    Collects the addresses of the subnets with which traffic was exchanged.
    collectRemoteHostSubnet
  12. When you are done click
    Save & Close
    .
Virtual servers and applications configured to this profile collect and report traffic statistics according to specified settings. For more information about how to view an analyze application traffic, see
Monitoring and Managing Applications Using BIG-IQ
 on
support.f5.com
.

Reviewing captured traffic details

Traffic capturing prompts the system to log traffic request and response headers and payload data, based on specific collection requirements. You enable traffic capturing in your Analytics profile to monitor a known application issue, such as trouble with throughput or latency, or a known factor that can impact application performance, such as HTTP method, or client IP address. You can specify these traffic aspects to later examine application statistics, and troubleshoot captured transactions.
Once enabled, you can examine the captured traffic to explore details, such as the payload of captured transactions, requested URLs and response size. When traffic capturing is enabled, you can view data about captured traffic within the charts for HTTP traffic statistics.

Configure traffic capturing for troubleshooting

Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
Devices
BIG-IP DEVICES
<DEVICE NAME>
STATISTICS COLLECTION
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.
To view log messages on an external server, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
 section of
BIG-IQ Centralized Management: Local Traffic and Network Implementations
 on
support.f5.com
.
You can configure your HTTP analytics profile to capture traffic headers and additional transaction details. Once configured, you can review captured traffic, based upon specific transaction parameters and performance thresholds.
  1. Go to
    Configuration
    LOCAL TRAFFIC
    Profiles
    .
    This screen lists the profiles that are configured for the managed BIG-IP devices in your network.
  2. Select the HTTP Analytics profile you wish to edit.
    The
    analytics
     profile is a default profile for all HTTP Analytics management. If you are creating a new HTTP Analytics profile, make sure to select the
    Override All
     check box to change the settings inherited by the parent profile.
  3. For
    Captured Traffic Internal Logging
    , select
    Enable
     to manage the Capture Filter settings.
    AS3 Attribute
    capturedTrafficInternalLogging
    Once you enable a traffic capturing, the Capture Filter area becomes available. This allows you to further configure which traffic you would like to capture.
  4. (Optional) To send captured traffic to an external server, enable
    Captured Traffic External Logging
    .
    AS3 Attribute
    capturedTrafficExternalLogging
    To specify Remote Publisher:
    externalLoggingPublisher
    Once you enable this field, you can select a pre-configured server from the
    Remote Publisher
     field.
  5. From the
    Capture Request Details
     and
    Capture Response Details
     lists, select the options that indicate the part of the traffic to capture.
    Detail options for request and response capture:
    Entity
    Description
    None
    Specifies that the system does not capture request (or response) data.
    Headers
    Specifies that the system captures request (or response) header data only.
    Body
    Specifies that the system captures the body of requests (or responses) only.
    All
    Specifies that the system captures all request (or response) data, including header and body.
    Entity
    AS3 Attribute
    Capture Request Details
    requestCapturedParts
    Capture Response Details
    responseCapturedParts
  6. For
    DoS Activity
    , select the option that indicates which DoS traffic is captured.
    Option
    Description
    Any
    Specifies that the system captures any traffic regardless of DoS activity.
    Mitigated by Application DoS
    Specifies that the system only captures DoS traffic if it was mitigated.
    AS3 Attribute
    dosActivity
  7. For
    Protocols
    , specify whether the system captures
    All
    traffic, or traffic with
    HTTP
    , or
    HTTPS
     protocols.
    AS3 Attribute
    capturedProtocols
  8. For
    Qualified for JavaScript Injection
    , you can select
    Qualified only
     to specify that the system only captures traffic that qualifies for JavaScript injection, which includes the following conditions:
    • The HTTP content is not compressed
    • The HTTP content-type is
      text/html
      .
    • The HTTP content contains an HTML
      <head>
       tag
    AS3 Attribute
    capturedReadyForJsInjection
  9. Customize the dimension filters, according to your application needs, to capture the portion of traffic to that you need for troubleshooting.
    Dimension filters capture traffic according to defined aspects of the transaction's configuration, or header/payload contents. By focusing in on the data and limiting the type of information that is captured, you can troubleshoot particular areas of an application more efficiently. For example, capture only requests or responses, specific status codes or methods, or headers containing a specific string.
    Entity
    Description
    AS3 Attribute
    Response Status Codes
    Select
    All
    to capture traffic, regardless of the HTTP status response code.
    Select
    Only
     to capture traffic with specific response status codes. To specify, add response status codes to the
    Selected Status Codes
     list from the
    Available Status Codes
     list.
    responseCodes
    HTTP Methods
    Select
    All
    to capture traffic, regardless of the HTTP request method.
    Select
    Only
     to capture traffic with requests that contain a specific HTTP method. To specify, add methods to the
    Selected Methods
     list from the
    Available Methods
     list.
    methods
    URL
    Select
    All
     to capture traffic with requests for any URL.
    Select
    Starts With
     to only capture traffic with requests for URLs that start with a specific string.
    If you select this option, and leave the list blank, the system will not capture any traffic.
    Select
    Does not start with
     to capture traffic with requests for URLs except for those that start with a specific string.
    You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic with requests for any URL.
    urlFilterType
    To add URL prefixes:
    urlPathPrefixes
    User Agent
    Select
    All
     to capture traffic sent from any browser.
    Select
    Contains
     to only capture traffic sent from a browser that contains a specific string.
    You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic sent from any browser.
    userAgent
    To add User Agent substrings
    userAgentSubstrings
    Client IP Address
    Select
    All
     to capture traffic sent to, or from, any client IP address.
    Select
    Only
     to only capture traffic sent to or from a specific client IP address.
    You can add up to 10 different IP addresses to the list. If the list is blank, the system will capture traffic sent to, or from, any IP address.
    clientIps
    Request Containing String
    Select
    All
     to capture all traffic.
    Select
    Search in
     filter captured traffic that includes a specific string contained in the request.
    requestContentFilterSearchString
    Response Containing String
    Select
    All
     to capture all traffic.
    Select
    Search in
     filter captured traffic that includes a specific string contained in the response.
    responseContentFilterSearchString
  10. Click
    Save & Close
    .
Your
analytics
 profile is now configured for traffic capturing.You can assign this profile to your virtual servers, if they do not yet have an Analytics profile configured.

Review captured traffic

To display captured traffic, your virtual server must be assigned an HTTP analytics profile that has captured traffic enabled, with external logging.
You can troubleshoot details of captured HTTP traffic to your applications and virtual servers. This information can provide details of request/response headers and payload sent to your managed application. Captured traffic information is found within the following dashboards that provide HTTP traffic visibility:
  • Device Traffic:
    Monitoring
    DASHBOARDS
    Device
    Traffic
    .
  • DDoS HTTP Analysis:
    Monitoring
    DASHBOARDS
    DDoS
    HTTP Analysis
    .
  • Local Traffic:
    Monitoring
    DASHBOARDS
    Local Traffic
    HTTP
    .
  1. Navigate to one of the monitoring dashboards that display HTTP traffic data.
  2. Select the
    Traffic Capturing
     button above the charts.
    Selecting this option overlays captured traffic data over the charts, and adds a traffic capturing filter in the Dimensions pane.
  3. To filter captured traffic based on a specific host object, such as a BIG-IP system (
    BIG-IP Host Names
    ), application (
    Applications Services
    ), or virtual server (
    Virtual Servers
    ), expand the dimension widgets in the Dimensions pane to the right of the charts.
    You can select multiple dimension objects from multiple dimensions. With each selection, the charts and dimensions filter displayed data according to your selections.
  4. To filter captured traffic based on server latency and payload volume metrics, expand the
    Traffic Capturing Filters
     found in the dimensions pane.
    For latency metrics, you can enter a range, or set a greater or less than filter value.
  5. To view traffic details, select a traffic capturing icon from within the chart to display an information table.
    You can click the rows within the displayed table to view additional request/response header and payload information.
Contact Support Contact Support

Have a Question?

Support and Sales >

About F5

Education

F5 Sites

Support Tasks




©2023 F5, Inc. All rights reserved.


Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information