Manual Chapter :
Configuring Statistics Collection
Applies To:
Show VersionsBIG-IQ Centralized Management
- 8.0.0
Configuring Statistics Collection
BIG-IP device configuration requirements for viewing statistics from
BIG-IQ
Before you can enable statistics collection for centralized management,
you must ensure that the BIG-IP device has the proper configuration. The proper
configuration varies depending on the version of the BIG-IP device. The minimum supported
BIG-IP device is version 12.1.0. BIG-IQ has limited visibility for BIG-IP devices prior to
13.1.0.5.
For details about how to configure statistics visibility, based on the
BIG-IP version, see
Enabling statistics collection during
device discovery
on support.f5.com
.For details on how to access statistic information, based on the BIG-IP
version and service, refer to
Statistics compatibility and
visibility
on support.f5.com
.For details on how to configure the connection between your BIG-IP
devices and your BIG-IQ data collection cluster, refer to
Connecting Devices to a Data Collection Device Cluster
on support.f5.com
.Monitoring BIG-IP statistics in BIG-IQ
Visibility of statistics in BIG-IQ depends on the version of your managed BIG-IP devices. Devices running versions 13.1.X, or earlier, have limited statistics visibility support within BIG-IQ. Below outlines the compatibility and what to expect when accessing Analytics (AVR) data within BIG-IQ. For more information, see the supporting documentation found in the
BIG-IQ Centralized Management: Monitoring and Reports
guide.Statistics visibility of managed BIG-IP devices
The format in which statistics are presented in the BIG-IQ environment, depends on the managed version of BIG-IP and the service presented. Refer to the table to access statistics visibility, based on the managed device version. Ensure that the managed device configuration meets the requirements outlined below.
Application data is visible to SC (service cluster), Legacy, and AS3 configurations.
Minimum configuration requirements:
- BIG-IP Version 13.1.x or earlier
- Ports 22 and 443 on each BIG-IP device must be open for the BIG-IQ DCD to retrieve data.
- There must be a Data Collection Device (DCD) configured to your BIG-IQ.
- BIG-IP Version 13.1.0.5 or later
- You must have AVR provisioned for each BIG-IP device.
- It is strongly recommended that monitored applications and virtual servers are associated with an analytics profile (HTTP and/or TCP).
- BIG-IQ needs to provide access on Port 443 to receive BIG-IP AVR data.
- There must be a Data Collection Device (DCD) configured to your BIG-IQ.To view statistics, ensure that the licenses for your managed BIG-IP devices include root access. A BIG-IP license running in Appliance Mode, will not allow for statistics visibility in the BIG-IQ environment.
Where to view statistics
BIG-IP v12.1 | BIG-IP v13.0 | BIG-IP v13.1 | BIG-IP v13.1.0.5 | BIG-IP v14.0 | BIG-IP v14.1 | BIG-IP v15.0 or later | |
---|---|---|---|---|---|---|---|
Device Traffic | |||||||
Local Traffic (General) | |||||||
Local Traffic (HTTP) | Not available to this version | ||||||
Local Traffic (TCP) | Not available to this version | ||||||
DNS (General)* | |||||||
Network Firewall (General) + | Network Firewall information is provided by ACL, IP Reputation, and IPS. | ||||||
Network Firewall (ACL) | Not applicable to this version | ||||||
Network Security (IP Reputation) | Not applicable to this version | ||||||
Network Firewall (IPS) | Not applicable to this version | ||||||
Web Application Security (General) | |||||||
Web Application Security (Bot) | Not available to this version | ||||||
DDoS (Shared Security) | Not available to this version | ||||||
Behavioral DoS (Shared Security) | Not applicable to this version | Visible on the analytics tab of shared security virtual server dashboard. : Protected Objects : Selected Object Name *** | |||||
Application Summary | (limited statistics visibility) | ||||||
Secure Web Gateway | Not available to this version | ||||||
SSLO** | Not available to this version | ||||||
Access |
*Top Charts are only available to BIG-IP version 13.1.0.5 or later
+
Does not require AVR on host device for visibility. **SSLO support is available to versions 5.4 to 8.2. Please note, SSLO support depends on the compatibility with the BIG-IP device.
***BIG-IP versions 14.1 only displays transaction outcomes/ L3 protocols (depending on virtual server configuration). Version 15.0 includes limited charts and metrics for Behavioral DoS. For more information see
Monitoring Behavioral DoS protection
.How do I start viewing BIG-IP device
statistics from BIG-IQ?
To start viewing statistics for a BIG-IP device, you must have enabled statistics collection for that device.
You can do that either during or after adding the device to the BIG-IP Devices inventory
list on the BIG-IQ system. You also need to
install, configure, and add a data collection device before you can view statistics for
your managed BIG-IP devices.
Enabling statistics collection during device discovery
Before you can enable statistics for BIG-IP devices:
- There must be a BIG-IQ data collection device (DCD) configured for the BIG-IQ device.
- The BIG-IP device must be located in your network and running a compatible software version. Refer to K34133507#cm6.0.1 for more information.
- For BIG-IP devices running version 13.1.0.5 or later, you must have AVR provisioned.
- For BIG-IP devices running versions prior to 13.1.0.5, configure Ports 22 and 443 must be open to traffic from the BIG-IQ DCD to the managed BIG-IP devices.
- For BIG-IP devices running version 13.1.0.5 or later, BIG-IQ needs to provide access on Port 443 so that the BIG-IP AVR module can send statistics to the BIG-IQ DCD
One way to enable statistics collection for BIG-IP devices is to do it when you add those devices to the BIG-IQ system inventory. Adding devices to the inventory is referred to as
device discovery
. If the devices you want to enable have already been discovered, refer to Enabling collection after device discovery
. The ADC component is automatically included (first) any time you discover or import services for a device.
You do not need to discover and import a device’s configuration to collect and view statistics for it. You just need to establish trust between your BIG-IQ and the device. If you do not discover and import the device configuration, the virtual servers, pool, pool members, and iRules will be visible in the statistics dimension panes, but these objects will not appear in the configuration page for those objects. Also, you will not be able to manage these objects in BIG-IQ. If you decide you want to manage these objects, you can discover and import the BIG-IP device’s configuration later without interrupting statistics collection.
- At the top of the screen, clickDevices.
- Click theAdd Device(s)button.
- ForIP Address, type the IPv4 or IPv6 address of the device.
- Type theUser NameandPasswordfor the device.
- If this device is part of a DSC group, for theCluster Display Namesetting, specify how to handle it:
- For an existing DSC group, selectUse Existingfrom the list, and then select the name of the DSC group from the next list.
- To create a new DSC group, selectCreate Newfrom the list, and type a name in the field.
For BIG-IQ to properly associate the devices in the same DSC group, theCluster Display Namemust be the same for all members in a group.There can be up to eight members in a DSC group.For BIG-IP devices with ASM services, you can only add five devices at a time. If the BIG-IP device(s) provisioned with ASM is part of a DSC cluster, that device must also be a member of a sync-only device group, and ASM synchronization must be enabled for the device group. Without these DSC group settings, deploying changes to the ASM device can cause the cluster to get out of sync. For more information see K12200102, or the ASM Implementations chapterAutomatically Synchronizing Application Security Configurationsonsupport.f5.com. - If this device is configured in a DSC group or you are creating a new DSC group, for theCluster Properties, specify how to handle it:
- Initiate BIG-IP DSC sync when deploying configuration changes (Recommended): Select this option if you want this device to automatically synchronize configuration changes with other members in the DSC.
- Allow deployment when DSC configured devices have changes pending ( Not Recommended): Select this option if you want to deploy changes to this device even if there are changes pending for devices in the DSC group.This option is not recommended, because it can lead to unpredictable results.
- Ignore BIG-IP DSC sync when deploying configuration changes: Select this option if you want to manually synchronize configurations changes between members in the DSC group.
- Click theAddbutton at the bottom of the screen.The BIG-IQ system opens communication to the BIG-IP device, and checks the BIG-IP device framework.The BIG-IQ system can properly manage a BIG-IP device only if the BIG-IP device is running a compatible version of the REST framework.
- If a framework upgrade is required, in the popup window, in theRoot User NameandRoot Passwordfields, type the root user name and password for the BIG-IP device, and clickContinue.
- If in addition to basic management tasks (like software upgrades, license management, and UCS backups) you also want to centrally manage this device's configurations for licensed services, select the check box next to each service you want to discover.You can also select these service configuration after you add the BIG-IP device to the inventory.
- To enable statistics collection for this BIG-IP device, under Statistics monitoring, select the check box next to each service you want to collect statistics for, and then clickContinue.For Network Security, enable theAFM Statistics Collection.If you want to enable statistics collection without managing any services, clear the check boxes for all services.
- Click theAddbutton at the bottom of the screen.
Enable statistics collection for devices
Before you can enable statistics collection for a BIG-IP device:
- The device must already be in the BIG-IQ system inventory.
- There must be a BIG-IQ data collection device configured for the BIG-IQ device.
- For BIG-IP device version 13.1.0.5 or later, AVR must be provisioned.
To collect statistics for a BIG-IP device, you enable statistics collection when you discover it, however, you can enable or disable statistics collection for a device any time it is convenient for you. When enabling statistics for services and modules (including AFM), local storage is disabled on the device to prevent data duplication and extraneous resource usage.
- At the top of the screen, clickDevices.
- Click the name of the device you want to enable statistics collection for.
- On the left, clickStatistics Collection.
- To begin statistics collection, forCollect Statistics Data, selectEnabled.If this option is disabled, no statistics will be collected from your device, regardless of service or module.
- ForModules/Services, click the check box to enable statistics collection for specific system modules and services.
- ForAFM Statistics CollectionselectEnabledto collect statistics from your device's Network Security module.
- ForFrequency, next toCollect every, select the interval at which you want to collect statistics from this device.
- ClickSave & Close.
After you enable statistics collection for a device, data for that device begins aggregating along with any other devices for which you are collecting data. Two buttons (
View Health Statistics
, and View Traffic Statistics
) are added to the properties page for enabled devices. Clicking either of these takes you directly to the overview page for the statistics type you clickedStatistics retention policy overview
When you choose how much raw data to retain, you need to consider how
much disk space you have available. The controls on this screen are simple to set up, but
understanding how they work takes a bit of explanation.
The fields on the Statistics Retention Policy screen all work in similar
fashion. One way to understand how these fields work is to think of your data storage space
as a set of containers. The values you specify on this screen determine how much storage
space each container consumes. Because data is saved for the time periods you specify, the
longer the time period that you specify, the more space you consume. The disk storage that
is consumed depends on several factors.
- The number of BIG-IP devices you manage
- The number of objects on the BIG-IP devices you manage (for example, virtual servers, pools, pool members, and iRules)
- The frequency of statistics collection
- The data retention policy
- The data replication policy
How long is data in each container retained? | Data is retained in each container for the time
period you specify. When the specified level is reached, the oldest chunk of
data is deleted. For example, if you specify a raw data value of 48 hours, then
when 48 hours of raw data accumulate, the next hour of incoming raw data causes
the oldest hour to be deleted. |
When does data from one container pass on to the
next? | Data passes from one container to the next in
increments that are the size of the next (larger) container. That is, every 60
minutes, the last 60 minutes of raw data is aggregated into a data set and
passed to the Hour(s)
container. Every 24 hours, the last 24 hours of hourly data is
aggregated into a data set and passed to the Day(s) container, and so on
for the Month(s)
container. |
What about limits? | Limit
Max Storage to specifies the percentage of total disk space that
you want data to consume on the data collection devices in your cluster. If more disk space is consumed than the percentage you
specified, BIG-IQ takes two actions:
|
The aggregation policy for your statistics data
There is a default statistics aggregation policy for the data added to
your data collection device. The aggregation policy impacts the quality of the entity data,
per dimension, over time. This optimizes the disk usage, and allows for high quality data
for short-term analysis and troubleshooting for raw, hour, or even day time layers of data.
Long term data storage provides insights into global statistics over time, but are not
recommended for troubleshooting.
Manage the retention policy for your statistics data
Before you can set the statistics retention policy, you must have added a data collection device.
You can manage the settings that determine how your statistics data is retained. The highest quality data is the raw data, (data that has not been averaged), but that consumes a lot of disk space, so you need to consider your needs in choosing your data retention settings.
- From BIG-IQ, at the top of the screen, clickSystem, then, on the left, clickBIG-IQ DATA COLLECTIONand then selectBIG-IQ Data Collection Cluster.The BIG-IQ Data Collection Cluster screen opens. On this screen, you can view summary status for the Data Collection Device (DCD) cluster and access the screens that you can use to configure the DCD cluster.
- UnderSUMMARY, you can access screens detailing how much data is stored, as well as how the data is stored.
- UnderCONFIGURATION, you can access the screens that control DCD cluster performance.
- Under the screen name, click.The Statistics Collection Status screen opens.
- Click theConfigure Retentionbutton.The Statistics Retention Policy screen opens.
- In theKeep real-time (raw) data up tofield, type the number of hours of raw data to retain.You must specify a minimum of 1 hour, so that there is sufficient data to average and create a data point for theKeep hourly data up tocontainer.
- In theKeep hourly data up tofield, type the number of hourly data points to retain.You must specify a minimum of 24 hours, so that there is sufficient data to average and create a data point for theKeep daily data up tocontainer.
- In theKeep daily data up tofield, type the number of daily data points to retain.You must specify a minimum of 31 days, so that there is sufficient data to average and create a data point for theKeep monthly data up tocontainer.
- In theKeep monthly data up tofield, type the number of monthly data points to retain.Once the specified number of months passes, the oldest monthly data set is deleted.
- In theLimit max storage tofield, type the percentage of disk space that you want collected data to consume before the oldest monthly data set is deleted.
- In theKeep events up tofield, type the number of days that you want keep events before the oldest events data set is deleted.
- In theKeep traffic capturing up tofield, type the number of days that you want keep captured traffic before the oldest traffic data set is deleted.
- Expand Advanced Settings:
- Select theReplicascheck box to enable high availability for the stored data on your DCD cluster.Replicasare copies of a data sets available to the DCD cluster when one or more devices within that cluster become unavailable. By default, data replication for statistics is enabled. Disabling replication reduces the amount of disk space required for data retention. However, this provides no protection from data corruption that can occur when you remove a DCD. You should enable replicas to provide this protection.
- Select theAuto expand replicascheck box to enable automatic duplication of the number of replicas for a specific data set.This allows the DCD cluster to dynamically host up to 2 separate replicas for a given data set, based on the number of DCDs available. This provides redundancy that protects from data loss even when more than one DCD becomes unavailable.This option is only available whenEnable Replicasis selected. In addition, your system must include at least 3 DCDs (one primary and two replicas) with sufficient disk space.
- When you are satisfied with the values specified for data retention, clickSave & Close.
About Analytics profiles
An
Analytics profile
is a set of definitions that determines the circumstances under which the managed BIG-IP system gathers, logs, notifies, and displays information regarding HTTP or TCP traffic to an application. Each monitored application is associated with an Analytics profile. You associate an Analytics profile with one or more virtual servers used by the application. Each virtual server can have one HTTP and/or one TCP Analytics profile associated with it.In the HTTP Analytics profile, you can customize:
- Which statistics to collect, and their collection value thresholds
- Location of data collection (locally, remotely, or both)
- Notifications
- Traffic capturing specifications (HTTP only)SeeTroubleshoot HTTP Traffic by Reviewing Captured Traffic for more information
The system includes a default HTTP Analytics profile called
analytics
and a TCP Analytics profile called tcp-analytics
. These serve as the parent of all other Analytics profiles that you create on the system. You can modify the default profile, or create custom Analytics profiles for each application if you want to track different data for each one. Certain settings, such as SMTP Configuration, Transaction
Sampling, and the Subnets list, can only be set in the default HTTP Analytics profile.
Analytics profiles for AS3 applications
If you are managing AS3 applications, update analytics profiles using an AS3 template or directly add profile settings to the application declaration JSON. Information about the field attribute is listed within the customizing procedures. For the full reference index for analytics profile attributes, including traffic capturing, go to schema-reference.html.
For HTTP Analytics properties, see
Analytics_Profile
.For Traffic Capturing settings, see
Capture_Filter
For TCP Analytics properties, see
Analytics_TCP_Profile
.Customizing the default HTTP Analytics profile
Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.To view log messages on an external server, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
section of BIG-IQ Centralized Management: Local Traffic and Network Implementations
on support.f5.com
.An HTTP Analytics profile directs the system to store various HTTP statistics for troubleshooting application-layer issues. The system includes a default HTTP Analytics profile called
analytics
. You can edit the settings in the default profile so it uses the values you want.Certain information can be specified only in the default HTTP Analytics profile: the SMTP configuration (a link to an SMTP server), transaction sampling (whether enabled or not), and subnets (assigning names to be used in the reports). To edit these values, you need to open and edit the default profile.
- Go to.If you would like to edit the parent profile, selectanalyticsfrom the Parent Profile column and proceed to step 7.
- ClickCreate.The New Profile screen opens.
- ForName, type a name for the new profile.
- If required by your managed BIG-IP device, change theSiloandPartitionfield.
- From theParent Profilelist, select the profile from which you want to inherit settings.The default profile is often used as the parent profile.The new profile inherits the values from the parent profile. If the parent is changed, the inherited values in the new profile also change.
- To make all the fields editable, clickOverride All. This applied to both the Settings, Metrics Gathering Configuration, and Dimensions Gathering Configuration options.
- ForCollected Statistics Internal Logging, selectEnableto specify the system collects a portion of the application traffic data.AS3 AttributecollectedStatsInternalLoggingOnce enabled, you can manage which Dimension Gathering Configuration settings are collected.
- (Optional) To send HTTP traffic data to an external server, enableCollected Statistics External Logging.AS3 AttributecollectedStatsExternalLoggingTo specify Remote Publisher:externalLoggingPublisherSelecting this option allows you to choose aRemote Publisher. You must select a remote publisher configured to BIG-IQ to view log data using BIG-IQ.AS3 Attribute: externalLoggingPublisher
- ForCaptured Traffic Internal Logging, selectEnableto manage the Capture Filter settings.AS3 AttributecapturedTrafficExternalLoggingTo specify Remote Publisher:externalLoggingPublisherOnce traffic capturing is enabled, you can configure the capture criteria in theCapture Filterarea at the bottom of the screen. For more information about the dimension and metric options for traffic capturing, seeConfigure traffic capturing for troubleshooting.AS3 AttributecapturedTrafficInternalLogging
- (Optional) To send TCP traffic data to an external server, enableCollected Statistics External Logging.AS3 AttributecapturedTrafficExternalLoggingTo specify remote SMTP server:externalLoggingPublisherOnce you enable this field, you can select a pre-configured server from theSMTP Configurationfield.
- To send email alerts, specify anSMTP Configuration.You can change the SMTP configuration only in the default profile. It is used globally for the system. If no configuration is available, clickCreateto create one.
- For theNotification by...settings, enable the settings to send alerts and notifications.EntityDescriptionAS3 AttributeSyslogSelectSyslogif you want the system to send notification and alert messages to the local (Host BIG-IP) log system.notificationBySyslogSNMPSelectSNMPif you want the system to send notification and alert messages as SNMP traps. You create these settings directly on the host BIG-IP device.notificationBySnmpE-mailSelectE-mailif you want the system to send notification and alert messages to configured email addresses. Type each email address in theNotification E-Mailsfield, and click+to create the list. This option requires that the default analytics profile includes an SMTP configuration.notificationByEmailSpecify e-mail addresses:notificationEmailAddresses
- If you want the system to perform traffic sampling, make sure that forTransaction Samplingarea, theSamplecheck box is selected.You can change this setting only in the default profile.Sampling improves system performance. F5 recommends that you enable sampling if you generally use more than 50 percent of the BIG-IP system CPU resources, or if you have at least 100 transactions in 5 minutes for each entity.
- If you want the system to collect and display statistics, according to the expressions written in an iRule, enablePublish iRule Statistics.AS3 AttributepublishIruleStatisticsWhen you select this option, iRule statistics are visible per analytics profile. In addition, these iRule events are displayed in near real time (delay of 10 seconds), while statistics in the Configuration utility have a delay of at least 5 minutes. You can view iRule statistics per Analytics profile on the command line by typingISTATS dump.For the system to collect iRule statistics, you must also write an iRule describing which statistics the system should collect.
- In the Metrics Gathering Configuration area, enable additional statistics you want the system to collect from the HTTP requests:By default, the system collects many metrics, including TPS, throughput, server latency, response time, and network latency. You can select the metrics here, in addition to the ones already collected, once the HTTP Analytics profile is attached to one or more virtual servers.EntityDescriptionAS3 AttributeMax TPS and ThroughputCollects and logs statistics regarding the maximum number of transactions per second (TPS) and the amount of traffic moving through the system.collectMaxTpsAndThroughputPage Load TimeCollects and logs statistics regarding the time it takes an application user to get a complete response from the application, including network latency and completed page processing.End-user response times and latency can vary significantly based on geographic location and connection types.collectPageLoadTimeHTTP Timing (RTT, TTFB, Duration)Collects and logs statistics regarding the HTTP request and response times, including round-trip time, time to first byte and overall transaction duration time.N/AUser SessionsCollects and logs statistics regarding the number of unique user sessions.collectUserSessionCookie Secure AttributeSpecifies how to log secure session cookies:
- Always, the secure attribute is always added to the session cookie.
- Never, the secure attribute is never added to the session cookie.
- Only SSL, the secure attribute is added to the session cookie only when the virtual server has a client SSL profile (the default value).
sessionCookieSecurityTimeoutLogs data by the allowed minutes of user inactivity before the system considers the session to be over.sessionTimeoutMinutes - In the Dimensions Gathering Configuration area, enable additional entities to collect statistics for each request.By default, the system collects many entity statistics, including virtual servers, pool members, browser names, and operating systems You can select the ones here, in addition to the ones already collected, once the HTTP Analytics profile is attached to one or more virtual servers.When you selectURLs,Countries,Client IP AddressesorClient Subnetsyou have additional options configure specific statistics filtering options.EntityDescriptionAS3 AttributeURLsCollects all, or only specified, URLs.collectUrlTo specify URLs:urlsForStatCollectionCountriesCollects all, or only specified, countries. Country information is based on where the request came from, and is based on the client IP address criteria.collectGeoTo specify countries:countriesForStatCollectionClient IP AddressesCollects all, or only specified, IP address. IP address information is based on where the request originated. The address saved also depends on whether the request has an XFF (X-forwarded-for) header and whether the HTTP profile accepts XFF headers.collectIpClient SubnetsCollects statistics for predefined client subnets. Client subnets can be added in the Subnets area of the default HTTP Analytics profile.collectSubnetTo specify subnets:subnetsForStatCollectionResponse CodesCollects HTTP response codes that the server returned in response to requests.collectResponseCodeUser AgentsCollects information about browsers making the request.collectUserAgentMethodsCollects HTTP methods in requests.collectMethod
- When you are done clickSave & Close.
Virtual servers and applications configured to this profile collect and report traffic statistics according to specified settings. For more information about how to view an analyze application traffic, see
Monitoring and Managing Applications Using BIG-IQ
on support.f5.com
.Customizing the default TCP Analytics profile
Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.To view log messages on an external publisher, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
section of BIG-IQ Centralized Management: Local Traffic and Network Implementations
on support.f5.com
.A TCP Analytics profile directs the system to store TCP statistics about specific entities for use in diagnosing network problems. The system includes a default TCP Analytics profile called
tcp-analytics
. You can edit the values in the default profile, or create a new one, as described here. - Go to.If you would like to edit the parent profile, selecttcp-analyticsfrom the Parent Profile column and proceed to step 7.
- ClickCreate.The New Profile screen opens.
- ForName, type a name for the new profile.
- If required by your managed BIG-IP device, change theSiloandPartitionfield.
- From theParent Profilelist, select the profile from which you want to inherit settings.The default profile is often used as the parent profile.The new profile inherits the values from the parent profile. If the parent is changed, the inherited values in the new profile also change.
- To make all the fields editable, clickOverride All. This applied to both the Settings and Dimensions Gathering Configuration options.
- ForCollected Statistics Internal Logging, selectEnableto manage the Dimension Gathering Configuration settings.AS3 AttributecollectedStatsInternalLogging
- (Optional) To send TCP traffic data to an external server, enableCollected Statistics External Logging.AS3 AttributecollectedStatsExternalLoggingTo specify Remote Publisher:externalLoggingPublisherSelecting this option allows you to choose aRemote Publisher. You must select a remote publisher configured to BIG-IQ to view log data on an external server.
- ForCollected Statistics By Server Sideenable to specify that statistics from the server side of the TCP transaction are collected.AS3 AttributecollectedByServerSide
- ForCollected Statistics By Client Sideenable to specify that statistics from the client side of the TCP transaction are collected.AS3 AttributecollectedByClientSide
- From Dimensions Gathering Configuration, select the entities for which you want the system to collect information.The more entities you enable, the greater the impact on system performance.EntityDescriptionAS3 AttributeCityCollects the name of the city with which traffic was exchanged.collectCityCountryCollects the name of the country with which traffic was exchanged.collectCountryContinentCollects the name of the continent with which traffic was exchanged.collectContinentNext Hop Ethernet AddressCollects the addresses to which traffic is being routed.collectNexthopPost CodeCollects the name of the postal code with which traffic was exchanged.collectPostCodeRemote Host IP AddressCollects the IP addresses with which traffic was exchangedcollectRemoteHostIpRegionCollects the name of the region with which traffic was exchanged.collectRegionRemote Host SubnetCollects the addresses of the subnets with which traffic was exchanged.collectRemoteHostSubnet
- When you are done clickSave & Close.
Virtual servers and applications configured to this profile collect and report traffic statistics according to specified settings. For more information about how to view an analyze application traffic, see
Monitoring and Managing Applications Using BIG-IQ
on support.f5.com
.Reviewing captured traffic details
Traffic capturing prompts the system to log traffic request
and response headers and payload data, based on specific collection
requirements. You enable traffic capturing in your Analytics profile to
monitor a known application issue, such as trouble with throughput or
latency, or a known factor that can impact application performance, such
as HTTP method, or client IP address. You can specify these traffic
aspects to later examine application statistics, and troubleshoot
captured transactions.
Once enabled, you can examine the captured traffic to explore details, such as
the payload of captured transactions, requested URLs and response size.
When traffic capturing is enabled, you can view data about captured
traffic within the charts for HTTP traffic statistics.
Configure traffic capturing for troubleshooting
Before you begin, you need to ensure that AVR is provisioned on your managed BIG-IP devices, and that Statistics Collection is enabled on your BIG-IQ per device (
). Enabling Statistics Collection ensures that traffic data from BIG-IP is logged on BIG-IQ.To view log messages on an external server, you must configure a Remote Publisher. For more information about configuring a Remote Publisher, see the
Managing Logs
section of BIG-IQ Centralized Management: Local Traffic and Network Implementations
on support.f5.com
.You can configure your HTTP analytics profile to capture traffic headers and additional transaction details. Once configured, you can review captured traffic, based upon specific transaction parameters and performance thresholds.
- Go to.This screen lists the profiles that are configured for the managed BIG-IP devices in your network.
- Select the HTTP Analytics profile you wish to edit.Theanalyticsprofile is a default profile for all HTTP Analytics management. If you are creating a new HTTP Analytics profile, make sure to select theOverride Allcheck box to change the settings inherited by the parent profile.
- ForCaptured Traffic Internal Logging, selectEnableto manage the Capture Filter settings.AS3 AttributecapturedTrafficInternalLoggingOnce you enable a traffic capturing, the Capture Filter area becomes available. This allows you to further configure which traffic you would like to capture.
- (Optional) To send captured traffic to an external server, enableCaptured Traffic External Logging.AS3 AttributecapturedTrafficExternalLoggingTo specify Remote Publisher:externalLoggingPublisherOnce you enable this field, you can select a pre-configured server from theRemote Publisherfield.
- From theCapture Request DetailsandCapture Response Detailslists, select the options that indicate the part of the traffic to capture.Detail options for request and response capture:EntityDescriptionNoneSpecifies that the system does not capture request (or response) data.HeadersSpecifies that the system captures request (or response) header data only.BodySpecifies that the system captures the body of requests (or responses) only.AllSpecifies that the system captures all request (or response) data, including header and body.EntityAS3 AttributeCapture Request DetailsrequestCapturedPartsCapture Response DetailsresponseCapturedParts
- ForDoS Activity, select the option that indicates which DoS traffic is captured.OptionDescriptionAnySpecifies that the system captures any traffic regardless of DoS activity.Mitigated by Application DoSSpecifies that the system only captures DoS traffic if it was mitigated.AS3 AttributedosActivity
- ForProtocols, specify whether the system capturesAlltraffic, or traffic withHTTP, orHTTPSprotocols.AS3 AttributecapturedProtocols
- ForQualified for JavaScript Injection, you can selectQualified onlyto specify that the system only captures traffic that qualifies for JavaScript injection, which includes the following conditions:
- The HTTP content is not compressed
- The HTTP content-type istext/html.
- The HTTP content contains an HTML<head>tag
AS3 AttributecapturedReadyForJsInjection - Customize the dimension filters, according to your application needs, to capture the portion of traffic to that you need for troubleshooting.Dimension filters capture traffic according to defined aspects of the transaction's configuration, or header/payload contents. By focusing in on the data and limiting the type of information that is captured, you can troubleshoot particular areas of an application more efficiently. For example, capture only requests or responses, specific status codes or methods, or headers containing a specific string.EntityDescriptionAS3 AttributeResponse Status CodesSelectAllto capture traffic, regardless of the HTTP status response code.SelectOnlyto capture traffic with specific response status codes. To specify, add response status codes to theSelected Status Codeslist from theAvailable Status Codeslist.responseCodesHTTP MethodsSelectAllto capture traffic, regardless of the HTTP request method.SelectOnlyto capture traffic with requests that contain a specific HTTP method. To specify, add methods to theSelected Methodslist from theAvailable Methodslist.methodsURLSelectAllto capture traffic with requests for any URL.SelectStarts Withto only capture traffic with requests for URLs that start with a specific string.If you select this option, and leave the list blank, the system will not capture any traffic.SelectDoes not start withto capture traffic with requests for URLs except for those that start with a specific string.You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic with requests for any URL.urlFilterTypeTo add URL prefixes:urlPathPrefixesUser AgentSelectAllto capture traffic sent from any browser.SelectContainsto only capture traffic sent from a browser that contains a specific string.You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic sent from any browser.userAgentTo add User Agent substringsuserAgentSubstringsClient IP AddressSelectAllto capture traffic sent to, or from, any client IP address.SelectOnlyto only capture traffic sent to or from a specific client IP address.You can add up to 10 different IP addresses to the list. If the list is blank, the system will capture traffic sent to, or from, any IP address.clientIpsRequest Containing StringSelectAllto capture all traffic.SelectSearch infilter captured traffic that includes a specific string contained in the request.requestContentFilterSearchStringResponse Containing StringSelectAllto capture all traffic.SelectSearch infilter captured traffic that includes a specific string contained in the response.responseContentFilterSearchString
- ClickSave & Close.
Your
analytics
profile is now configured for traffic capturing.You can assign this profile to your virtual servers, if they do not yet have an Analytics profile configured.Review captured traffic
To display captured traffic, your virtual server must
be assigned an HTTP analytics profile that has captured traffic enabled, with external
logging.
You can troubleshoot details of captured HTTP
traffic to your applications and virtual servers. This information can provide details
of request/response headers and payload sent to your managed application. Captured
traffic information is found within the following dashboards that provide HTTP traffic
visibility:
- Device Traffic:.
- DDoS HTTP Analysis:.
- Local Traffic:.
- Navigate to one of the monitoring dashboards that display HTTP traffic data.
- Select theTraffic Capturingbutton above the charts.Selecting this option overlays captured traffic data over the charts, and adds a traffic capturing filter in the Dimensions pane.
- To filter captured traffic based on a specific host object, such as a BIG-IP system (BIG-IP Host Names), application (Applications Services), or virtual server (Virtual Servers), expand the dimension widgets in the Dimensions pane to the right of the charts.You can select multiple dimension objects from multiple dimensions. With each selection, the charts and dimensions filter displayed data according to your selections.
- To filter captured traffic based on server latency and payload volume metrics, expand theTraffic Capturing Filtersfound in the dimensions pane.For latency metrics, you can enter a range, or set a greater or less than filter value.
- To view traffic details, select a traffic capturing icon from within the chart to display an information table.You can click the rows within the displayed table to view additional request/response header and payload information.