Manual Chapter : Troubleshooting Applications by Capturing Traffic

Applies To:

  • BIG-IP AAM

    15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP APM

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP Link Controller

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP Analytics

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP LTM

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP PEM

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP AFM

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP DNS

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

  • BIG-IP ASM

    21.0.0, 17.5.1, 17.5.0, 17.1.3, 17.1.2, 17.1.1, 17.1.0, 17.0.0, 16.1.6, 16.1.5, 16.1.4, 16.1.3, 16.1.2, 16.1.1, 16.1.0, 16.0.1, 16.0.0, 15.1.10, 15.1.9, 15.1.8, 15.1.7, 15.1.6, 15.1.5, 15.1.4, 15.1.3, 15.1.2, 15.1.1, 15.1.0

Troubleshooting Applications by Capturing Traffic

This implementation describes how to set up the BIG-IP system to collect application traffic so that you can troubleshoot problems that have become apparent by monitoring application statistics. For example, by examining captured requests and responses, you can investigate issues with latency, throughput, or reduced transactions per second to understand what is affecting application performance.

When Application Visibility and Reporting (AVR) is provisioned, you can create an Analytics profile that includes traffic capturing instructions. The system can collect application traffic locally, remotely, or both. If the system is already monitoring applications, you can also update an existing Analytics profile to make it so that it captures traffic.

If logging locally, the system logs the first 1000 transactions and displays charts based on the analysis of those transactions. For VIPRION systems, the local logging consists of the first 1000 transactions multiplied by however many blades are installed. If logging remotely, the system logs information on that system; log size is limited only by any constraints of the remote logging system. To see updated application statistics, you can clear the existing data to display the current statistics.

  1. Capturing traffic for troubleshooting

  2. Reviewing captured traffic

After you finish a basic networking configuration of the BIG-IP system, you must complete these prerequisites for setting up application statistics collection:

  • Provision Application Visibility and Reporting (AVR): System > Resource Provisioning.
  • Create an iApps application service (go to iApp > Application Services), or configure at least one virtual server with a pool pointing to one or more application servers.

You can set up the system for capturing application traffic either locally or remotely (or both).

Tip: Before setting up, clear the captured transaction log. On the Captured Transactions screen, click Clear All to clear all previously captured data records.

Traffic capturing prompts the system to log traffic request and response headers and payload data, based on specific collection requirements. You typically use traffic capturing to monitor a known application issue, such as trouble with throughput or latency, or a known factor that can impact application performance, such as HTTP method, or client IP address. You can specify these traffic aspects to later examine application statistics, and troubleshoot captured transactions.

You can configure the BIG-IP system to capture application traffic and store the information locally or remotely (on Syslog servers or SIEM devices, such as Splunk). To do this, you create an Analytics profile designed for capturing traffic. The profile instructs the BIG-IP system to collect a portion of application traffic using the Application Visibility and Reporting (AVR) module.

  1. On the Main tab, click Local Traffic > Profiles > Analytics > HTTP Analytics.

    Tip: If Analytics is not listed, this indicates that Application Visibility and Reporting (AVR) is not provisioned, or you do not have rights to create profiles.

    The HTTP Analytics screen opens.

  2. In the Profile Name column, click analytics (the name of the default profile).

    Note: If you are creating or editing a profile other than the default analytics profile, you will need to select the Custom box located to the top right of the screen.

  3. In the General Configuration area, clear the Sample check box from the Transaction Sampling field.

    The system now analyzes all traffic to the associated virtual servers. This can improve the troubleshooting accuracy for captured traffic.

  4. Above the menu bar, click the Profiles: Analytics link to return to the Analytics list screen.

  5. Click Create.

    The New HTTP Analytics profile screen opens.

  6. In the Profile Name field, type a unique name for the Analytics profile.

  7. For Traffic Capturing Logging Type, specify where to store captured traffic.

    • To store traffic locally, click Internal. You can view details on the Captured Transactions screen (System > Logs > Captured Transactions).
    • To store traffic on a remote logging server, click External and provide the requested information.
  8. In the Associated Virtual Servers area, specify the virtual servers for which to capture application statistics:

    1. For the Virtual Servers setting, click Add.

    2. From the Select Virtual Server popup that opens, select the virtual servers to include and then click Done.

    Note: Only virtual servers previously configured with an HTTP profile display in the list (because the data being collected applies to HTTP traffic). Also, you can assign only one HTTP Analytics profile to a virtual server; therefore, the list displays only virtual servers that have not been assigned an Analytics profile.

    Special considerations apply if using Analytics on a BIG-IP system with both Application Security Manager and Access Policy Manager, where security settings (in Portal Access webtop or an iRule) redirect traffic from one virtual server to another. In this case, you need to attach the HTTP Analytics profile to the second virtual server to ensure that the charts show accurate statistics.

  9. If you added virtual servers in step 8, and want to make changes to any of the selections, above the Statistics Gathering Configuration area, select the Custom check box.

  10. In the Capture Filter area, enter a unique name for the capturing profile in the Capturing Profile Name area.

  11. From the Capture Request Details and Capture Response Details lists, select the options that indicate the part of the traffic to capture.

    Option Description
    None Specifies that the system does not capture request (or response) data.
    Headers Specifies that the system captures request (or response) header data only.
    Body Specifies that the system captures the body of requests (or responses) only.
    All Specifies that the system captures all request (or response) data, including header and body.
  12. For DoS Activity, select the option that indicates which DoS traffic is captured.

    Option Description
    Any Specifies that the system captures any traffic regardless of DoS activity.
    Mitigated by Application DoS Specifies that the system only captures DoS traffic if it was mitigated.
  13. For Protocols, specify whether the system only captures traffic with HTTP, or HTTPS protocols.

  14. For Qualified for JavaScript Injection, select Qualified only to specify that the system only captures traffic that qualifies for JavaScript injection, which includes the following conditions:

    • The HTTP content is not compressed
    • The HTTP content-type is text/html.
    • The HTTP content contains an HTML <head> tag
  15. Customize the dimension filters, according to your application needs, to capture the portion of traffic to that you need for troubleshooting.

    Tip: Dimension filters capture traffic according to defined aspects of the transaction’s configuration, or header/payload contents. By focusing in on the data and limiting the type of information that is captured, you can troubleshoot particular areas of an application more efficiently. For example, capture only requests or responses, specific status codes or methods, or headers containing a specific string.

Virtual Servers

Select All to capture traffic for all Virtual servers.Select Only to capture traffic from specific virtual servers. To specify, add virtual servers to the Selected Virtual Servers list from the Available Virtual Servers list.

Nodes

Select Allto capture traffic from all nodes.Select Only to capture traffic from specific nodes. To specify, add nodes to the Selected Nodes list from the Available Nodes list.

Response Status Codes

Select Allto capture traffic, regardless of the HTTP status response code.Select Only to capture traffic with specific response status codes. To specify, add response status codes to the Selected Status Codes list from the Available Status Codes list.

HTTP Methods

Select Allto capture traffic, regardless of the HTTP request method.Select Only to capture traffic with requests that contain a specific HTTP method. To specify, add methods to the Selected Methods list from the Available Methods list.

URL

Select All to capture traffic with requests for any URL.Select Starts With to only capture traffic with requests for URLs that start with a specific string.

Note: If you select this option, and leave the list blank, the system will not capture any traffic.

Select Does not start with to capture traffic with requests for URLs except for those that start with a specific string.

Note: You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic with requests for any URL.

User Agent

Select All to capture traffic sent from any browser.Select Contains to only capture traffic sent from a browser that contains a specific string.

Note: You can add up to 10 different strings to the list. If the list is blank, the system will capture traffic sent from any browser.

Client IP Address

Select All to capture traffic sent to, or from, any client IP address.Select Only to only capture traffic sent to or from a specific client IP address.

Note: You can add up to 10 different IP addresses to the list. If the list is blank, the system will capture traffic sent to, or from, any IP address.

Request Containing String

Select All to capture all traffic.Select Search in filter captured traffic that includes a specific string contained in the request.

Response Containing String

Select All to capture all traffic.Select Search in filter captured traffic that includes a specific string contained in the response.

  1. Customize the metric filters, to capture traffic based on statistic thresholds for a defined measurement over time. See the table below for a definition of each metric.

    1. To capture traffic based on a metric that is greater than a certain value, enter a number (0 or greater) under the Min field.

    2. To capture traffic based on a metric that is less than a certain value, select the check box next to the Max field and enter a value.

      If you do not select the check box, the max value setting will not be enabled.

    3. To capture traffic based on a metric range, enter values in both the Min and Max fields. Ensure you have select the box for enabling a maximum value.

Application Response Time

The time (in ms) from when the server receives the first request byte from the BIG-IP system until the server sends the first byte of the response.

Client TTBF

Client time to first byte (TTFB) is the time (in ms) from when the client sends the first byte of a request until the client receives the first byte of the response.

Client Side Network Latency

The client side network latency is the time (in ms) from when the client send the request to BIG-IP, until BIG-IP sends the response to the client (not including response/request duration times).

Request Duration

The time it takes (in ms) the BIG-IP system to send the first byte until the last byte of a request to the server.

Response Duration

The time it takes (in ms) the BIG-IP system to send the first byte until the last byte of a response to the client.

Server Latency

Server latency is the time (in ms) from when the BIG-IP system sends the first request byte to the web application server, until the BIG-IP system receives the first response byte.

Server Side Network Latency

The server side network latency the time (in ms) from when BIG-IP sends the client request to the server, until BIG-IP sends the response to the client (not including response duration and application response time).

Request Size

The detected request size.

Response Size

The detected response size.

  1. Click Finished.

The BIG-IP system captures the application traffic described by the Analytics profile for 1000 transactions locally (or until system limits are reached). If logging remotely, the system logs information on that system; log size is limited only by constraints of the remote logging system.

Note: System performance is affected when traffic is being captured.

Before you can review captured traffic details on the BIG-IP system, you need to create an HTTP Analytics profile that is capturing application traffic locally. The settings you enable in the Capture Filter area of the profile determine what information the system captures. You need to associate the Analytics profile with one or more virtual servers, or with an iApps application service.

The system starts capturing application traffic as soon as you enable it on the HTTP Analytics profile. You can review the captured transactions locally on the BIG-IP system. The system logs the first 1000 transactions. On a VIPRION system, the system logs the first 1000 transactions multiplied by however many blades are installed.

  1. On the Main tab, click System > Logs > Captured Transactions.

    The Captured Transactions screen opens and lists all of the captured transactions.

  2. Optionally, use the time period and filter settings to limit which transactions are listed.

  3. In the Captured Traffic area, click any transaction that you want to examine.

    Details of the request display on the screen.

  4. Review the general details of the request.

    Tip: The general details, such as the response code or the size of the request and response, help with troubleshooting.

  5. For more information, click Request or Response to view the contents of the actual transaction.

    Review the data for anything unexpected, and other details that can help troubleshoot the application.

  6. On the Captured Transactions screen, click Clear All to clear all previously captured data records (including those not displayed on the screen) and start collecting transactions again.

    The system captures up to 1000 transactions locally and displays them on the screen. Captured transactions are visible a few seconds after they occur.