Manual Chapter : Configuring Statistics Collection

Applies To:

  • BIG-IQ Centralized Management

    7.0.0, 6.1.0

Configuring Statistics Collection

Before you can enable statistics collection for centralized management, you must ensure that the BIG-IP device has the proper configuration. The proper configuration varies depending on the version of the BIG-IP device. The minimum supported BIG-IP device is version 12.1.0. BIG-IQ has limited visibility for BIG-IP devices prior to 13.1.0.5.

For details about how to configure statistics visibility, based on the BIG-IP version, see Enabling statistics collection during device discovery.

For details on how to access statistic information, based on the BIG-IP version and service, refer to Statistics compatibility and visibility.

To start viewing statistics for a BIG-IP® device, you must enable statistics collection for that device. You can do that either during or after adding the device to the BIG-IP Devices inventory list on the BIG-IQ® system. You also need to install, configure, and add a data collection device before you can view device statistics.

Before you can enable statistics for BIG-IP devices:

  • There must be a BIG-IQ data collection device (DCD) configured for the BIG-IQ device.
  • The BIG-IP device must be located in your network and running a compatible software version. Refer to https://support.f5.com/csp/article/K34133507#cm6.0.1 for more information.
    • For BIG-IP devices running version 13.1.0.5 or later, you must have AVR provisioned.
  • For BIG-IP devices running versions prior to 13.1.0.5, configure Ports 22 and 443 must be open to traffic from the BIG-IQ DCD to the managed BIG-IP devices.
  • For BIG-IP devices running version 13.1.0.5 or later, BIG-IQ needs to provide access on Port 443 so that the BIG-IP AVR module can send statistics to the BIG-IQ DCD.

One way to enable statistics collection for BIG-IP devices is to do it when you add those devices to the BIG-IQ system inventory. Adding devices to the inventory is referred to as device discovery. If the devices you want to enable have already been discovered, refer to Enabling collection after device discovery.

Note: The ADC component is automatically included (first) any time you discover or import services for a device.

Note: You do not need to discover and import a device’s configuration to collect and view statistics for it. You just need to establish trust between your BIG-IQ and the device. If you do not discover and import the device configuration, the virtual servers, pool, pool members, and iRules will be visible in the statistics dimension panes, but these objects will not appear in the configuration page for those objects. Also, you will not be able to manage these objects in BIG-IQ. If you decide you want to manage these objects, you can discover and import the BIG-IP device’s configuration later without interrupting statistics collection.

  1. At the top of the screen, click Devices.

  2. Click the Add Device button.

  3. In the IP Address field, type the IPv4 or IPv6 address of the device.

  4. In the User Name and Password fields, type the user name and password for the device.

  5. If this device is part of a Device Service Cluster (DSC) group, for the Cluster Display Name setting, specify how to handle it:

    • For an existing DSC group, select Use Existing from the list, and then select the name of the DSC group from the next list.
    • To create a new DSC group, select Create New from the list, and type a name in the field. For BIG-IQ to properly associate the devices in the same DSC group, the Cluster Display Name must be the same for all members in a group.

    There can be up to 8 members in a DSC group.

    Important: For BIG-IP devices with ASM services, you can only add five devices at a time. If the BIG-IP device(s) provisioned with ASM is part of a DSC cluster, that device must also be a member of a sync-only device group, and ASM synchronization must be enabled on one of the devices in the DSC group. Without these DSC group settings, deploying changes to the ASM device can cause the cluster to become out of sync. For details on configuring these groups, refer to https://support.f5.com/csp/article/K12200102 or the ASM implementations chapter Automatically Synchronizing Application Security Configurations on support.f5.com.

  6. If this device is configured in a Device Service Clustering (DSC) group, for the Deployment Settings, specify how to handle it:

    Note: For more information about DSC groups, see What is a BIG-IP Device Service Clustering (DSC) group and how do I start managing it from BIG-IQ?.

    • Initiate BIG-IP DSC sync when deploying configuration changes (Recommended): Select this option if this device is part of a DSC group and you want BIG-IQ to automatically synchronize configuration changes with other members in the DSC group from BIG-IQ.

      • Allow deployment when DSC configured devices have changes pending (Not Recommended) this option pushes changes to all the devices in the cluster independently, ignoring the logical device cluster configuration. This option is intended for temporary use when a BIG-IP cluster node becomes unavailable, you would like to push changes to individual devices, until the node comes back online.

        Note: If multiple nodes are online, the entire BIG-IP DSC group will go into a Change Pending state. This can only be resolved by the system administrator.

    • Ignore BIG-IP DSC sync when deploying configuration changes: Select this option if you want to manually synchronize configurations changes between members in the DSC group.

  7. Click the Add button at the bottom of the screen.

    The BIG-IQ system opens communication to the BIG-IP device, and checks the BIG-IP device framework.

    Note: The BIG-IQ system can properly manage a BIG-IP device only if the BIG-IP device is running a compatible version of the REST framework.

  8. If a framework upgrade is required, in the popup window, in the Root User Name and Root Password fields, type the root user name and password for the BIG-IP device, and click Continue.

  9. If in addition to basic management tasks (like software upgrades, license management, and UCS backups) you also want to centrally manage this device’s configurations for licensed services, select the check box next to each service you want to discover.

    You can also select these service configuration after you add the BIG-IP device to the inventory.

  10. To enable statistics collection for this BIG-IP device, under Statistics monitoring, select the check box next to each service you want to collect statistics for, and then click Continue.

    Note: If you want to enable statistics collection without managing any services, just clear the check boxes for all services.

  11. Click the Add button at the bottom of the screen.

Before you can enable statistics collection for a BIG-IP device using this method:

  • The device must already be in the BIG-IQ system inventory.
  • There must be a BIG-IQ data collection device configured for the BIG-IQ device.

Generally, if you want to collect statistics for a BIG-IP device, you enable statistics collection when you discover it. But you can enable or disable statistics collection for a device any time it is convenient for you.

  1. At the top of the screen, click Devices.

  2. Click the name of the device you want to enable statistics collection for.

  3. On the left, click Statistics Collection.

  4. To begin statistics collection, for Collect Statistics Data, select Enabled.

  5. For Modules/Services, click the check box for the types of statistics you want to collect.

  6. For Frequency, next to Collect every, select the interval at which you want to collect statistics from this device.

After you enable statistics collection for a device, data for that device begins aggregating along with any other devices for which you are collecting data. Two buttons (View Health Statistics, and View Traffic Statistics) are added to the properties page for enabled devices. Clicking either of these takes you directly to the overview page for the statistics type you clicked.

When you choose how much raw data to retain, you need to consider how much disk space you have available. The controls on this screen are simple to set up, but understanding how they work takes a bit of explanation.

The fields on the Statistics Retention Policy screen all work in similar fashion. One way to understand how these fields work is to think of your data storage space as a set of containers. The values you specify on this screen determine how much storage space each container consumes. Because data is saved for the time periods you specify, the longer the time period that you specify, the more space you consume. The disk storage that is consumed depends on several factors.

  • The number of BIG-IP devices you manage
  • The number of objects on the BIG-IP devices you manage (for example, virtual servers, pools, pool members, and iRules)
  • The frequency of statistics collection
  • The data retention policy
  • The data replication policy

There are three key concepts to understand about how the retention policy works.

How long is data in each container retained?

Data is retained in each container for the time period you specify. When the specified level is reached, the oldest chunk of data is deleted. For example, if you specify a raw data value of 48 hours, then when 48 hours of raw data accumulate, the next hour of incoming raw data causes the oldest hour to be deleted.

When does data from one container pass on to the next?

Data passes from one container to the next in increments that are the size of the next (larger) container. That is, every 60 minutes, the last 60 minutes of raw data is aggregated into a data set and passed to the Hour(s) container. Every 24 hours, the last 24 hours of hourly data is aggregated into a data set and passed to the Day(s) container, and so on for the Month(s) container.

What about limits?

Limit Max Storage to specifies the percentage of total disk space that you want data to consume on the data collection devices in your cluster. If more disk space is consumed than the percentage you specified, BIG-IQ takes two actions:

  1. New statistical data is not accepted until the available disk space complies with the Limit max storage to setting.
  2. Statistical data not required to calculate the next higher time layer is removed (for example, you need 60 minutes of raw data to aggregate to the Hours level). Data is removed starting with the raw data container, then the hourly data container, then the daily time container. This process stops when storage consumption is below the Limit max storage to setting.

The BIG-IQ takes this action to prevent data corruption when storage is completely exhausted.

There is a default statistics aggregation policy for the data added to your data collection device. The aggregation policy impacts the quality of the entity data, per dimension, over time. This optimizes the disk usage, and allows for high quality data for short-term analysis and troubleshooting for raw, hour, or even day time layers of data. Long term data storage provides insights into global statistics over time, but are not recommended for troubleshooting.

Before you can set the statistics retention policy, you must have added a data collection device.

You can manage the settings that determine how your statistics data is retained. The highest quality data is the raw data, (data that has not been averaged), but that consumes a lot of disk space, so you need to consider your needs in choosing your data retention settings.

  1. At the top of the screen, click System, then, on the left, click BIG-IQ DATA COLLECTION and then select BIG-IQ Data Collection Cluster.

    The BIG-IQ Data Collection Cluster screen opens. On this screen, you can either view summary status for the data collection device cluster or access the screens that you can use to configure the DCD cluster.

    • Under Summary, you can view information detailing how much data is stored, as well as how the data is stored.
    • Under Configuration, you can access the screens that control DCD cluster performance.
  2. Under the screen name, click Configuration > Statistics Data Collection.

    The Statistics Collection Status screen opens.

  3. On the left, click Statistics Data Collection.

    The Statistics Collection Status screen displays the percentage of available disk space currently consumed by statistics data for each container.

  4. To change the retention settings for your statistics data, click Configure.

    The Statistics Retention Policy screen opens.

  5. In the Keep real-time (raw) data up to field, type the number of hours of raw data to retain.

    You must specify a minimum of 1 hour, so that there is sufficient data to average and create a data point for the Keep hourly data up to container.

  6. In the Keep hourly data up to field, type the number of hourly data points to retain.

    You must specify a minimum of 24 hours, so that there is sufficient data to average and create a data point for the Keep daily data up to container.

  7. In the Keep daily data up to field, type the number of daily data points to retain.

    You must specify a minimum of 31 days, so that there is sufficient data to average and create a data point for the Keep monthly data up to container.

  8. In the Keep monthly data up to field, type the number of monthly data points to retain.

    Once the specified number of months passes, the oldest monthly data set is deleted.

  9. In the Limit max storage to field, type the percentage of disk space that you want collected data to consume before the oldest monthly data set is deleted.

  10. Expand Advanced Settings, and then select the Enable Replicas check box.

    Replicas are copies of a data set that are available to the DCD cluster when one or more devices within that cluster become unavailable. By default, data replication for statistics is not enabled. Disabling replication reduces the amount of disk space required for data retention. However, this provides no protection from data corruption that can occur when you remove a data collection device. You should enable replicas to provide this protection.

  11. When you are satisfied with the values specified for data retention, click Save & Close.