Manual Chapter : Connect BIG-IP data logs to a Data Collection Device Cluster

Applies To:

  • BIG-IQ Centralized Management

    8.0.0

Updated Date: 07/07/2026

Connect BIG-IP data logs to a Data Collection Device Cluster

Data logging over multiple data collection devices (DCDs) streamlines the data collection process across your BIG-IQ system. This is done by load balancing log messages, sent from BIG-IP devices, among the multiple DCDs in your configuration. Completing this configuration ensures that your logging profiles are configured to have high availability, in the event that a DCD is unavailable, while preventing unnecessary duplication of data.

To ensure high availability among your DCDs, for the purpose of logging events from your BIG-IP devices, you need to configure a pool of DCDs. For more information about configuring the DCD pool, see Create a remote logging pool of DCDs.

Important: Each service module requires its own DCD pool. This ensures that the pool is configured with the proper port listeners for each module’s log messages. For modules that share the same port number, such as ASM and Bot Defense, you can use the same DCD pool.

Once you have created your DCD pool(s), you can configure remote logging to your BIG-IQ DCDs. Depending on your service module, this process may vary. For more information about each modules’ log configuration requirements, see Overview of configuring HA DCD logging per service module.

Using BIG-IQ, you can view and manage statistics and event data generated by the services on your BIG-IP devices. You collect BIG-IP logging data on a Data Collection Device (DCD) cluster. If your system configuration includes multiple DCDs it is recommended to create a pool that optimizes and streamlines data collection from your managed BIG-IP devices. Before you create and configure your logging profiles, you must complete this procedure. To create a pool of DCDs, see Create a remote logging pool of DCDs.

Note: If you only have one DCD in your configuration, you do not need to configure a remote logging pool, and you can connect your logging profile directly to your DCD. This option does not provide high availability and log messages will not be collected in the event that your DCD is not available.

The work flow to configure data to route from the BIG-IP devices to your data collection device (DCD) cluster depends on the type of data you want to collect and the services that run on your BIG-IP devices. The table below outlines the procedures required for each service module. For end to end configuration of high availability logging, including configuration of a logging profile using BIG-IQ, go to the documentation referred to in Logging profile configuration.

Note: If you have multiple modules, you need to configure one remote logging DCD pool per service port.

To configure

Procedure overview

Logging profile configuration

Web Application Security (ASM or Adv. WAF)
  1. Create a remote logging pool of DCDs for service port 8514.
  2. Create a logging profile that is configured to the remote logging pool of DCDs.

Important: Web Application Security does not require a Log Publisher. As such, it requires two separate BIG-IP host devices: One to distribute log messages over the DCD pool, and another to host the logging profile.

Managing Web Application Security Logging section in BIG-IQ Web Application Security

Bot Defense
  1. Create a remote logging pool of DCDs for service port 8514
  2. Create Log Destination for your DCD pool.
  3. Create a high-speed remote Log Publisher.
  4. Create a logging profile for Bot Defense and associate it with the Log Publisher. See Configure logging for Bot Defense Requests.

Logging Bot Defense Requests section in Managing Bot Defense Using BIG-IQ

Fraud Protection Services (FPS)

The following process is for BIG-IP devices running version 15.0 or higher. For information about setting up logging to BIG-IQ for versions 14.1 or earlier, see Configure BIG-IP a remote logging profile for BIG-IP FPS.Important: This process can only be done using the BIG-IP interface (version 15.0 or higher). You cannot create an FPS logging profile using BIG-IQ.

  1. Create a remote logging pool of DCDs for service port 8008
  2. Create Log Destination for your DCD pool.
  3. Create a high-speed remote Log Publisher.
  4. (optional) Create an Anti-Fraud logging profile on your host BIG-IP device. See Configure BIG-IP a remote Log Profile for BIG-IP FPS for version support information.
  5. On your BIG-IP device, associate the remote DCD pool and Log Publisher with the Anti-Fraud profile alerts.

Logging FPS over multiple DCDs section in BIG-IQ: Fraud Protection Service.

Dos Protection
  1. Create a remote logging pool of DCDs for service port 8020
  2. Create Log Destination for your DCD pool.
  3. Use the automated tool to create a logging profile directly on the virtual server. See Configure logging for DoS Protection and Network Security.
  4. Add the DCD pool Log Destination to the automatically generated remote Log Publisher.

Logging DoS Protection events section in Managing DDoS attacks Using BIG-IQ

Network Security (AFM)
  1. Create a remote logging pool of DCDs for service port 8018
  2. Create Log Destination for your DCD pool.
  3. Use the automated tool to create a logging profile directly on the virtual server.
  4. Add the DCD pool Log Destination to the automatically generated remote Log Publisher.

Logging Network Security events section in Managing Network Security Using BIG-IQ

Access (APM and IP Security)
  1. Create a remote logging pool of DCDs for service port 9997
  2. Create Log Destination for your DCD pool.
  3. Use the automated tool to create a logging profile. See Configure logging for Access Policy Manager.
  4. Add the DCD pool log destination to the automatically generated remote Log Publisher.

Logging Access events section in Centrally Managing Access Groups Using BIG-IQ

For this process you will need the following:

  • Three or more data collection devices (recommended).

    Note: This process only applies to BIG-IQ configurations with multiple DCD devices.

  • A logging profile that has remote storage enabled, and is attached to a virtual server on a BIG-IP device that hosts the service module and its policy.

  • [ASM/Web Application Security only] A load balancing BIG-IP device. This is a device that hosts the virtual server that load balances logging messages to the pool of DCDs, but does NOT host ASM service module policies or logging profiles.

To optimize data logging of messages from your BIG-IP devices to multiple DCDs, you can configure a BIG-IP system to load balance these messages among the DCDs in your BIG-IQ configuration. This process prevents duplication of information in the consolidated data repository, while also providing high availability for your log messages in the case that one or more DCDs become unavailable.

  1. At the top of the screen, click Configuration, then, on the left, click LOCAL TRAFFIC > Pools.

  2. Click Create

  3. Type a unique Name for the pool.

  4. From the Device list, select a load balancing BIG-IP device that provides the load balancing service to the DCD pool.

    Important: For ASM/ Web Application security logging information, be sure to select a BIG-IP device that is different from the device that hosts your virtual server with the service module policy.

  5. In the Health Monitors field, select the /Common/http option.

  6. For Node Type, select New Node.

  7. Add a Node Name (optional).

  8. Add the DCD IP address in the Address field and enter a service port for the Port field.

    Service module port numbers:

    Service Module Module in BIG-IP Port Number
    Web Application Security ASM and Adv. WAF 8514
    Bot Defense   8514
    Fraud Protection Service FPS 8008
    DoS Protection DoS Protection 8020
    Network Security AFM 8018
    Access (APM), IP Security (IPSec) APM and IP Security 9997

    Important: If you have multiple service modules sending logging data, you will need to create a separate DCD pool for each module’s port number.

  9. Ensure that State (on BIG-IQ) is Enabled.

  10. Click Save & Close

  11. Repeat steps 6-10 for all DCDs in your configuration.

  12. Create a virtual server to host your DCD pool

    1. Go to Configuration > LOCAL TRAFFIC > Virtual Servers.

    2. Click Create.

    3. From the Name field add a name.

    4. From the Device field, select the host device from step 4.

    5. In the Destination Address/Mask field add the IP address of the virtual server that hosts the logging profile.

      Warning: If your managed BIG-IP device uses high speed traffic logging (HSL) pools, you must apply the Self-IP address. You cannot apply the Management IP address. For more information about using HSL pools on BIG-IP devices, see https://support.f5.com/csp/article/K17398.

      Important: For Web Application Security, you must configure the DCD pool on a separate BIG-IP device from the device used to host the logging profile.

    6. In the Service Port field enter the service port numbers that matches your logging profile’s module (see table in step 8).

    7. In the Source Address Translation field select Auto Map.

    8. In the Resources area, click Default Pool and select the name of the DCD pool.

    9. Click Save & Close.

The load balancing configuration for your DCD pool is complete, you now need to ensure that the log messages from the virtual server that hosts your service module policy is directed to your newly configured virtual server.

Configure your remote logging protocol to the newly created virtual server that hosts the DCD pool. This process generally includes creation of a log publisher, log destination, and a logging profile, but varies depending on the service module. Ensure that you follow the procedures that match the logging profile’s module data. For more information about your module’s work flow, and its configuration documentation see Overview of configuring HA DCD logging per service module.