Manual Chapter : Logging Bot Defense requests

Applies To:

  • BIG-IQ Centralized Management

    8.4.0, 8.3.0, 8.2.0, 8.1.0, 8.0.0

Logging Bot Defense requests

BIG-IQ receives Bot Defense messages from BIG-IP via it’s Data Collection Devices (DCD). To optimize the process, while ensuring high availability, it is best to load balance log events to a remote logging pool of DCDs. This will prevent data loss, in the instance that a DCD becomes unavailable, without unnecessary duplication of information.

To complete this process for Bot Defense, you must have previously configured the following:

  • An imported and discovered BIG-IP device that hosts your Bot Defense profile and Bot Request logging profile.
  • A remote logging pool of DCDs configured to the service port number 8514.

For more information about configuring a remote pool of DCDs, see Connect Devices to a Data Collection Device Cluster in the Planning and Implementing a BIG-IQ Deploymentguide at support.f5.com.

Note: If you have already created or imported your logging profile, use this process to adjust the existing settings to include the remote logging pool of DCDs.

You must create a remote logging pool for the DCDs configured to the service port of your module. For more information see Connect Devices to a Data Collection Device cluster in the Planning and Implementing a BIG-IQ Deployment guide at support.f5.com.

Create a Remote High-Speed Log and Splunk-type Log Destination to specify that log messages are sent to your pool of DCDs.

  1. At the top of the screen, click Configuration, then, on the left, click LOCAL TRAFFIC > Logs > Log Destinations.

    The Log Destinations screen displays a list of the log destinations that are defined on this device.

  2. Click Create.

  3. Type a unique Namefor this destination.

  4. From the Type list, select Remote High-Speed Log

  5. From the Protocol list, select TCP.

  6. From the Device list, select the BIG-IP device that hosts your service module’s policy or profile.

  7. From the Pool list, select your pool of DCDs.

  8. Click Save & Close.

    The Log Destinations screen opens.

  9. Click Create.

  10. Type a unique Name for this destination.

  11. From the Type list, select Splunk.

  12. Under the Forward To field, select Remote High-Speed Log, and select the Remote High-Speed log saved in step 8.

  13. Click Save & Close.

You have now designated your DCD pool as a remote destination for BIG-IP to send its logging data. If your system has multiple modules that require event logging, ensure that you repeat this process for the module’s designated DCD pool.

Create a Log Publisher to specify that BIG-IP system sends log messages to BIG-IQ. When configuring your Log Publisher ensure you are adding the Splunk-type Log Destination.

Before you configure monitoring of bot requests, you need to ensure that the Web Application Security service is enabled on the DCD.

Verify this by reviewing the services installed on the DCD on the BIG-IQ Data Collection Devices screen. Click System > BIG-IQ DATA COLLECTION > BIG-IQ Data Collection Devices.

If the Web Application Security service is not running, click Activate to start it.

Note: If you deactivate the Web Application Security service for a DCD, or remove a DCD with that service enabled, the associated pool member will be removed from the pool when you next deploy to the BIG-IP device (or devices).

You configure the collection and viewing of Bot Defense requests so that you can better view and monitor information about your bot protection. The BIG-IQ Centralized Management system provides a single-button configuration process that creates and configures the needed configuration objects. The system automatically creates the following configuration objects, if needed:

  • One or more logging profiles
  • A log publisher
  • A log destination
  • A pool for each device
  • Pool members
  • A pool monitor
  1. Click Configuration > SECURITY > Shared Security > Virtual Servers.

  2. In the list, select the check box to the left of the object that will host the logging profile.

  3. Click Manage Logging and select Configure Bot Logging.

    The Bot Logging Configuration dialog box opens.

  4. In the dialog box, click Continue.

    The dialog box shows the configuration status, including which objects were created.

  5. Click Close.

  6. Use the Deployment screens to deploy the BIG-IP device associated with the virtual server using the Local Traffic service using these steps.

    1. Click Deployment > EVALUATE & DEPLOY > Local Traffic & Network.

    2. In the Deployments area, click Create.

    3. Specify a Name and Description, and select the appropriate deployment options.

    4. In the Target Device(s) area, select the device used by the application and click Create.

    The deployment causes some of the objects created by the Bot logging configuration process to be deployed to the device.

  7. Deploy the same BIG-IP device using the Web Application Security service using these steps.

    You can use either service since both include the Shared Security objects.

    1. Click or Deployment > EVALUATE & DEPLOY > Web Application Security.

    2. In the Deployments area, click Create.

    3. Specify a Name and Description, and select the appropriate deployment options.

    4. In the Target Device(s) area, select the device used by the application and click Create.

    The deployment causes the rest of the objects created by the Bot logging configuration process to be deployed to the device.

You have now configured your logging profile to send Bot Defense requests from the BIG-IP devices associated with the virtual servers. Once you have deployed your changes, you can view these events on Monitoring > EVENTS > Bot screens.

To view or manage your logging profile, go to Configuration > SECURITY > Shared Security > Logging Profiles and select your DoS logging profile name.

To ensure that data is load balanced among your DCD devices, you must change the remote log destination. For more information see Edit log publisher destinations.

Note: Once you have completed this process, ensure that all your changes to your Local Traffic and Shared Security virtual servers are deployed over the host BIG-IP device. You can deploy your changes by going to, Deployment > EVALUATE & DEPLOY > Local Traffic & Network Deployment > EVALUATE & DEPLOY > Shared Security

You must have created the log destination before you can add it to the an existing Log Publisher. For more information see Managing Logs in support.f5.com.

Edit the Log Publisher destination settings to change the pools that receive remote logging messages from BIG-IP.

  1. At the top of the screen, click Configuration, then, on the left, click LOCAL TRAFFIC > Logs > Log Publishers.

    The screen displays a list of the Log Publishers that are defined on this device.

  2. Select the name of the log publisher you wish to edit.

    The log publisher properties screen opens.

  3. To add log destinations, select the Log Destination(s) from the Available list and use the arrow to move your selection to the Selected list.

    You can filter the Available list by selecting the type of destination from the drop-down list.

  4. To remove log destinations, select the Log Destination(s) from the Selected list and use the arrow to move your selection to the Available list.

  5. Click Save & Close

You have changed the remote destinations associated with the Log Publisher. This will alter where the BIG-IP device sends its log data.

Deploy changes to your BIG-IP device.

Before you can log bot requests, you must first have the following:

  • One or more BIG-IP devices that are provisioned to have Bot Defense.
  • A remote logging pool of your DCDs that is connected to a virtual server deployed over a load balancing BIG-IP device.
  • Web Application Security is active for DCD services (see System > BIG-IQ DATA COLLECTION DEVICES > SERVICES)

Important: The following procedure is for Bot Defense profiles configured to BIG-IP devices version 14.1 or later. For logging bot request information from earlier versions of BIG-IP, see Configuring logging for DoS Protection and Network Security.

You can view bot request information by attaching a logging profile to the virtual servers that host your Bot Defense profile. To access Bot Defense information, you need to configure the BIG-IP system to send log information to BIG-IQ. This is done by:

  • Creating a log publisher and pin it to your BIG-IP device(s)
  • Creating and attaching a bot request logging profile in Shared Security
  • Deploying your changes over your BIG-IP device(s)

For more details about specific settings within the logging profile, see Configure logging for Bot Defense requests**.**

  1. Click Configuration > SECURITY > Shared Security > Logging Profiles.

  2. Click Create to create a remote bot logging profile.

  3. Type a unique Name for this logging profile.

  4. On the left, click BOT DEFENSE.

  5. For Status, select the Enabled check box.

    The screen displays the Bot Defense request logging properties.

  6. From the Remote Publisher list, select the logging publisher for your DCD pool.

  7. Enable the for the appropriate request types of logging in the remaining fields.

  8. When you are done, click Save & Close.

  9. Attach the new logging profile to a Shared Security virtual server.

    1. Go to Configuration > SECURITY > Shared Security > Virtual Servers.

    2. Select the virtual server that hosts your Bot Defense profile.

    3. From the Logging Profiles field, select the logging profile created in step 6, and use the arrow to move it to the Selected list.

    4. Click Save & Close.

    5. Repeat step 6 for any additional virtual servers that host Bot Defense profiles.

  10. Deploy your new pool, log destinations and log publisher over your BIG-IP device.

    1. Go to Deployment > EVALUATE & DEPLOY > Local Traffic & Network.

    2. In the Deployments list at the bottom half of the screen and click Create.

    3. In the Name field add a unique name.

    4. Ensure that Source and Source Scope fields are marked Current Changes and All Changes, respectively.

    5. From the Target Devices list, select the host BIG-IP device(s) over which to deploy changes.

    6. Click Create.

      The deployment is added the to Evaluations list.

    7. Once the evaluation is complete, click the box next to the deployment name and click Deploy.

    The new local traffic objects are deployed over the BIG-IP device.

  11. Deploy changes to your Shared Security virtual server.

    1. Go to Deployment > EVALUATE & DEPLOY > Shared Security.

    2. Repeat steps 10b-g.

      The new logging profile on your Shared Security virtual server is now deployed over the BIG-IP device.

You can now monitor detected bot requests from the bot request log, from Monitoring > EVENTS > Bot > Bot Requests.