F5 Logo MyF5
Search
  1. MyF5 Home
  2. Managing Bot Defense Using BIG-IQ
  3. Configuring Bot Defense
Manual Chapter : Configuring Bot Defense

Applies To:

Show Versions Show Versions

BIG-IQ Centralized Management

  • 8.3.0, 8.2.0, 8.1.0, 8.0.0
Manual Chapter

Configuring Bot Defense

About bot defense profile templates

Bot defense profile templates specify
Mitigation Settings
 and
Browser Verification
 default values. When selecting a bot defense profile template, users should consider security and business requirements . It is also important to consider the BIG-IP version over which the policy is deployed. See the notes below for differences among the supported BIG-IP versions. For more information about device version support, see
Unified Bot Defense Version Compatibility.

Mitigation Settings

Bot Classes
Relaxed
Balanced
Strict
Trusted Bot
Alarm
Alarm
Alarm
Untrusted Bot
Alarm
Alarm
Block
Suspicious Browser
Alarm
CAPTCHA
Block
Malicious Bot
Block
Block
Block
Unknown
None
Rate limit
Block
Strict Mitigation Enforcement Cases
Relaxed
Balanced
Strict
DoS Attack Mitigation Mode
Disabled
Enabled
Enabled
API Access for Browsers and Mobile Applications
Disabled
Enabled
Enabled

Browser Verification and Device ID

Browser Access is enabled (
Allowed
) for all profile templates. This means that all browser types have access to the application, so long as the request passes browser verification challenges.
Relaxed
Balanced
Strict
Browser Verification
Challenge-free verification
Verify after access (blocking)
Verify before access
Device ID Mode
None
Generate after access
Generate before access

Bot defense relaxed template

A relaxed bot defense profile defines a permissive security policy that performs basic non-intrusive verification of browsers; strong verification of mobile apps using Anti-Bot Mobile Security SDK; blocks malicious bots and allows all other clients. Malicious bots are detected mostly by using bot signatures. This template provides basic protection with very low risk of false positives.

Bot defense balanced template

A bot defense balanced template defines a moderate security policy that performs advanced verification of browsers; strong verification of mobile apps using Anti-bot Mobile Security SDK; blocks malicious bots; initiates a CAPTCHA challenge for suspicious browsers; limits the total request rate produced by unknown bots and allows trusted and untrusted bots. Malicious bots and suspicious browsers are identified by using both anomaly detection algorithms and bot signatures. This template provides an advanced protection level with reduced latency impact because browser verification is performed by injecting the challenge in the HTTP response.

Bot defense strict template

A strict bot defense profile defines a strict security policy that performs advanced verification of browsers; strong verification of mobile apps using Anti-Bot Mobile Security SDK; and blocks all bots except trusted bots. This template provides the most advanced and strict protection level using all capabilities of bot defense. Browser clients are not allowed access unless they pass proactive verification. Mobile client security access requires the use of the Anti-Bot Mobile SDK.

Create a new bot defense profile

Because this defense mechanism uses reverse lookup, you need to configure a DNS Server (
System
Configuration
Device
DNS
) and a DNS Resolver (
Network
DNS Resolver
DNS Resolver List
) for it to work. The DNS Resolver must use the default route domain in its Route Domain Name field. If you are not sure of the default route domain, you can check it under
Network
Route Domains
. The Partition Default field is defined as Yes for the default route domain.
This task describes how to configure and save the general properties of a bot defense profile. The profile's mitigation and browser verification settings are based on the selected profile template, however you can later adjust the configuration to better suit your anti-bot needs.
Once you save your general properties settings, the
Profile Name
,
Partition
, and
Profile Template
 settings cannot be edited.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Click
    Create
    .
    The
    Bot Profile Configuration
     screen opens on the
    BOT PROPERTIES
     tab.
  3. Enter the
    Profile Name
     (required).
  4. Enter the
    Partition
    , if you do not want the default Common partition.
  5. Enter the
    Description
     of the profile.
  6. Select an
    Enforcement Mode
     depending on the readiness of your application environment and system protection requirements:
    Transparent
    The system logs traffic mitigation and verification settings, according to your logging profile settings, but does not provide the following:
    • JavaScript-based verification.
    • Device ID collection.
    • CAPTCHA challenge.
    Blocking
    The system performs traffic mitigation and verification. Actions are logged according to your logging profile settings.
  7. Select a
    Profile Template
     to determine your MITIGATION and BROWSER VERIFICATION settings.
    For more information about profile templates, see
    About bot defense profile templates
    .
  8. To enable
    Signature Staging Upon Update
    , select
    Enabled
    .
    By default this field is
    Disabled
    , and the system immediately enforces mitigation action, as defined by the mitigation settings. When enabled, the system will automatically put new or modified signatures in staging (either version update or custom signature creation). This means that the system logs and does not block, signature requests regardless of mitigation settings.
  9. To change the number of days a signature is in staging, select an option from the
    Enforcement Readiness Period
     field.
  10. For
    Redirect to Pool
     you must select one of your server pool redirects.
    This step is only for bot classes with the mitigation setting
    Redirect to Pool
    . If you do not have this mitigation action, or you plan to deploy this profile over a BIG-IP device running version 14.1, you do not need configure this field.
  11. If you have configured your mitigation settings to include
    CAPTCHA
    ,
    Blocking
    , or
    Honeypot Page
     you can select
    Custom Response
     to customize the HTML response message sent to a suspected bot request.
    The default response page varies based on the BIG-IP version. The response message provided in the view is for version 15.1. If you wish to have the same response, regardless of BIG-IP version, use the custom response setting. To view the response message click
    Preview On
    .
  12. Click
    Save & Close
    .
You have now configured the general settings a bot defense profile. The new profile can now be assigned to a virtual server.
You can adjust the profile's extended settings, which include:
  • Mitigation settings and exceptions
  • Browser verification settings
  • Mobile application client protection settings
  • Monitor and Mange signature enforcement
  • Manage allowlisted items

Add a bot mitigation exception

This task describes how to configure exceptions to a specific bot or bot type. Once you select your exception, you can choose the mitigation option that best protects your application. In addition, you can enable protection from pre-defined enforcement cases. When selecting exceptions to your template's mitigation settings, ensure that you have considered the BIG-IP versions to which this profile is deployed.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Select the name of the bot profile you would like to edit.
  3. From the menu to the left select
    MITIGATION SETTINGS
    .
  4. To add a known signature, anomaly, or category exception to your bot profile:
    1. Under the Signature Exception, Anomaly Exception, or Category Exception areas, click
      Add
      .
    2. From the
      Name
       field, select an exception from the list.
    You can type key words to filter the list.
    1. From the
      Mitigation
       field, select a mitigation action for the exception.
      Certain options allow you to adjust the default thresholds once you select a mitigation action.
    2. Click
      Add
      .
      The options provided in each section are based on the system's database of known identifiers of bot attacks. Options (specifically signatures) may change following a system update.
  5. To apply additional mitigation and verification for the Strict Mitigation Enforcement Cases listed below, select
    Enabled
    .
    Dos Attack Mitigation Mode
    Strict bot protection during a detected DoS attack. When enabled, the following mitigation and verification settings are applied:
    • Browser Verification: Verify Before Access
    • Trusted Bots: Alarm
    • Untrusted Bots: Block
    • Suspicious Browsers: Block
    • Malicious Bots: Block
    • Unknown: Block
    The following setting requires that you assign a DoS profile to a virtual server or application.
    API Access for Browsers and Mobile Applications
    Strict protection against bot requests for access to API endpoints or URLs. When enabled, the following mitigation and verification settings are applied:
    • Browser Verification: Verify Before Access
    • Trusted Bots: Block
    • Untrusted Bots: Block
    • Suspiciuos Browsers: Block
    • Malicious Bots: Block
    • Unknown: Block
    A URL is considered an API URL when:
    • Content-Type
       request header matches *json*, *xml*
    • Content-Type
       response header with values *json*, *xml*
    • Request has
      X-Security-Request
       header and the Single Page Application option is enabled.
    • Request is already AJAX-qualified.
  6. Click
    Save
    .
Mitigation exceptions to your template's mitigation settings are now configured to your bot profile.

Edit bot browser verification settings

Once you have configured the general properties and defined the profile template, you can customize the default settings that verify clients requests that they are a browser. This allows you to fine-tune the verification methods of headers and other attributes of the request, in addition to client-side JavaScript challenges.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Select the name of the bot profile you would like to edit.
  3. From the menu to the left, select
    BROWSER VERIFICATION
    .
    Depending on your profile template, the default settings may vary. For more information about the template settings, see
    About bot defense profile templates
    .
  4. To restrict
    Browser Access
     for all browser types (including legitimate browsers) to your applications deselect
    Allowed
    setting.
    When browser access is completely restricted, you must select a mitigation action.
  5. To change the
    Browser Verification
     setting, select an option that specifies if and when the system sends a challenge:
    This action will not be performed if your enforcement mode is set to
    Transparent
    None
    JavaScript and header-based verification is not performed, however some anomaly detection may be performed.
    Challenge-Free Verification
    Only header-based verification is performed.
    Verify Before Access
    The system uses JavaScript challenge to the client. If the client fails the challenge, the anomaly is logged, and the configured mitigation action is performed. If the client passes the challenge, the system forwards the request to the server.
    Verify After Access (Blocking)
    The system injects a JavaScript challenge as a server response. If the client fails the challenge, the anomaly is logged, and the configured mitigation action is performed. If the client passes the challenge, the system forwards the request to the server.
    Verify After Access (Detection only)
    The system injects a JavaScript challenge as a server response. If the client fails the challenge, the anomaly is logged, but performs no mitigation action. If the client passes the challenge, the system forwards the request to the server.
    The
    Verify Before Access
     and
    Verify After Access
    settings prompt a grace period. This prevents full mitigation action following any Bot Defense profile configuration changes.
  6. To change the
    Device ID Mode
     setting, which prompts the system to generate a unique ID for the client device, select one of the following options:
    This action will not be performed if your enforcement mode is set to
    Transparent
    None
    No device ID collection
    Generate After Access
    The JavaScript injection is added to the server response before it is forwarded to the client.
    Generate Before Access
    The system sends a JavaScript challenge to the client before forwarding the request to the server. This guarantees that every request has a Device ID before it reaches the server.
  7. Enable
    Verification and Device-ID Challenges in Transparent Mode
     to perform JavaScript challenges and browser verification tests when the enforcement mode is
    Transparent
    .
  8. Enable
    Single Page Application
     to send JavaScript response challenges for a recently updated application page, without triggering a full page-reload.
  9. Select a
    Cross Domain Requests
     setting to enable a redirect-cookie challenge for non-HTML resources (images, CSS, XML, JavaScript, and Flash) that do not have a valid cookie and have a referer header with a different domain:
    Allow all Requests
    (Default setting) Requests are sent to the server once they pass the system's redirect-cookie challenge.
    Allow configured domains; validate in bulk
    The system fetches the cookies from the domains configured in the
    Related Site Domains
     setting in advance. Requests are then sent to the server if the domain in the referer header matches a domain in
    Related Site Domains
     or
    Related External Domains
    .
    F5 recommends this option if your application has many cross-domain resources.
    Allow configured domains; validate upon request
    The system fetches the cookies from the domains configured in the
    Related Site Domains
     setting in real time, when they are requested. Requests are then sent to the server if the domain in the referer header matches a domain in
    Related Site Domains
     or
    Related External Domains
    .
    F5 recommends this option if your application does not have many cross-domain resource

Edit mobile applications settings

If your Android applications are integrated with F5 Anti-Bot Mobile SDK, you need to ensure that the publisher's SSL certificate is imported to the system.
You can configure special bot protections to test mobile application traffic. These settings are specific to bot traffic management of client requests from an mobile application, which can increase accuracy of protection measures, while reducing instances of false positives. Configuration varies depending on whether your system is licensed with Anti-bot Mobile SDK, or not. If you do not enable mobile settings, these requests are handled as any other request.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Select the name of the bot profile you would like to edit.
  3. From the menu to the left select
    MOBILE APPLICATIONS
    .
  4. For applications with Anti-bot Mobile SDK:
    1. Enable
      Ani-Bot Mobile SDK
       to apply bot protection specific to mobile applications.
      This will provide additional options to fine-tune your configuration.
    2. Apply
      iOS
       protection settings:
      • Allow Any iOS Package
        : Detects authentic mobile application traffic, without verifying which application sent the request. If you do not want to allow any package, you can disable this field, and manually configure specific packages you want to allow.
      • Allow Jailbroken Devices
        : Allow request access from jailbroken devices. This is not recommended, as it allows system access for unchecked applications with spoofed identities.
    3. Apply
      Android
       protection settings:
      • Allow Any Publisher
        : Detects authentic mobile application traffic without verifying which application sent the request. If you do not want to allow any publisher, you can disable this field and select the publisher you want to allow.
      • Allow Rooted Devices
        : Allow request access from rooted devices. This is not recommended, as rooted devices can allow attackers to hijack mobile application sessions.
    4. Enable
      Debugger Enabled Devices
       to allow traffic from a mobile application with an external debugger.
    5. From
      CAPTCHA Substitute for Mobile Applications
      , select
      Human Behavior Challenge
       to allow the SDK to check for human interactions with the screen over the last few seconds.
    6. Enable
      Emulators
       to allow traffic from applications with emulators.
  5. For applications without Anti-bot Mobile SDK, select the name of a signature from the
    Available signatures
     list and use the arrows to move it to the
    Selected Signatures List
    .
    If you need to create a new signature, click the
    Create New Signature
     link to add a new
    Mobile App without SDK
     signature to your bot signatures list. For more information about how to create a new bot signature, see
    Create bot signatures
    .
  6. Click
    Save & Close
Your new mobile application settings are now included in the bot profile and can be deplo

Monitoring bot signature enforcement

The bot signatures list regularly undergo live updates. To prevent false positives, updated signatures are automatically placed in staging. When in staging, requests that match these updated signatures are logged, but not mitigated, regardless of your profile's enforcement settings. Once the staging period is over, the system marks that the signature is ready to be enforced.
It is recommended to manage your bot signature enforcement settings regularly. You can view the status of staged signatures by going to
Configuration
SECURITY
Shared Security
Bot Defense
Bot Profiles
and selecting
SIGNATURE ENFORCEMENT
 from the menu to the left. In the Signature Enforcement area, you can see:
  • Signatures ready to be enforced
    , the number of signatures that have completed staging and are ready to be deployed.
  • Signatures waiting for traffic samples
    , the number of signatures that are currently in staging and are awaiting enough traffic sampling. Once enough traffic is sampled, the signature is ready to be enforced.

Manage bot signature enforcement

You can manually change the current status of all bot signatures. Bot signatures that undergo changes in live updates are moved into staging. This means that requests matching these signatures are staged, but not mitigated.
You can manually set any (or all) signatures in the list into enforcement or staging using the
Enforce
 or
Stage
 buttons. It is not recommended to change the signature status independently of system recommendations.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Select the name of the bot profile you would like to edit.
  3. From the menu to the left select
    SIGNATURE ENFORCEMENT
    .
  4. From the chart, select the
    Ready to Be Enforced
     column to sort the list by signatures that have completed staging.
    This allows you to evaluate details of the signatures are ready to be enforced.
  5. Click the
    Enforce
     button and select
    Enforce All Ready Signatures (n)
    .
    If you do not wish to enforce all ready signatures, you can select signatures from the list and then select
    Enforce Selected Signatures
    .
The system now mitigates traffic matching these enforced bot signatures according to your bot profile's settings.

Manage bot profile allowlist

Manage the URLs, IP addresses, and geolocations that require mitigation actions and/or challenge settings that differ from the profile. Modify your allowlist by creating and deleting items or changing their order of priority.
By default, the system includes 2 predefined items to avoid false negative detection: /favicon.ico and /apple-touch-icon*.png
.
  1. Go to
    Configuration
    SECURITY
    Shared Security
    Bot Defense
    Bot Profiles
    .
  2. Select the name of the bot profile you would like to edit.
  3. From the menu to the left, select
    allowLIST
    .
  4. To create a new allowlist item, click
    Add
    .
    1. Select a
      Source
      . If the source is not
      Any
      , add the item details in the corresponding field.
    2. In the
      Specified URL
       field, add a explicit, or wild type URL.
    3. Enable
      Mitigation Action
       to apply the profile's mitigation against traffic matching the item.
    4. Enable
      Browser Verification and Device ID Challenges
       to apply the profile's challenges against traffic matching the item.
    5. Click
      Save & Close
      .
  5. To reorder the allowlist items, click
    Change Order
     to drag and drop the row.
    allowlist items are applied in the order in which they appear in the list. Changing the order will impact the items' priority.
    When you complete this task, click
    Save & Close
    .
  6. To delete a allowlisted item, select one or more list items, and click
    Delete
    .
Changes to your allowlist are applied to the bot profile, and can be deployed over BIG-IP devices.
Contact Support Contact Support

Have a Question?

Support and Sales >

About F5

Education

F5 Sites

Support Tasks




©2023 F5, Inc. All rights reserved.


Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information