Manual Chapter :
Configuring Bot Defense
Applies To:
Show Versions![Show Versions](/etc/designs/pcx/techdocs/images/expandversions.gif)
BIG-IQ Centralized Management
- 8.3.0, 8.2.0, 8.1.0, 8.0.0
Configuring Bot Defense
About bot defense profile templates
Bot defense profile templates specify
Mitigation Settings
and Browser Verification
default
values. When selecting a bot defense profile template, users should consider security and business requirements
. It is also important to consider the BIG-IP version over which the
policy is deployed. See the notes below for differences among the supported BIG-IP versions.
For more information about device version support, see Unified Bot Defense Version
Compatibility.
Mitigation Settings
Bot Classes
Relaxed | Balanced | Strict | |
---|---|---|---|
Trusted Bot | Alarm | Alarm | Alarm |
Untrusted Bot | Alarm | Alarm | Block |
Suspicious Browser | Alarm | CAPTCHA | Block |
Malicious Bot | Block | Block | Block |
Unknown | None | Rate limit | Block |
Strict Mitigation Enforcement Cases
Relaxed | Balanced | Strict | |
---|---|---|---|
DoS Attack Mitigation Mode | Disabled | Enabled | Enabled |
API Access for Browsers and Mobile
Applications | Disabled | Enabled | Enabled |
Browser Verification and Device ID
Browser Access is enabled (
Allowed
) for all profile templates.
This means that all browser types have access to the
application, so long as the request passes browser verification challenges. Relaxed | Balanced | Strict | |
---|---|---|---|
Browser Verification | Challenge-free verification | Verify after access (blocking) | Verify before access |
Device ID Mode | None | Generate after access | Generate before access |
Bot defense relaxed template
A relaxed bot defense profile defines a permissive security policy that performs basic non-intrusive verification of browsers; strong verification of mobile apps using Anti-Bot Mobile Security SDK; blocks malicious bots and allows all other clients. Malicious bots are detected mostly by using bot signatures. This template provides basic protection with very low risk of false positives.
Bot defense balanced template
A bot defense balanced template defines a moderate security policy that performs advanced verification of browsers; strong verification of mobile apps using Anti-bot Mobile Security SDK; blocks malicious bots; initiates a CAPTCHA challenge for suspicious browsers; limits the total request rate produced by unknown bots and allows trusted and untrusted bots. Malicious bots and suspicious browsers are identified by using both anomaly detection algorithms and bot signatures. This template provides an advanced protection level with reduced latency impact because browser verification is performed by injecting the challenge in the HTTP response.
Bot defense strict template
A strict bot defense profile defines a strict security policy that performs advanced verification of browsers; strong verification of mobile apps using Anti-Bot Mobile Security SDK; and blocks all bots except trusted bots. This template provides the most advanced and strict protection level using all capabilities of bot defense. Browser clients are not allowed access unless they pass proactive verification. Mobile client security access requires the use of the Anti-Bot Mobile SDK.
Create a new bot defense profile
Because this defense mechanism uses reverse lookup, you need to configure a DNS Server (
) and a DNS Resolver ( ) for it to work. The DNS Resolver must use the default route domain in its Route Domain Name field. If you are not sure of the default route domain, you can check it under . The Partition Default field is defined as Yes for the default route domain.This task describes how to configure and save the general properties of a bot defense profile. The profile's mitigation and browser verification settings are based on the selected profile template, however you can later adjust the configuration to better suit your anti-bot needs.
Once you save your general properties settings, the
Profile Name
, Partition
, and Profile Template
settings cannot be edited.- Go to.
- ClickCreate.TheBot Profile Configurationscreen opens on theBOT PROPERTIEStab.
- Enter theProfile Name(required).
- Enter thePartition, if you do not want the default Common partition.
- Enter theDescriptionof the profile.
- Select anEnforcement Modedepending on the readiness of your application environment and system protection requirements:TransparentThe system logs traffic mitigation and verification settings, according to your logging profile settings, but does not provide the following:
- JavaScript-based verification.
- Device ID collection.
- CAPTCHA challenge.
BlockingThe system performs traffic mitigation and verification. Actions are logged according to your logging profile settings. - Select aProfile Templateto determine your MITIGATION and BROWSER VERIFICATION settings.For more information about profile templates, seeAbout bot defense profile templates.
- To enableSignature Staging Upon Update, selectEnabled.By default this field isDisabled, and the system immediately enforces mitigation action, as defined by the mitigation settings. When enabled, the system will automatically put new or modified signatures in staging (either version update or custom signature creation). This means that the system logs and does not block, signature requests regardless of mitigation settings.
- To change the number of days a signature is in staging, select an option from theEnforcement Readiness Periodfield.
- ForRedirect to Poolyou must select one of your server pool redirects.This step is only for bot classes with the mitigation settingRedirect to Pool. If you do not have this mitigation action, or you plan to deploy this profile over a BIG-IP device running version 14.1, you do not need configure this field.
- If you have configured your mitigation settings to includeCAPTCHA,Blocking, orHoneypot Pageyou can selectCustom Responseto customize the HTML response message sent to a suspected bot request.The default response page varies based on the BIG-IP version. The response message provided in the view is for version 15.1. If you wish to have the same response, regardless of BIG-IP version, use the custom response setting. To view the response message clickPreview On.
- ClickSave & Close.
You have now configured the general settings a bot defense profile. The new profile can now be assigned to a virtual server.
You can adjust the profile's extended settings, which include:
- Mitigation settings and exceptions
- Browser verification settings
- Mobile application client protection settings
- Monitor and Mange signature enforcement
- Manage allowlisted items
Add a bot mitigation exception
This task describes how to configure exceptions to a specific bot or bot type. Once you select your exception, you can choose the mitigation option that best protects your application. In addition, you can enable protection from pre-defined enforcement cases. When selecting exceptions to your template's mitigation settings, ensure that you have considered the BIG-IP versions to which this profile is deployed.
- Go to.
- Select the name of the bot profile you would like to edit.
- From the menu to the left selectMITIGATION SETTINGS.
- To add a known signature, anomaly, or category exception to your bot profile:
- Under the Signature Exception, Anomaly Exception, or Category Exception areas, clickAdd.
- From theNamefield, select an exception from the list.
You can type key words to filter the list.- From theMitigationfield, select a mitigation action for the exception.Certain options allow you to adjust the default thresholds once you select a mitigation action.
- ClickAdd.The options provided in each section are based on the system's database of known identifiers of bot attacks. Options (specifically signatures) may change following a system update.
- To apply additional mitigation and verification for the Strict Mitigation Enforcement Cases listed below, selectEnabled.Dos Attack Mitigation ModeStrict bot protection during a detected DoS attack. When enabled, the following mitigation and verification settings are applied:
- Browser Verification: Verify Before Access
- Trusted Bots: Alarm
- Untrusted Bots: Block
- Suspicious Browsers: Block
- Malicious Bots: Block
- Unknown: Block
The following setting requires that you assign a DoS profile to a virtual server or application.API Access for Browsers and Mobile ApplicationsStrict protection against bot requests for access to API endpoints or URLs. When enabled, the following mitigation and verification settings are applied:- Browser Verification: Verify Before Access
- Trusted Bots: Block
- Untrusted Bots: Block
- Suspiciuos Browsers: Block
- Malicious Bots: Block
- Unknown: Block
A URL is considered an API URL when:- Content-Typerequest header matches *json*, *xml*
- Content-Typeresponse header with values *json*, *xml*
- Request hasX-Security-Requestheader and the Single Page Application option is enabled.
- Request is already AJAX-qualified.
- ClickSave.
Mitigation exceptions to your template's mitigation settings are now configured to your bot profile.
Edit bot browser verification settings
Once you have configured the general properties and defined the profile template, you can customize the default settings that verify clients requests that they are a browser. This allows you to fine-tune the verification methods of headers and other attributes of the request, in addition to client-side JavaScript challenges.
- Go to.
- Select the name of the bot profile you would like to edit.
- From the menu to the left, selectBROWSER VERIFICATION.Depending on your profile template, the default settings may vary. For more information about the template settings, seeAbout bot defense profile templates.
- To restrictBrowser Accessfor all browser types (including legitimate browsers) to your applications deselectAllowedsetting.When browser access is completely restricted, you must select a mitigation action.
- To change theBrowser Verificationsetting, select an option that specifies if and when the system sends a challenge:This action will not be performed if your enforcement mode is set toTransparentNoneJavaScript and header-based verification is not performed, however some anomaly detection may be performed.Challenge-Free VerificationOnly header-based verification is performed.Verify Before AccessThe system uses JavaScript challenge to the client. If the client fails the challenge, the anomaly is logged, and the configured mitigation action is performed. If the client passes the challenge, the system forwards the request to the server.Verify After Access (Blocking)The system injects a JavaScript challenge as a server response. If the client fails the challenge, the anomaly is logged, and the configured mitigation action is performed. If the client passes the challenge, the system forwards the request to the server.Verify After Access (Detection only)The system injects a JavaScript challenge as a server response. If the client fails the challenge, the anomaly is logged, but performs no mitigation action. If the client passes the challenge, the system forwards the request to the server.TheVerify Before AccessandVerify After Accesssettings prompt a grace period. This prevents full mitigation action following any Bot Defense profile configuration changes.
- To change theDevice ID Modesetting, which prompts the system to generate a unique ID for the client device, select one of the following options:This action will not be performed if your enforcement mode is set toTransparentNoneNo device ID collectionGenerate After AccessThe JavaScript injection is added to the server response before it is forwarded to the client.Generate Before AccessThe system sends a JavaScript challenge to the client before forwarding the request to the server. This guarantees that every request has a Device ID before it reaches the server.
- EnableVerification and Device-ID Challenges in Transparent Modeto perform JavaScript challenges and browser verification tests when the enforcement mode isTransparent.
- EnableSingle Page Applicationto send JavaScript response challenges for a recently updated application page, without triggering a full page-reload.
- Select aCross Domain Requestssetting to enable a redirect-cookie challenge for non-HTML resources (images, CSS, XML, JavaScript, and Flash) that do not have a valid cookie and have a referer header with a different domain:Allow all Requests(Default setting) Requests are sent to the server once they pass the system's redirect-cookie challenge.Allow configured domains; validate in bulkThe system fetches the cookies from the domains configured in theRelated Site Domainssetting in advance. Requests are then sent to the server if the domain in the referer header matches a domain inRelated Site DomainsorRelated External Domains.F5 recommends this option if your application has many cross-domain resources.Allow configured domains; validate upon requestThe system fetches the cookies from the domains configured in theRelated Site Domainssetting in real time, when they are requested. Requests are then sent to the server if the domain in the referer header matches a domain inRelated Site DomainsorRelated External Domains.F5 recommends this option if your application does not have many cross-domain resource
Edit mobile applications settings
If your Android applications are integrated with F5 Anti-Bot Mobile SDK, you need to ensure that the publisher's SSL certificate is imported to the system.
You can configure special bot protections to test mobile application traffic. These settings are specific to bot traffic management of client requests from an mobile application, which can increase accuracy of protection measures, while reducing instances of false positives. Configuration varies depending on whether your system is licensed with Anti-bot Mobile SDK, or not. If you do not enable mobile settings, these requests are handled as any other request.
- Go to.
- Select the name of the bot profile you would like to edit.
- From the menu to the left selectMOBILE APPLICATIONS.
- For applications with Anti-bot Mobile SDK:
- EnableAni-Bot Mobile SDKto apply bot protection specific to mobile applications.This will provide additional options to fine-tune your configuration.
- ApplyiOSprotection settings:
- Allow Any iOS Package: Detects authentic mobile application traffic, without verifying which application sent the request. If you do not want to allow any package, you can disable this field, and manually configure specific packages you want to allow.
- Allow Jailbroken Devices: Allow request access from jailbroken devices. This is not recommended, as it allows system access for unchecked applications with spoofed identities.
- ApplyAndroidprotection settings:
- Allow Any Publisher: Detects authentic mobile application traffic without verifying which application sent the request. If you do not want to allow any publisher, you can disable this field and select the publisher you want to allow.
- Allow Rooted Devices: Allow request access from rooted devices. This is not recommended, as rooted devices can allow attackers to hijack mobile application sessions.
- EnableDebugger Enabled Devicesto allow traffic from a mobile application with an external debugger.
- FromCAPTCHA Substitute for Mobile Applications, selectHuman Behavior Challengeto allow the SDK to check for human interactions with the screen over the last few seconds.
- EnableEmulatorsto allow traffic from applications with emulators.
- For applications without Anti-bot Mobile SDK, select the name of a signature from theAvailable signatureslist and use the arrows to move it to theSelected Signatures List.If you need to create a new signature, click theCreate New Signaturelink to add a newMobile App without SDKsignature to your bot signatures list. For more information about how to create a new bot signature, seeCreate bot signatures.
- ClickSave & Close
Your new mobile application settings are now included in the bot profile and can be deplo
Monitoring bot signature enforcement
The bot signatures list regularly undergo live updates. To prevent false
positives, updated signatures are automatically placed in staging. When in staging,
requests that match these updated signatures are logged, but not mitigated, regardless
of your profile's enforcement settings. Once the staging period is over, the system
marks that the signature is ready to be enforced.
It is recommended to manage your bot signature enforcement settings
regularly. You can view the status of staged signatures by going to
and selecting SIGNATURE ENFORCEMENT
from the menu
to the left. In the Signature Enforcement area, you can see:- Signatures ready to be enforced, the number of signatures that have completed staging and are ready to be deployed.
- Signatures waiting for traffic samples, the number of signatures that are currently in staging and are awaiting enough traffic sampling. Once enough traffic is sampled, the signature is ready to be enforced.
Manage bot signature enforcement
You can manually change the current status of all bot signatures. Bot signatures that undergo changes in live updates are moved into staging. This means that requests matching these signatures are staged, but not mitigated.
You can manually set any (or all) signatures in the list into enforcement or staging using the
Enforce
or Stage
buttons. It is not recommended to change the signature status independently of system recommendations.- Go to.
- Select the name of the bot profile you would like to edit.
- From the menu to the left selectSIGNATURE ENFORCEMENT.
- From the chart, select theReady to Be Enforcedcolumn to sort the list by signatures that have completed staging.This allows you to evaluate details of the signatures are ready to be enforced.
- Click theEnforcebutton and selectEnforce All Ready Signatures (n).If you do not wish to enforce all ready signatures, you can select signatures from the list and then selectEnforce Selected Signatures.
The system now mitigates traffic matching these enforced bot signatures according to your bot profile's settings.
Manage bot profile allowlist
Manage the URLs, IP addresses, and geolocations that require mitigation actions and/or challenge settings that differ from the profile. Modify your allowlist by creating and deleting items or changing their order of priority.
By default, the system includes 2 predefined items to avoid false negative detection: /favicon.ico and /apple-touch-icon*.png
.
- Go to.
- Select the name of the bot profile you would like to edit.
- From the menu to the left, selectallowLIST.
- To create a new allowlist item, clickAdd.
- Select aSource. If the source is notAny, add the item details in the corresponding field.
- In theSpecified URLfield, add a explicit, or wild type URL.
- EnableMitigation Actionto apply the profile's mitigation against traffic matching the item.
- EnableBrowser Verification and Device ID Challengesto apply the profile's challenges against traffic matching the item.
- ClickSave & Close.
- To reorder the allowlist items, clickChange Orderto drag and drop the row.allowlist items are applied in the order in which they appear in the list. Changing the order will impact the items' priority.When you complete this task, clickSave & Close.
- To delete a allowlisted item, select one or more list items, and clickDelete.
Changes to your allowlist are applied to the bot profile, and can be deployed over BIG-IP devices.