Manual Chapter : Refining Security Policies with Learning

Applies To:

Show Versions Show Versions

BIG-IP ASM

  • 11.5.10, 11.5.9, 11.5.8, 11.5.7, 11.5.6, 11.5.5, 11.5.4, 11.5.3, 11.5.2, 11.5.1
Manual Chapter

About learning

You can use learning resources to help build a security policy, particularly if you are building a security policy manually. When you send client traffic through the Application Security Manager (ASM), the learning data provides information on requests or responses that do not comply with the current security policy and have triggered a violation. The reason for triggering a violation can be either a false positive (typically seen during the process of building a policy), or an actual attack on the site.

ASM generates learning suggestions for requests that cause violations and do not pass the security policy checks. You can examine the requests that cause learning suggestions, and then use the suggestions to refine the security policy. In some cases, learning suggestions may contain recommendations to relax the security policy. When dealing with learning suggestions, make sure to relax the policy only where false positives occurred, and not in cases where a real attack caused a violation.

If you are generating a security policy automatically, ASM handles all learning for you, adjusting the security policy based on traffic characteristics. In that case, the learning screens show only the elements the security policy is in the process of learning.

Learning resources

This table describes the screens in Application Security Manager (ASM) where you can view and handle learning suggestions.

Resource Description
Manual Traffic Learning screen Displays learning suggestions that the system generates. The learning suggestions are categorized by violation type, and can represent actual threats or false-positives. Learning suggestions are for the currently active security policy. When you accept a learning suggestion, you are updating the currently active security policy.
Enforcement Readiness screen Summarizes the security policy entities in staging or with learn explicit entities enabled, that may have learning suggestions, and may be ready to be enforced. For file types, parameters, URLs, cookies, and signatures, you can review the entities, and decide whether to add them to the security policy.
Ignored Entities screen Lists the file types, URLs, and flows that you have instructed the system to disregard, that is, to stop generating learning suggestions for. Typically, the ignored entities are items that you do not want to be a part of the security policy.
IP Address Exceptions screen Lists IP address exceptions with specific characteristics that you can configure. You can instruct the system not to generate learning suggestions for traffic sent from any of these IP addresses.
View Full Request Information screen Displays any violations and details associated with a request. You can review this information, and then if you want to accept the learning suggestion, click the Learn button to update the active security policy. To display the View Full Request Information screen, from the Event Logs > Application > Requests screen, click a Requested URL in the Requests List.

About learning suggestions

Application Security Manager (ASM) generates learning suggestions for violations if the Learn flag is enabled for the violations on the Blocking Settings screen. When the system receives a request that triggers a violation, the system updates the Manual Traffic Learning screen with learning suggestions based on the violating request information. From this screen, you can review the learning suggestions to determine whether the request triggered a legitimate security policy violation, or if the violation represents a need to update the security policy.

Making decisions about which learning suggestions to use requires some general understanding of application security, and specific knowledge of the protected application (for example, recognizing valid traffic). Often, you should consider accepting a learning suggestion when you see that it has occurred multiple times, from many different source IP addresses. Repeated learning suggestions typically indicate valid traffic behavior that warrants relaxing the security policy.

The Manual Traffic Learning screen also displays violations for which the system does not generate learning suggestions. Typically, these violations are related to RFC compliance and system resources; the resolution for these violations may be to disable the violation or sub-violation rather than to perform any specific configuration. The system displays these violations along with the learning suggestions to ease the security policy management tasks.

Fine-tuning a security policy

After you create a security policy, the system provides learning suggestions concerning additions to the security policy based on the traffic that is accessing the application. For example, you can have users or testers browse the web application. By analyzing the traffic to and from the application, Application Security Manager generates learning suggestions or ways to fine-tune the security policy to better suit the traffic and secure the application.

Note: If you are using the Policy Builder to add elements to the security policy, you can skip this task.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning. The Manual Traffic Learning screen opens, and lists violations and learning suggestions that the system has made based on real traffic.
  2. In the Traffic Learning area, click each violation hyperlink, then review and handle learning suggestions:
    Option Description
    Accept Select a learning suggestion, click Accept, and then click Apply Policy. The system updates the security policy to allow the file type, URL, parameter, or other element.
    Clear Select a learning suggestion, and click Clear. The system removes the learning suggestion and continues to generate suggestions for that violation.
    Cancel Click Cancel to return to the Manual Traffic Learning screen.
    By default, a security policy is put into a staging-tightening period for seven days. During this time, you can examine learning suggestions and adjust the security policy without blocking traffic.
  3. On the Manual Traffic Learning screen, review the violations and consider whether you want to permit any of them (for example, if a violation is causing false positives). Select any violations you do not want the system to trigger, and click Disable Violation. A popup screen opens, and you can verify that you want to disable the violations or cancel the action.
  4. To put the security policy changes into effect immediately, click Apply Policy.
  5. On the Main tab, click Security > Overview > Application > Action Items. The Action Items screen opens.
  6. Examine the Action Items screen for information about recommended actions that you need to complete.
    1. Review the Suggested Action Items area, which lists system tasks and security policy tasks that should be completed.
    2. Click the links in the Suggested Action Items area to go to the screen where you can perform the recommended action.
    3. In the Quick Links area, click any of the links to gain access to common configuration and reporting screens.
The security policy now includes elements unique to your web application.
It is a good idea to periodically review the learning suggestions on the Manual Traffic Learning screen to determine whether the violations are legitimate, or if they are false positives that indicate a need to update the security policy.

Configuring explicit entities learning

You can adjust the explicit entities learning settings for file types, URLs, parameters, cookies, and redirection domains. Explicit learning settings specify when Real Traffic Policy Builder adds, or suggests you add, explicit entities to the security policy.

  1. On the Main tab, click Security > Application Security > Policy Building > Settings. The Settings screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the General Policy Building Settings area, for Explicit Entities Learning, for each type of entity (File Types, URLs, Parameters, Cookies, and Redirection Domains), select the option that determines which Learning suggestions are provided by the system (based on real traffic).
    Option Description
    Never (wildcard only) Specifies that when false positives occur, the system suggests relaxing the settings of the wildcard. This option results in a security policy that is easy to manage, but is not as strict. If Policy Builder is running, it does not add explicit entities that match a wildcard to the security policy. The wildcard entity remains in the security policy. The Policy Builder changes the attributes of any matched wildcard. If not running, Policy Builder suggests changing the attributes of matched wildcard entities, but does not suggest you add explicit entities that match the wildcard entity.
    Selective Applies only to * wildcard entity. When false positives occur, adds an explicit entity with relaxed settings. This option serves as a good balance between security, policy size, and ease of maintenance. If Policy Builder is running, it adds explicit entities that do not match the attributes of the * wildcard, and does not remove the * wildcard. If Policy Builder is not running, the system suggests adding explicit entities that match the * wildcard. (Option not applicable to Redirection Domains.)
    Add All Entities Creates a comprehensive whitelist policy that includes all web site entities. This option results in a large, more granular configuration with stricter security. If Policy Builder is running, it adds explicit entities that match a wildcard to the security policy. When the security policy is stable, the * wildcard is removed. If Policy Builder is not running, the system suggests adding explicit entities that match the wildcard.
    Changing the explicit entities learning settings may change the Policy Type to Custom.
  4. Click Save to save your settings.
  5. To put the security policy changes into effect immediately, click Apply Policy.

The security policy now learns new file types, parameters, URLs, cookies, and redirection domains according to the explicit learning settings you specified.

Viewing requests that caused learning suggestions

To review requests related to learning suggestions, you need to have a security policy that is already handling traffic that is causing violations. If no violations have occurred, you will not see any learning suggestions.
Before you process a learning suggestion, it is very helpful to examine the details of the request that caused the learning suggestion. By viewing the request, you can determine whether the violation was caused by an attack or if it is a false positive.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning. The Manual Traffic Learning screen opens, and lists violations and learning suggestions that the system has made based on real traffic.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, click a violation hyperlink to view either the Requests List, or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
  4. In the Occurrences column, click the number. The Requests List popup screen opens, and displays all of the requests that triggered the learning suggestion. Close the popup when you are done.
  5. In the Recent Incidents column (if attack signatures were detected), click the number. The Requests List popup screen displays the requests that contained an item that triggered the learning suggestion.
  6. In the Requests List area of the popup screen, in the URL column, click a URL link. The View Full Request Information screen or View Request Information opens in the popup screen, where you can review the request that triggered the learning suggestion
  7. For each violation with a Learn button, click Learn to go back to the violation learning screen where you can accept or clear the learning suggestions for the security policy one value at a time.
  8. To view the actual contents of the request, click Full Request (on the View Request Information screen) or HTTP Request (on the View Full Request Information screen). and when you are done looking at the request details, click Close.
  9. On the screen showing learning suggestions for the violation, to accept the suggestion and change the security policy, click Accept.
  10. To remove learning suggestions without changing the security policy, select the ones to remove, and then click the Clear button.
  11. On the Manual Traffic Learning screen, continue to review the violations and associated learning suggestions.
When you accept a learning suggestion, the system updates the current edited security policy to accept the request entity that triggered the violation. When you clear a learning suggestion, the system deletes the learning suggestion, and does not update the security policy; the system continues to generate learning suggestions for future instances of the violation.

Accepting learning suggestions

If you have reviewed a learning suggestion and want to make the suggested change to the security policy, you can accept the suggestion.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning. The Manual Traffic Learning screen opens, and lists violations and learning suggestions that the system has made based on real traffic.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, click a violation hyperlink. The screens vary for different violations. The learning suggestions properties screen opens.
  4. Select one or more learning suggestions, and then click the Accept, Apply, or Allow button, depending on the violation. The system updates the security policy, applies the learning suggestions, and opens the Requests List popup screen.

Clearing learning suggestions

If you want to ignore a learning suggestion and remove it from the screen, you can clear it, or you can clear all learning suggestions for a violation.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning. The Manual Traffic Learning screen opens, and lists violations and learning suggestions that the system has made based on real traffic.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. To clear all learning suggestions for a violation:
    1. Select one or more violations, and then click Clear.
    2. Click OK.
    The system deletes all of the learning suggestions and removes the violation from the list without changing the security policy.
  4. To clear specific learning suggestions for a violation:
    1. Click a violation hyperlink.
    2. Select one or more learning suggestions, and then click Clear.
    3. For URLs, file types, or flows, if you want to stop generating learning suggestions, select the Move to ignored entities check box
    4. Click OK.
    The system deletes the learning suggestion without changing the security policy.
Although the learning suggestions are cleared, the system continues to generate learning suggestions for future instances of the violation unless you added the entity to the Ignored Entities list. When the system receives subsequent requests for those items on the Ignored Entities list, the system no longer generates learning suggestions for them. The system does continue to log the requests.

Viewing ignored entities

You can view file types, URL, or flows that are on the Ignored Entities list and for which the system is not generating learning suggestions. You can also delete items from the list.
  1. On the Main tab, click Security > Application Security > Policy Building > Ignored Entities. The Ignored Entities screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. On the Ignored Entities screen, if ignored entities exist for an entity type, that type becomes a link; click one of the links to view a list of all entities logged within that category. The Ignored File Types screen, Ignored URLs screen, or Ignored Flows screen opens.
  4. If you want to remove an entity from the list, select it, then click Delete, and click OK to confirm. The system removes the selected item from the Ignored Entities list.

About enforcement readiness

When you are creating a security policy, you specify an enforcement readiness period that indicates a staging period for entities and attack signatures (typically 7 days). When entities or attack signatures are in staging, the system does not enforce them. Instead, the system posts learning suggestions for staged entities in the Violations Found for Staged Entities table in the request details.

When the enforcement readiness period is over and no learning suggestions are added for the staging period duration (the default is 7 days), the file type, URL, parameter, cookie, signature, or redirection domain is considered ready to be enforced. You can delve into the details to see if you want to enforce these entities in the security policy. From the Enforcement Readiness summary, you can add selected entities to the security policy, or you can enforce all of the entities and signatures that are ready to be enforced.

Enforcing entities

After you create a security policy and traffic is sent to the web application, new entities are added by means of learning explicit entities, and existing entities are modified through staging. You can review the entities and signatures that are in staging or that are ready to be enforced, and add them to the security policy.
  1. On the Main tab, click Security > Application Security > Policy Building > Enforcement Readiness. The Enforcement Readiness summary screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. To enforce all entities that are ready to be enforced, click Enforce Ready. If you select this option, you are done. Continue only if you want to enforce selected entities or signatures.
  4. In the Enforcement Readiness Summary, check to see if a number appears in the Not Enforced column. A number greater than zero indicates that entities of that type are in staging or with learn explicit entities enabled.
  5. Click the number in the Not Enforced column. The allowed file types, URLs, parameters, cookies, signatures or redirection protection list opens showing the entities that you can enforce.
  6. Select the entities you want the security policy to enforce, and click Enforce.
The system removes the selected entities or signatures from staging. If any of the entities are wildcards that are learning explicit entities, the wildcards are deleted.

Disabling learning on violations

F5 recommends that you review the violations that occur, and consider whether they represent legitimate violations or false-positives. You can disable learning on violations that are not applicable to your web application.
Note: Be sure that you understand the ramifications of disabling a violation before doing it.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning. The Manual Traffic Learning screen opens, and lists violations and learning suggestions that the system has made based on real traffic.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, select the box next to the violation name that you want to disable.
  4. Click the Disable Violation button, and click OK to confirm. The screen refreshes, and you no longer see the violation in the Traffic Learning area.
  5. To put the security policy changes into effect immediately, click Apply Policy.
Disabling a violation turns off the blocking policy so that you are no longer notified of requests that trigger the violation. The system then ignores future instances of the violation, and passes the requests on to the web application resources. Alternately, you can clear the learning suggestions, and the system continues to issue learning suggestions for the requests.