F5 Logo MyF5
Search
  1. MyF5 Home
  2. Knowledge Centers
  3. BIG-IP ASM
  4. BIG-IP Application Security Manager: Implementations
  5. Refining Security Policies with Learning
Manual Chapter : Refining Security Policies with Learning

Applies To:

Show Versions Show Versions

BIG-IP ASM

  • 11.6.5, 11.6.4, 11.6.3, 11.6.2, 11.6.1
Manual Chapter
Table of Contents | << Previous Chapter | Next Chapter >>

Refining Security Policies with Learning

About learning

You can use learning resources to help build a security policy, particularly if you are building a security policy manually. When you send client traffic through the Application Security Manager™ (ASM), the learning data provides information on requests or responses that do not comply with the current security policy and have triggered a violation. The reason for triggering a violation can be either a false positive (typically seen during the process of building a policy), or an actual attack on the site.

ASM™ generates learning suggestions for requests that cause violations and do not pass the security policy checks. You can examine the requests that cause learning suggestions, and then use the suggestions to refine the security policy. In some cases, learning suggestions may contain recommendations to relax the security policy. When dealing with learning suggestions, make sure to relax the policy only where false positives occurred, and not in cases where a real attack caused a violation. You can use the violation ratings to help determine how likely a request was caused by an attack.

If you are generating a security policy automatically, ASM handles all learning for you, adjusting the security policy based on traffic characteristics. In that case, the learning screens show only the elements the security policy is in the process of learning.

Learning resources

This table describes the screens in Application Security Manager™ (ASM) where you can view and handle learning suggestions.

Resource Description
Manual Traffic Learning screen Displays learning suggestions that the system generates. The learning suggestions are categorized by violation type, and can represent actual threats or false-positives. Learning suggestions are for the currently active security policy. When you accept a learning suggestion, you are updating the currently active security policy.
Enforcement Readiness screen Summarizes the security policy entities in staging or with learn explicit entities enabled, that may have learning suggestions, and may be ready to be enforced. For file types, parameters, URLs, cookies, and signatures, you can review the entities, and decide whether to add them to the security policy.
Ignored Entities screen Lists the file types, URLs, and flows that you have instructed the system to disregard, that is, to stop generating learning suggestions for. Typically, the ignored entities are items that you do not want to be a part of the security policy.
IP Address Exceptions screen Lists IP address exceptions with specific characteristics that you can configure. You can instruct the system not to generate learning suggestions for traffic sent from any of these IP addresses.
View Full Request Information screen Displays any violations, the violation rating, and details associated with a request. You can review this information, and then if you want to accept the learning suggestion, click the Learn button to update the active security policy. To display the View Full Request Information screen, from the Event Logs > Application > Requests screen, click a Requested URL in the Requests List.

About learning suggestions

Application Security Manager™ (ASM) generates learning suggestions for violations if the Learn flag is enabled for the violations on the Blocking Settings screen. When the system receives a request that triggers a violation, the system updates the Manual Traffic Learning screen with learning suggestions using information from the violating request. From this screen, you can review the learning suggestions to determine whether the request triggered a legitimate security policy violation, or if the violation represents a need to update the security policy.

Making decisions about which learning suggestions to use requires a general understanding of application security, and specific knowledge of the protected application (for example, recognizing valid traffic). Often, you should consider accepting a learning suggestion when you see that it has occurred multiple times, from many different source IP addresses. Repeated learning suggestions typically indicate valid traffic behavior that warrants relaxing the security policy.

You can also drill down into a request to review the violation rating. Learning suggestions associated with requests having a low violation rating are more likely to be false positives and can be accepted. But if a request has a high violation rating, the learning suggestion should not be accepted. It should be cleared because it is most likely indicative of an attack.

The Manual Traffic Learning screen also displays violations for which the system does not generate learning suggestions. Typically, these violations are related to RFC compliance and system resources; the resolution for these violations may be to disable the violation rather than to change the configuration. The system displays these violations along with the learning suggestions to ease the security policy management tasks.

What requests are unlearnable?

Some violations that occur indicate a real problem with a request that cannot be learned. These are called unlearnable requests. For example, the system considers requests that trigger the following HTTP protocol compliance violations to be unlearnable:

  • Bad HTTP version
  • Unparsable request content
  • Null in request

They are considered unlearnable because these violations indicate behavior that is never acceptable, so the security policy will never be changed to allow them. Consequently, the violating requests are not used for automatic or manual learning (even if they include additional violations that could be learned). Also, the violation rating for these transactions is always set to 5 (the highest severity).

Fine-tuning a security policy

After you create a security policy, the system provides learning suggestions concerning additions to the security policy based on the traffic that is accessing the application. For example, you can have users or testers browse the web application. By analyzing the traffic to and from the application, Application Security Manager™ generates learning suggestions or ways to fine-tune the security policy to better suit the traffic and secure the application.

Note: If you are using the Policy Builder to add elements to the security policy, you can skip this task. This option is primarily for building a security policy manually.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning .
    The Manual Traffic Learning screen opens, and lists violations that the system has detected.
  2. In the Traffic Learning area, click each violation hyperlink, then review and handle learning suggestions:
    Option Description
    Accept Select a learning suggestion, click Accept, and then click Apply Policy. The system updates the security policy to allow the file type, URL, parameter, or other element.
    Clear Select a learning suggestion, and click Clear. The system removes the learning suggestion and continues to generate suggestions for that violation.
    Cancel Click Cancel to return to the Manual Traffic Learning screen.
    By default, a security policy is put into a staging-tightening period for seven days. During this time, you can examine learning suggestions and adjust the security policy without blocking traffic.
  3. To find out more about a violation and its occurrences, when you click a violation hyperlink and see what caused the violation, click the number in the Occurrences column.
    The Requests List popup screen opens, and you can see the requests that caused the violation including a violation rating of the request. (Ratings are from 1 to 5, where is the most severe.)
  4. To decide whether the request is an attack or a false positive, look at the violation rating.
    1. Click Violation Rating on the Request List screen.
    2. Look at the bar chart that displays the violation rating and number of occurrences.
    3. If the violation rating is 1 or 2, it is most likely a false positive and you can close the Requests List, select the violation, and click Accept.
      This accepts the learning suggestion to the security policy. What this means depends on the violation. It could be to allow a parameter or URL that looks suspicious but is allowed on your web site, it might mean to unselect certain security failures, or it might mean to disable an attack signature.
    4. If the violation rating is 4 or 5, it is most likely an attack and you can close the Requests List, select the violation, and click Clear.
      You probably do not want to change the policy to accept a suggestion that would allow an attack, so you would clear the suggestion without changing the policy.
    5. If the violation rating is 3, the request needs further investigation. You can go back to the Requests List and click the request to examine it more closely.
  5. On the Manual Traffic Learning screen, review the violations and consider whether you want to permit any of them (for example, if a violation is causing false positives). Select any violations you do not want the system to trigger, and click Disable Violation.
    A popup screen opens, and you can verify that you want to disable the violations or cancel the action.
  6. To put the security policy changes into effect immediately, click Apply Policy.
  7. On the Main tab, click Security > Overview > Application > Action Items .
    The Action Items screen opens.
  8. Examine the Action Items screen for information about recommended actions that you need to complete.
    1. Review the Suggested Action Items area, which lists system tasks and security policy tasks that should be completed.
    2. Click the links in the Suggested Action Items area to go to the screen where you can perform the recommended action.
    3. In the Quick Links area, click any of the links to gain access to common configuration and reporting screens.
The security policy now includes elements unique to your web application.
It is a good idea to periodically review the learning suggestions on the Manual Traffic Learning screen to determine whether the violations are legitimate, or if they are false positives that indicate a need to update the security policy.

Configuring explicit entities learning

You can adjust the explicit entities learning settings for file types, URLs, parameters, cookies, and redirection domains. Explicit learning settings specify when Real Traffic Policy Builder® adds, or suggests you add, explicit entities to the security policy.

  1. On the Main tab, click Security > Application Security > Policy Building > Settings .
    The Settings screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the General Policy Building Settings area, for Explicit Entities Learning, for each type of entity (File Types, URLs, Parameters, Cookies, and Redirection Domains), select the option that determines which Learning suggestions are provided by the system (based on real traffic).
    Option Description
    Never (wildcard only) Specifies that when false positives occur, the system suggests relaxing the settings of the wildcard. This option results in a security policy that is easy to manage, but is not as strict. If Policy Builder is running, it does not add explicit entities that match a wildcard to the security policy. The wildcard entity remains in the security policy. The Policy Builder changes the attributes of any matched wildcard. If not running, Policy Builder suggests changing the attributes of matched wildcard entities, but does not suggest you add explicit entities that match the wildcard entity.
    Selective Applies only to * wildcard entity. When false positives occur, adds an explicit entity with relaxed settings. This option serves as a good balance between security, policy size, and ease of maintenance. If Policy Builder is running, it adds explicit entities that do not match the attributes of the * wildcard, and does not remove the * wildcard. If Policy Builder is not running, the system suggests adding explicit entities that match the * wildcard. (Option not applicable to Redirection Domains.)
    Add All Entities Creates a comprehensive whitelist policy that includes all web site entities. This option results in a large, more granular configuration with stricter security. If Policy Builder is running, it adds explicit entities that match a wildcard to the security policy. When the security policy is stable, the * wildcard is removed. If Policy Builder is not running, the system suggests adding explicit entities that match the wildcard.
    Changing the explicit entities learning settings may change the Policy Type to Custom.
  4. Click Save to save your settings.
  5. To put the security policy changes into effect immediately, click Apply Policy.

The security policy now learns new file types, parameters, URLs, cookies, and redirection domains according to the explicit learning settings you specified.

Viewing requests that caused learning suggestions

To review requests related to learning suggestions, you need to have a security policy that is already handling traffic that is causing violations. If no violations have occurred, you will not see any learning suggestions.
Before you process a learning suggestion, it is very helpful to examine the details of the request that caused the learning suggestion. By viewing the request, you can determine whether the violation was caused by an attack or if it is a false positive.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning .
    The Manual Traffic Learning screen opens, and lists violations that the system has detected.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, click a violation hyperlink to view either the Requests List, or the specific elements in the request that triggered the security policy violation and the corresponding learning suggestion.
  4. In the Occurrences column, click the number.
    The Requests List popup screen opens, and displays all of the requests that triggered the learning suggestion. Close the popup when you are done.
  5. In the Recent Incidents column (if attack signatures were detected), click the number.
    The Requests List popup screen displays the requests that contained an item that triggered the learning suggestion.
  6. In the Requests List area of the popup screen, in the URL column, click a URL link.
    The View Full Request Information screen or View Request Information opens in the popup screen, where you can review the request that triggered the learning suggestion
  7. For each violation with a Learn button, click Learn to go back to the violation learning screen where you can accept or clear the learning suggestions for the security policy one value at a time.
  8. To view the actual contents of the request, click Full Request (on the View Request Information screen) or HTTP Request (on the View Full Request Information screen). and when you are done looking at the request details, click Close.
  9. On the screen showing learning suggestions for the violation, to accept the suggestion and change the security policy, click Accept.
  10. To remove learning suggestions without changing the security policy, select the ones to remove, and then click the Clear button.
  11. On the Manual Traffic Learning screen, continue to review the violations and associated learning suggestions.
When you accept a learning suggestion, the system updates the current edited security policy to accept the request entity that triggered the violation. When you clear a learning suggestion, the system deletes the learning suggestion, and does not update the security policy; the system continues to generate learning suggestions for future instances of the violation.

Accepting learning suggestions

If you have reviewed a learning suggestion and want to make the suggested change to the security policy, you can accept the suggestion.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning .
    The Manual Traffic Learning screen opens, and lists violations that the system has detected.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, click a violation hyperlink.
    The screens vary for different violations.
    The learning suggestions properties screen opens.
  4. Select one or more learning suggestions, and then click the Accept, Apply, or Allow button, depending on the violation.
    The system updates the security policy, applies the learning suggestions, and opens the Requests List popup screen.

Clearing learning suggestions

If you want to ignore a learning suggestion and remove it from the screen, you can clear it, or you can clear all learning suggestions for a violation.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning .
    The Manual Traffic Learning screen opens, and lists violations that the system has detected.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. To clear all learning suggestions for a violation:
    1. Select one or more violations, and then click Clear.
    2. Click OK.
    The system deletes all of the learning suggestions and removes the violation from the list without changing the security policy.
  4. To clear specific learning suggestions for a violation:
    1. Click a violation hyperlink.
    2. Select one or more learning suggestions, and then click Clear.
    3. For URLs, file types, or flows, if you want to stop generating learning suggestions, select the Move to ignored entities check box
    4. Click OK.
    The system deletes the learning suggestion without changing the security policy.
Although the learning suggestions are cleared, the system continues to generate learning suggestions for future instances of the violation unless you added the entity to the Ignored Entities list. When the system receives subsequent requests for those items on the Ignored Entities list, the system no longer generates learning suggestions for them. The system does continue to log the requests.

Viewing ignored entities

You can view file types, URL, or flows that are on the Ignored Entities list and for which the system is not generating learning suggestions. You can also delete items from the list.
  1. On the Main tab, click Security > Application Security > Policy Building > Ignored Entities .
    The Ignored Entities screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. On the Ignored Entities screen, if ignored entities exist for an entity type, that type becomes a link; click one of the links to view a list of all entities logged within that category.
    The Ignored File Types screen, Ignored URLs screen, or Ignored Flows screen opens.
  4. If you want to remove an entity from the list, select it, then click Delete, and click OK to confirm.
    The system removes the selected item from the Ignored Entities list.

About enforcement readiness

When you are creating a security policy, you specify an enforcement readiness period that indicates a staging period for entities and attack signatures (typically 7 days). When entities or attack signatures are in staging, the system does not enforce them. Instead, the system posts learning suggestions for staged entities in the Violations Found for Staged Entities table in the request details.

When the enforcement readiness period is over and no learning suggestions are added for the staging period duration (the default is 7 days), the file type, URL, parameter, cookie, signature, or redirection domain is considered ready to be enforced. You can delve into the details to see if you want to enforce these entities in the security policy. From the Enforcement Readiness summary, you can add selected entities to the security policy, or you can enforce all of the entities and signatures that are ready to be enforced.

Enforcing entities

After you create a security policy and traffic is sent to the web application, new entities are added by means of learning explicit entities, and existing entities are modified through staging. You can review the entities and signatures that are in staging or that are ready to be enforced, and add them to the security policy.
  1. On the Main tab, click Security > Application Security > Policy Building > Enforcement Readiness .
    The Enforcement Readiness summary screen opens.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. To enforce all entities that are ready to be enforced, click Enforce Ready.
    If you select this option, you are done. Continue only if you want to enforce selected entities or signatures.
  4. In the Enforcement Readiness Summary, check to see if a number appears in the Not Enforced column.
    A number greater than zero indicates that entities of that type are in staging or with learn explicit entities enabled.
  5. Click the number in the Not Enforced column.
    The allowed file types, URLs, parameters, cookies, signatures or redirection protection list opens showing the entities that you can enforce.
  6. Select the entities you want the security policy to enforce, and click Enforce.
The system removes the selected entities or signatures from staging. If any of the entities are wildcards that are learning explicit entities, the wildcards are deleted.

Disabling learning on violations

F5 recommends that you review the violations that occur, and consider whether they represent legitimate violations or false-positives. You can disable learning on violations that are not applicable to your web application.
Note: Be sure that you understand the ramifications of disabling a violation before doing it.
  1. On the Main tab, click Security > Application Security > Policy Building > Manual Traffic Learning .
    The Manual Traffic Learning screen opens, and lists violations that the system has detected.
  2. In the Current edited policy list near the top of the screen, verify that the edited security policy is the one you want to work on.
  3. In the Traffic Learning area, select the box next to the violation name that you want to disable.
  4. Click the Disable Violation button, and click OK to confirm.
    The screen refreshes, and you no longer see the violation in the Traffic Learning area.
  5. To put the security policy changes into effect immediately, click Apply Policy.
Disabling a violation turns off the blocking policy so that you are no longer notified of requests that trigger the violation. The system then ignores future instances of the violation, and passes the requests on to the web application resources. Alternately, you can clear the learning suggestions, and the system continues to issue learning suggestions for the requests.
Table of Contents | << Previous Chapter | Next Chapter >>
Contact Support Contact Support

Have a Question?

Support and Sales >

About F5

Education

F5 Sites

Support Tasks




©2023 F5 Networks, Inc. All rights reserved.


Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information