Applies To:
Show Versions
BIG-IP APM
- 12.1.6, 12.1.5, 12.1.4, 12.1.3, 12.1.2, 12.1.1, 12.1.0
URL Categorization
About URL categorization
On a BIG-IP® system with an SWG subscription, URL categorization must be configured. The URL database must be downloaded and a download schedule must be set. Optionally, custom URL categories and filters can be created to extend the standard URL categories and URL filters that are provided.
On a BIG-IP system without an SWG subscription, URL categorization is an option. Standard URL categories and URL filters are not provided. URL filtering can be accomplished with user-defined URL categories and user-defined URL filters.
Overview: Downloading the URL database and updating standard URL filters
Secure Web Gateway (SWG) supplies over 150 URL categories and identifies over 60 million URLs that fit within these categories. In addition, you can create custom categories if needed and add URLs to any category, custom or otherwise. You can also use custom categories to define blacklists and whitelists.
SWG supplies default URL filters as a starting point for your configuration. For example, the URL filter named default blocks the majority of inappropriate web sites. You can use any default filter as a starting point from which to define your own URL filters to reflect your acceptable use policies.
Complete these tasks before you create a per-request policy to categorize and filter URL requests.
Task summary
Use these tasks to download URL categories initially, to refresh them over time, and to specify URL filters that support your use and compliance policy. Before you begin, the BIG-IP® system must be licensed and provisioned to support URL categorization.
Task list
About the Instant Messaging URL category
Secure Web Gateway (SWG) supports HTTP and HTTPS-based instant messaging protocols. As a result, when you use the Instant Messaging URL category to block messages, SWG can block messages to ICQ, for example, but cannot block messages from applications that use non-standard ports or tunneling over HTTP, such as, Yahoo Messenger, Skype, Google Talk, and so on.
Similarly, SWG cannot block messages from file-sharing and peer-to-peer protocols that do not use HTTP or HTTPS; most of these protocol types do not use either HTTP or HTTPS.
Downloading and updating URL categories
Adding custom URL categories to the URL database
Customizing standard categories from the URL database
Configuring URL filters
Looking up a URL category in the master database
Implementation result
Now you have BIG-IP® Secure Web Gateway (SWG) configured to regularly download updates to URL categories. URL filters are configured and ready to be added to per-request policies.
Configuring logging for the URL database
Viewing a URL database report
Secure Web Gateway database download log messages
When you deploy Secure Web Gateway (SWG), the database downloads output messages to the log destinations specified in the default-log-setting. This table lists messages that are available only when you enable debug.
Debug message | Description |
---|---|
Transfer Status 247 | The file is transferred successfully to the BIG-IP® system. If you see a Transfer Status other than 247, it might indicate an error. |
RTU Type | The RTU Type is always 1. If you see an RTU Type other than 1, it might indicate an error. |
Expiration Date | The BIG-IP system does not use the expiration date in this message. Instead, the BIG-IP system enforces the SWG license and the database download works accordingly. |
Overview: Configuring user-defined URL categories and filters
If you want to categorize and filter URL requests from your users, you need to use URL categories and URL filters. If URL categories and filters do not exist on a BIG-IP® system, you can create them.
Complete these tasks before you create a per-request policy that includes items to categorize (URL Category) and filter (URL Filter Assign) URL requests.