F5 Logo MyF5
Search
  1. MyF5 Home
  2. BIG-IP Access Policy Manager: Secure Web Gateway
  3. Configuring the URL Database for SWG
Manual Chapter : Configuring the URL Database for SWG

Applies To:

Show Versions Show Versions

BIG-IP APM

  • 16.0.1, 16.0.0, 15.1.0
Manual Chapter

Configuring the URL Database for SWG

About initial configuration steps for SWG

On a BIG-IP system with an SWG subscription, the first thing you must do is download the URL database. After that, if you want to use transparent user identification, you should install one of the Secure Web Gateway user identification agents: F5 DC Agent or F5 Logon Agent.

Overview: Downloading and updating the URL database for SWG

The URL database is available only on a BIG-IP-APM system with an SWG subscription.
On a system where a URL database download is available, you must complete the download before you start to configure per-request policies to categorize and filter URLs. You can download the URL database to the BIG-IP system or to an upstream proxy. The system must have Internet access.
For SWG to best protect your network from new threats, schedule regular database downloads to update the existing URL categories with new URLs. Without these updates, SWG uses obsolete security intelligence and as a result, protection of your networks is less effective.
SWG is made up of the following databases:
  • Master database
  • Real-time Security Database
  • Advanced Classification Engine (ACE) database
The databases are updated through the
download.websense.com
 website.
Through Forcepoint's Threat Seeker Intelligence Cloud, the system receives the following updates:
  • Master Database updates (MasterDB) - This database is updated by Forcepoint once a day and is required to be downloaded once per day. The MasterDB can be downloaded based on a configurable schedule
  • Real-time database updates (RTU) - polled every 10 minutes
  • Real-Time Security Updates (RTSU) - polled every 10 minutes ACE database updates - polled every 15 minutes
During the URL database update, the system goes to
download.websense.com/cgi-bin/
 on port 80 and downloads
nph-wsget20.exe
. The
nph-wsget20.exe
 file provides access to the Forcepoint website from which the databases are downloaded.

Configuring an upstream proxy for the BIG-IP system

If your network practices do not permit you to download data from the Internet to the BIG-IP system, configure an upstream proxy to use for this type of access instead.
You can configure only one upstream proxy for the BIG-IP system.
  1. On the Main tab, select
    System
    Configuration
    Device
    Upstream Proxy
    .
  2. In the
    Name
     field, type a name for the proxy server.
  3. In the
    IP Address
     field, type the IP address for the proxy server.
  4. In the
    Port
     field, type the port number for the proxy server.
  5. In the
    User Name
     and
    Password
     fields, type credentials for an account on the proxy server, if needed.
  6. Click
    Save
    .
The upstream proxy is configured.
You can update the IP address, port, and credentials for the upstream proxy if needed. To change the name, you must delete the configuration and create it again.

Downloading the URL database

Database download is required and available only on a BIG-IPsystem with an SWG subscription.
To download the database to the BIG-IP system, before you start you must have configured:
  • DNS for the BIG-IP device in the System area of the product.
  • A default route in the Network area of the product.
To download the database to a proxy for the BIG-IP system, before you start you must have configured an upstream proxy in the
System
 area of the product.
Download the URL database to supply URLs and URL categories.
Schedule database downloads to occur during off-peak hours (very little to no user activity), so that users are not impacted. Alternatively, you can initiate database downloads on-demand.
  1. On the Main tab, click
    Access Policy
    Secure Web Gateway
    Database Settings
    Database Download
    .
  2. In the Download Settings area from the
    Downloads
     list, select
    Enabled
    .
    Additional settings display.
    Download Schedule
     displays a default schedule for the download.
  3. To download the database to an upstream proxy, select the
    Use Proxy
     check box.
  4. In the
    Download Schedule
     settings, configure a two-hour period in which to start the download.
    Schedule the download to occur during off-peak hours. The default schedule is between one and three A.M.
    After the download completes, database indexing occurs. It consumes a high amount of CPU.
    The process of downloading the master database and the database indexing that follows can take 30 minutes to several hours depending on system capacity.
  5. Click
    Update Settings
    .
  6. To download the database immediately, click
    Download Now
    .
    A download occurs only when a newer version becomes available.
    Database indexing occurs after the download and impacts system performance.
    The ANTserver service is not available on the BIG-IP system for approximately 300 milliseconds after the database download completes.

Looking up a URL category in the master database

You can look up a URL to determine whether it already exists in the master database and, if it exists, to see which categories include it.
A URL database is available only on a BIG-IP system with an SWG subscription.
  1. On the Main tab, click
    Access Policy
    Secure Web Gateway
    Database Settings
    URL Category Lookup
    .
  2. In the
    URL
     field, type the URL that you want to look up.
    Type the complete URL, including the URI scheme.
    Type
    https://www.google.com
    ; not
    www.google.com
     or
    https://www.google
    .
  3. Click
    Search
    .
    Custom categories are not searched.
    Results display in the URL Category table.
If the URL is not found, you can add it to an existing or a custom category. If the URL is found, you do not need to do anything, but can recategorize it by adding it to another category.

Configuring logging for the URL database

Configure logging for the URL database so that log messages are published to the destinations, and at the minimum log level, that you specify. (Logging for the URL database occurs at the system level, not the session level, and is controlled using the default-log-setting log setting.)
A URL database is available only on a BIG-IP system with an SWG subscription.
  1. On the Main tab, click
    Access
    Overview
    Event Logs
    Settings
    .
    A log settings table screen opens.
  2. From the table, select
    default-log-setting
     and click
    Edit
    .
    A log settings popup screen displays.
  3. Verify that the
    Enable access system logs
     check box is selected.
  4. To configure settings for access system logging, select
    Access System Logs
     from the left pane.
    Access System Logs settings display in the right panel.
  5. From the
    Log Publisher
     list, select the log publisher of your choice.
    A log publisher specifies one or more logging destinations.
    The BIG-IP system is not a logging server and has limited capacity for storing, archiving, and analyzing logs. For this reason a dedicated logging server is recommended.
  6. To change the minimum log level, from the
    Secure Web Gateway
     list, select a log level.
    Setting the log level to
    Debug
     can adversely impact system performance.
    The default log level is
    Notice
    . At this level, logging occurs for messages of severity Notice and for messages at all incrementally greater levels of severity.
  7. Click
    OK
    .
    The popup screen closes. The table displays.

Viewing a URL database report

You can view URL database log messages in an Access System Logs report if local logging is configured for the URL database.
The BIG-IP system is not a logging server and has limited capacity for storing, archiving, and analyzing logs. For this reason a dedicated logging server is recommended.
Create a report to view URL database event logs.
A URL database is available only on a BIG-IP system with an SWG subscription.
  1. On the Main tab, click
    Access
    Overview
    Access Reports
    .
    The Reports Browser displays in the right pane. The Report Parameters popup screen opens and displays a description of the current default report and default time settings.
  2. Click
    Cancel
    .
    The Report Parameters popup screen closes.
  3. In the Reports Browser in the General Reports list, select
    URL DB Messages
    Run Report
    .
    The Report Parameters popup screen displays.
  4. Update the parameters, if necessary, and click
    Run Report
    .
    The popup screen closes. The report displays in the Report Browser.
The session ID for a URL database message is
00000000
 because URL database downloads occur outside of a client session.

Secure Web Gateway database download log messages

When you deploy Secure Web Gateway (SWG), the database downloads output messages to the log destinations specified in the default-log-setting. This table lists messages that are available only when you enable debug.
Database downloads are possible only on a BIG-IP system with an SWG subscription.
Debug message
Description
Transfer Status 247
The file is transferred successfully to the BIG-IP system. If you see a Transfer Status other than 247, it might indicate an error.
RTU Type
The RTU Type is always 1. If you see an RTU Type other than 1, it might indicate an error.
Expiration Date
The BIG-IP system does not use the expiration date in this message. Instead, the BIG-IP system enforces the SWG license and the database download works accordingly.
Contact Support Contact Support

Have a Question?

Support and Sales >

About F5

Education

F5 Sites

Support Tasks




©2023 F5, Inc. All rights reserved.


Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information