968945-3 : BIG-IQ user interface becomes unstable or unavailable

Links to More Info: BT968945

Component: REST Framework and TMOS Platform

Symptoms:
Most often occurring when first rebooting after upgrading a BIG-IQ 7000 system, the BIG-IQ user interface becomes unstable or unavailable.

Conditions:
This is most often seen on first bootup into an upgraded BIG-IQ 7000 system. Other rare cases might also exist.

Impact:
The postgres service does not start for some time, often up to an hour. During this period other critical services also cannot start or cannot operate correctly.

Workaround:
Find the remaining child processes with this command:

# pgrep -a -u postgres

Kill these processes with the "kill" command. Typically the only processes that need to be killed are listed with "autovacuum" in the process title, so start with those.

Fix:
This issue is now fixed and the BIG-IQ user interface displays properly.

Fixed Versions:
7.1.0.3, 8.2.0

961457-2 : Importing services for a BIG-IP device fails with an error message

Component: BIG-IQ Configuration - Security - Shared Security

Symptoms:
When importing services for a BIG-IP device, BIG-IQ returns a message similar to the following:

Failed to copy configuration to working-config; reason: Failed copying from source subcollections to target subcollections: java.lang.IllegalArgumentException: The attack vector(a Dos Profile Protocol DNS (...) thresholdMode value (manual-multiplier-mitigation) must be [manual, stress-based-mitigation, fully-automatic].

Conditions:
1. The BIG-IP device has DoS profile an attack vector configured and the vector is learn-only with a manual-multiplier-mitigation threshold.
2. Attempt to import the BIG-IP device into BIG-IQ.

Impact:
Import fails.

Fix:
Import now completes as expected.

Fixed Versions:
8.0.0

955145-11 : iControl REST unauthenticated remote command execution vulnerability CVE-2021-22986

Links to More Info: K03009991, BT955145

940317-10 : CVE-2020-13692: PostgreSQL JDBC Driver vulnerability

Links to More Info: K23157312, BT940317

913329 : BIG-IQ analytics data retention policy and data aggregation might not work as expected

Links to More Info: BT913329

Component: AppIQ

Symptoms:
BIG-IQ logs for AppIQ/postaggregator are filled with memory exceptions (java.lang.OutOfMemoryError)

Location of log: var/log/appiq/postaggregator.log

Sample trace in log:

ERROR c.f.a.a.TimeRangeAggregator [scheduling-1] Exception thrown while attempting to perform index aggregations java.lang.OutOfMemoryError: Java heap space

Conditions:
BIG-IQ is configured to collect BIG-IP data.

Impact:
The BIG-IQ disk space might fill up quickly, resulting in overall instability.

Workaround:
Expand JVM memory allocated to both post-aggregator service using the following steps from the BIG-IQ command line:

1. vi /etc/biq_daemon_provision.json

2. edit the restjavad memory setting:

"big_iq": {
    "restjavad": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "800m",
        "SYS_8GB": "3500m",
        "SYS_16GB": "6000m",
        "SYS_32GB": "12700m", -->>> change this to "9800m"
        "SYS_64GB": "20000m",
        "SYS_128GB": "20000m"
      },
      "new_ratio": {
        "SYS_32GB": "1"
      }
    },

3. edit the elasticsearch memory (the one under "big_iq")

 },
    "elasticsearch": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "100m",
        "SYS_8GB": "200m",
        "SYS_16GB": "500m",
        "SYS_32GB": "1600m", -->>> increase this to "4000m"
        "SYS_64GB": "3200m",
        "SYS_128GB": "6400m"
      }
    },

4. bigstart restart restjavad

5. bigstart restart elasticsearch



For post aggregator service:
for versions < 8.0.0:

1. From BIG-IQ, start an SSH session

2. Edit the post-aggregator service config file
       /etc/bigstart/scripts/appiqpostaggregator

3. Change the line:
       From: MAX_JVM_HEAP=200m
       To: MAX_JVM_HEAP=1000m

4. Save the config file.

5. Restart post-aggregator service by typing the following command:
       bigstart restart appiqpostaggregator

For BIG-IQ version later than 8.0.0:
1. vi /etc/biq_daemon_provision.json
2. edit the appiqpostaggregator memory (the one under "big_iq")

"appiqpostaggregator": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "100m",
        "SYS_8GB": "300m",
        "SYS_16GB": "500m",
        "SYS_32GB": "500m", -----> increase this to 1000m
        "SYS_64GB": "500m",
        "SYS_128GB": "500m"
      }
    },

3. Restart post-aggregator service by typing the following command:
       bigstart restart appiqpostaggregator

Fix:
This issue is now fixed and memory exceptions no longer occur.

Fixed Versions:
8.1.0.2

903905-6 : BIG-IQ or BIG-IP devices experience a service disruption during certain circumstances

Links to More Info: BT903905

Component: REST Framework and TMOS Platform

Symptoms:
When a BIG-IP device has been running for 8 weeks or longer without a restart of TMM or a system reboot, BIG-IP services can be disrupted.

Conditions:
-- BIG-IP device has been running for 8 weeks or longer without a TMM restart or system reboot.

-- The BIG-IP system's internal risk-policy subsystem (used by the security features) has not been configured to communicate with an external risk-policy server.

-- In a vCMP configuration, the BIG-IP 'host' instance is susceptible, since no security features can be configured in its context.

-- A BIG-IQ device running any BIG-IQ v8.x release prior to 8.1.0.1 is also susceptible.

Impact:
Traffic disrupted while TMM restarts.

Workaround:
None.

Fix:
This issue has been resolved and no longer occurs.

Fixed Versions:
8.1.0.1

876565-2 : Files in the /var/log/appiq log are not removed as expected

Links to More Info: BT876565

Component: AppIQ

Symptoms:
BIG-IQ has a limit to the number and size of log files logged per day, but there is no limit over the number of days the logs are saved. Due to this, log files accumulate, creating a problem for /var partition and functionality of certain features.

Conditions:
Frequent logging.

Impact:
The /var/ partition space fills up and manual intervention for clean up is required.

Workaround:
Manually clean up the /var/log/appiq sub folder log files.

Fix:
This issue is fixed and manually cleaning up the /var/ partition space is no longer required.

Fixed Versions:
7.1.0.3

846157-4 : TMM may crash while processing traffic on AWS

Links to More Info: K01054113, BT846157

1087885 : BIG-IQ screen might logout user while importing BIG-IP with LTM or DNS services

Links to More Info: BT1087885

Component: BIG-IQ Local Traffic & Management

Symptoms:
When discovering and importing servcies for a BIG-IP device, the BIG-IQ screen displays a message: Waiting for BIG-IQ services to become available. After that, the process might temporarily or permanently be halted, and BIG-IQ logs you out.

Conditions:
An import of BIG-IP devices with the LTM/DNS device with large configurations to BIG-IQ can cause resource constraints on BIG-IQ.

Impact:
BIG-IP import and service discovery fails, the screen becomes unavailable, and you're unable to import and discover services for that device even after logging back into BIG-IQ.

Workaround:
None.

Fix:
This issue is fixed and importing and discovering BIG-IP devices with large LTM/DNS services is successful.

1067565 : In some instances, unable to clone or create APM policies

Links to More Info: BT1067565

Component: BIG-IQ Access

Symptoms:
When attempting to clone or create a new APM policy, BIG-IQ returns the following error:

 "adding non-group-specific item to object map for group"

Conditions:
Attempting to clone an existing APM policy or create a new APM policy. This issue happens only occasionally.

Impact:
You might need to recreate the policy manually.

Workaround:
Remove the affected BIG-IP from BIG-IQ and re-discover/re-import it and clone or create an APM policy.

Fix:
This issue is fixed and you can now successfully clone and create APM policies reliably.

1055309 : Could not write JSON: Null key for a Map not allowed in JSON exception in appiqquery service

Links to More Info: BT1055309

Component: AppIQ

Symptoms:
The appiqquery log in the BIG-IQ /var/log/appiq/queryservice.log has periodic warnings and exceptions with the text:
"Could not write JSON: Null key for a Map not allowed in JSON"

Additionally, BIG-IQ dashboards do not display the BIG-IP device's health.

Conditions:
This can hapen when discovering BIG-IP devices and BIG-IQ is left with a corrupted configuration that holds all the discovered BIG-IPs.

The following query from the BIG-IQ command line displays items similar to the following:
curl -X GET localhost:8100/shared/resolver/device-groups/cm-bigip-allBigIpDevices/devices
.....
"items": [
    {
      "kind": "shared:resolver:device-groups:restdeviceresolverdevicestate",
      "uuid": "b09f4af2-b1d3-4c72-ae75-d3f9593a85ae",
      "state": "UNDISCOVERED",
      "address": "10.46.175.147",
      "selfLink": "https://localhost/mgmt/shared/resolver/device-groups/cm-bigip-allBigIpDevices/devices/b09f4af2-b1d3-4c72-ae75-d3f9593a85ae",
      "deviceUri": "https://10.46.175.147:443/undiscovered",
      "groupName": "cm-bigip-allBigIpDevices",
      "httpsPort": 443,
      "generation": 1,
      "lastUpdateMicros": 1626741958467591
    },

Unlike a BIG-IP with a successful configuration, corrupted configurations have the state "UNDISCOVERED" and there is no hostname field.

Impact:
BIG-IQ dashboards do not display device health statuses.

Workaround:
Delete the corrupted configuration (the items that are in an "UNDISCOVERED" state).

For example at the "Condition" section, run from the BIG-IQ shell the rest DELETE with the "uuid" of the corrupted item:
curl -X DELETE localhost:8100/shared/resolver/device-groups/cm-bigip-allBigIpDevices/devices/b09f4af2-b1d3-4c72-ae75-d3f9593a85ae

Fix:
This issue is now fixed.

Fixed Versions:
8.0.0.1

1052933 : DDOS Protection Summary page is slow on system with many active alerts

Links to More Info: BT1052933

Component: AppIQ

Symptoms:
The BIG-IQ DDOS Protection Summary page (https://{CM-IP}/ui/monitoring/dashboards/ddos/protection-summary) is slow and can take a number of minutes to load. At the same time, the BIG-IQ CPU load can spike to one hundred percent and also spike the load average.

Conditions:
This happens when BIG-IQ has several active alerts.

Impact:
You cannot access the BIG-IQ DDOS Protection summary page and BIG-IQ CPU spikes for a number of minutes.

Fix:
This issue is fixed and the DDOS Protection Summary page load time is now less than a second.

1052637 : BIG-IQ DDoS DNS summary page is slow when multiple users use this page

Links to More Info: BT1052637

Component: AppIQ

Symptoms:
The BIG-IQ DDoS DNS summary page loads slowly or does cannot be accessed.

Conditions:
This can happen when multiple users attempt to access the DDoS DNS Summary page simultaneously.

Impact:
The DDoS DNS summary page loads slowly or does cannot be accessed.

Workaround:
Reduce the refresh rate of the page from 5 seconds to 1 minute.

Fix:
This issue has been resolved and no longer happens.

1052149 : F5-dtuser is unable to access BIG-IQ

Links to More Info: BT1052149

Component: REST Framework and TMOS Platform

Symptoms:
When restjavad frequently restarts, the BIG-IQ-created f5-dtuser is unable to access BIG-IQ and restjavad errors are logged.

Conditions:
When there is a frequent restart of restjavad.

Impact:
F5-dtuser is unable to access BIG-IQ and restjavad errors are logged.

Workaround:
Run command "tmsh modify auth user f5-dtuser partition-access replace-all-with { all-partitions { role guest }}"

Fix:
This issue is fixed.

1051505 : Scheduled Snapshots fail

Links to More Info: BT1051505

Component: BIG-IQ Device Management

Symptoms:
A scheduled Snapshot fails and the /var/log/restjavad.x.log displays the following message:

[WARN][xxx][/cm/shared/event/alerts AlertCollectionWorker] Data Collection device data snapshot xxxxxxxz has failed because the following error occurred, null

Conditions:
When a BIG-IQ Data Collection Device has performance issues, some scheduled Snapshot requests on external storage can take more than 30 seconds to initialize. This can cause the Snapshot to fail.

Impact:
Scheduled storage Snapshots that are saved to external storage can fail.

Workaround:
None

Fix:
This issue is fixed.

1047481-1 : Wildcard certificate support for Let's Encrypt

Component: BIG-IQ Local Traffic & Management

Symptoms:
Lets Encrypt provides support for certificate wildcard domains, but BIG-IQ cannot generate certificates for wildcard domains using a Let's Encrypt CA.

Conditions:
When generating a certificate for a wildcard domain using Let's Encrypt CA from BIG-IQ.

Impact:
BIG-IQ cannot generate a certificate for wildcard domains using Let's Encrypt, event though Let's Encrypt supports wildcard certificates.

Fix:
Certificate for wildcard domain can now be generated from BIG-IQ using a Let's Encrypt CA.

1046429 : DNS email reports show empty charts

Links to More Info: BT1046429

Component: BIG-IQ Monitoring - Dashboards & Reports

Symptoms:
Email reports for DNS statistics are received without any content.

Conditions:
Reports are scheduled on BIG-IQ for DNS.

Impact:
Scheduled emails for DNS statistics reports are empty.

Fix:
This is fixed in BIG-IQ v8.1.0.2

Fixed Versions:
8.1.0.2

1044337-5 : Statistics might be missing from BIG-IQ or data retention might not work as expected

Component: AppIQ

Symptoms:
In some instances, you cannot see the BIG-IP statistics data from BIG-IQ, or the data retention processes do not work as expected. BIG-IQ reports that Statistics overflow protection is enabled.

You can identify if this issue is occurring by viewing the BIG-IQ /var/log/appiq/configserver.log and seeing a message similar to the following:

WARN c.f.a.c.r.BookkeeperUpdate [scheduling-1] Failed update bookkeeper for time level 'tl0' with first full data of 'java.lang.NullPointerException'
WARN c.f.a.c.r.BookkeeperUpdate [scheduling-1] Failed update bookkeeper for time level 'tl1' with first full data of 'java.lang.NullPointerException

Conditions:
This can occur intermittently during statistics collection from BIG-IP to BIG-IQ and can happen when the retention processor causes a loss of some retention records (book-keeper-start entries are lost while book-keeper-end entries are retained).

Impact:
Statistics are missing or not retained as expected.

Workaround:
From the BIG-IQ command line:

1) Make a backup of the book-keeper-end index:
  curl -sk https://localhost:9200/_reindex?pretty -H "content-type: application/json" -d '{"source":{"index":"book-keeper-end"},"dest":{"index":"bu-book-keeper-end"}}'
2) Delete the book-keeper-end index:
  curl -skX DELETE https://localhost:9200/book-keeper-end

Note: The book-keeper-end index will be rebuilt automatically and will cause no loss of statistic data.

Fix:
This issue is now fixed.

Fixed Versions:
8.1.0.2

1043397 : DOS statistics dashboards not available for BIG-IP devices version > 13.1.x

Links to More Info: BT1043397

Component: BIG-IQ Network Security

Symptoms:
The DOS statistics dashboard for BIG-IP devices shows a blank screen

Conditions:
The BIG-IP device version is higher than 13.1.x

Impact:
DOS statistic dashboards are empty.

Workaround:
None

1042593 : DOS event page does not allow filtering by BIG-IP

Component: BIG-IQ Configuration - Security - Shared Security

Symptoms:
Unable to filter the DOS event page by BIG-IP.

Conditions:
After adding BIG-IP to BIG-IQ for management using its discovery address, you cannot filter the DOS event page using the BIG-IP column.

Impact:
You will be unable to use DOS event page as intended.

Workaround:
Remove BIG-IP devices and re-add them using the management address.

Fix:
This issue is fixed.

1041689-2 : Restjavad logs IO Reactor errors and becomes unstable

Links to More Info: BT1041689

Component: REST Framework and TMOS Platform

Symptoms:
restjavad stop functioning and logs errors similar to:

"I/O reactor has been shut down", "Client is not running", "too many files open" and "Restarting apache async client". Further restjavad process was unable to write logs to restjavad.0.log, async error logs, logs with text logs logged.

Conditions:
This occurs when Usage Data collection is enabled from the System > Usage Data screen.

Impact:
restjavad eventually stops functioning.

Workaround:
Disable usage data collection by navigating to System > Usage Data and deselect the Usage Data Collection check box.

Fix:
This issue is fixed.

Fixed Versions:
8.1.0.2

1041665 : Managed BIG-IP devices deployed with SSL Orchestrator applications fails to import APM

Links to More Info: BT1041665

Component: BIG-IQ Access

Symptoms:
When deploying an SSL Orchestrator application containing an APM configuration from BIG-IQ to a managed BIG-IP device, the following error occurs:

Coordinator failed during working-config update, roll-back working-config started; reason: Worker http://localhost:8100/cm/access/working-config/apm/policy/policy-item failed validation with status 500: java.lang.IllegalStateException: <SSL Orchestrator app object> refers to nonexistent object.

Conditions:
If any APM policy objects are created as part of the SSL Orchestrator application, the APM import fails.

Impact:
The BIG-IP device's APM service is affected.

Fix:
This issue is fixed.

1040345 : IPSec service is not working&start;

Links to More Info: K63534030, BT1040345

Component: BIG-IQ Configuration - Network

Symptoms:
The IPsec service is not creating logs.

Conditions:
This can be seen as soon as IPSec service is activated.

When you run the following command from BIG-IQ
   # curl -sk -XGET -u admin:admin https://localhost:9200/_template | jq keys

..the output does not display the following entry
    "ipsec-event-logs_template"

Impact:
IPSec service does not function.

Workaround:
Use the following article to work around this issue: https://support.f5.com/csp/article/K63534030

Fix:
This issue is now fixed.

Fixed Versions:
8.1.0.2

1038981 : BIG-IQ High availability (HA) configuration fails when initial database copy takes longer than 15 minutes

Links to More Info: BT1038981

Component: REST Framework and TMOS Platform

Symptoms:
You might see one or more of following symptoms:

1. When you run the following command on the standby BIG-IQ to examine the restjavad log

    $ grep 'Re-starting restjavad' /var/log/daemon*

  You might see several restjavad service restarts logged. For example:
    .... logger[10602]: Re-starting restjavad
    .... logger[7385]: Re-starting restjavad


2. The BIG-IQ GUI might halt with a message similar to:
    'Waiting for BIG-IQ services to become available'
  
3. If you run the following command to view the standby BIG-IQ log:

    $ tail -f /var/log/ha_pg_basebackup.log

When database replication starts you will see the following message:

    ...Setup_slave] Configuring pg_basebackup
waiting for checkpoint

If the log shows has not reached 100% , you might see errors similar to the following:
    538210/545848 kB (98%), 0/1 tablespace
    pg_basebackup: could not create directory "/var/lib/pgsql/data/base/1": File exists
    pg_basebackup: removing data directory "/var/lib/pgsql/data"
could not remove file or directory "/var/lib/pgsql/data": Directory not empty
    pg_basebackup: failed to remove data directory

Under normal circumstances, you would typically see a success message similar to:
    ..Setup_slave] Completed pg_basebackup successfully.

Conditions:
When creating a BIG-IQ high availability configuration, the standby BIG-IQ pulls the database from the active BIG-IQ. If this takes longer than 15 minutes to complete, the high availability (HA) configuration fails. This can happen on low bandwidth networks or when the database is very large.

Impact:
BIG-IQ coinfigurations with large database or limited network bandwidth cannot form a successful BIG-IQ high availability configuration.

Workaround:
Contact support for a hotfix.

The following process is only to recover the standby BIG-IQ.

1. Browse to System::THIS DEVICE::BIG-IQ high availability (HA) on the primary BIG-IQ

2. DO NOT click the 'Repair Standby Database' button.

3. Log in to the standby BIG-IQ device from the command line.

4. If you see messages that restjavad service restarts, you might be unable to type commands. Type the following command to stop the service:
     $ bigstart stop restjavad

5. Type the following command to reset the database on the standby BIG-IQ:
     $ pgsh -i -f

6. After the standby BIG-IQ has recovered, click 'Revert to standalone' on the active BIG-IQ.

If these steps work, delete the database on the standby BIG-IQ by stopping postgres, deleting the /var/lib/pgsql/data directory, then starting the postgres service.

Fix:
The initial database copy of the BIG-IQ high availability (HA) process now waits for 60 minutes before failing.

1035277 : PostgresDB bloat can cause upgrade to fail&start;

Links to More Info: BT1035277

Component: REST Framework and TMOS Platform

Symptoms:
The /var partition can fill up and the bulk of the data in /var is in /var/lib/pgsql/data/base. This will result in incomplete configuration of PostgresDB. This can cause recurring errors in postgres/bootstrap logs.

Conditions:
This occurs due to certain RBAC configurations (in postgresDB prior) present prior to the upgrade (for example, a large number of custom roles).

Impact:
The upgrade can fail if the disk fills up during upgrade. Or you will see the device left in an inoperable state following a successful upgrade. Usually the latter occurs.

Fix:
This issue is fixed.

1034113-4 : RADIUS authentication fails

Links to More Info: BT1034113

Component: REST Framework and TMOS Platform

Symptoms:
BIG-IQ fails to authenticate users with RADIUS.

Conditions:
When BIG-IQ is configured to use RADIUS for authentication and the RADIUS server attributes are appended with Unicode characters.

Impact:
Users are unable to log in to BIG-IQ because RADIUS authentication failed.

Fix:
This issue is now fixed and RADIUS authentication works as expected.

Fixed Versions:
8.1.0, 8.1.0.1

1033125 : Missing usernames in sessions related to network tunnels

Links to More Info: BT1033125

Component: BIG-IQ Access

Symptoms:
There are various symptoms

1. Session records related to network tunnels do not display all attributes in reports/dashboards.

2. BIG-IQ logs exceptions related to bulk update failure:

[WARN][/cm/access/session-summary-builder SessionSummaryBuilderWorker] 2 Session Items discarded after max retries
[INFO][/cm/access/session-summary-builder SessionSummaryBuilderWorker] Bulk request failure errors 3 error description

--------------------------------------

NOTE: If any of the following commands reports found = false, then you are impacted by ID1040777 and not the ID1033125

   #curl -s -u admin:admin --insecure -X GET https://localhost:9200/_scripts/updateNetworkAccessTunnels | jq .found
   #curl -s -u admin:admin --insecure -X GET https://localhost:9200/_scripts/updateSessionSummary | jq .found
   #curl -s -u admin:admin --insecure -X GET https://localhost:9200/_scripts/updateTokenFromPhase1Task | jq .found
   #curl -s -u admin:admin --insecure -X GET https://localhost:9200/_scripts/updateTokenFromPhase2Task | jq .found
   #curl -s -u admin:admin --insecure -X GET https://localhost:9200/_scripts/updateTokenFromSessionSummary | jq .found

Conditions:
-- BIG-IQ DCD cluster with Access listener enabled
-- Active tunnel traffic

Impact:
Session information (like the Username field) related to network tunnels are missing in reports/dashboards.

Workaround:
You can resolve this issue by the following steps. (Note the additional flag in Step 3.)

1. Download the repair scripts/tools following steps in https://support.f5.com/csp/article/K63534030 (Step1 under 'Recommended Actions')

2. Backup, Overwrite .painless scripts by following Step 2.A/2.B under 'Recommended Actions'

3. Update .painless files in Elasticsearch cluster (Step 2.C under 'Recommended Actions') using below command
    # sh PushAPMScriptsToES.sh -force

  NOTE: You MUST use '-force' if not it will not resolve the current issue

4. You do not need to do Step 2.D
    
5. Finish by running Step 4

Fix:
This issue is fixed.

Fixed Versions:
8.1.0.1

1032641 : BIG-IQ analytics performance is reduced when receiving high rate of DNS statistics from BIG-IP

Links to More Info: BT1032641

Component: AppIQ

Symptoms:
When BIG-IQ receives a large number of DNS statistics from a BIG-IP (From appiqagent at DCD), BIG-IQ logs the following error:

2021-05-26 20:09:20,009 INFO c.f.a.a.r.ReactorPipeline [XNIO-2 task-45] Source buffer high watermark reached (available-capacity: 5905, max-capacity: 65536, usage: 90.99%, please retry later
2021-05-26 20:09:20,009 WARN c.f.a.a.c.RawDataAPIController [XNIO-2 task-45] Sending HTTP response 504 Gateway Timeout with Retry-After : 3434ms

Conditions:
One or more BIG-IP devices are configured to send DNS statistics.

Impact:
When the high-water mark level is too high, statistics are lost.

Fix:
This issue has been resolved and DNS statistics are properly reported.

Fixed Versions:
8.1.0.1

1031321 : BIG-IQ analytics are unresponsive

Links to More Info: BT1031321

Component: AppIQ

Symptoms:
BIG-IQ analytics become unresponsive and errors display in the /var/log/elasticsearch log.

Conditions:
BIG-IQ configured with Data Collection Devices (DCD)s.

Impact:
Analytics are not report as expected.

Workaround:
Perform the following steps on each DCD:

1. Edit /var/config/appiq/agentmanager/config/application-bigiq.yml

For"indexes-overflow-protection" change the "initial-delay" from 7200000 to 0.

2. Save the configuration.

3. Restart appiqagent to reload the configuration by typing:

bigstart restart appiqagent

Fix:
This issue is now fixed and analytics work as expected.

Fixed Versions:
8.1.0.1

1029429-1 : Rediscovery of BIG-IP with ASM service fails post upgrade to BIG-IQ v8.1.0 for certain versions of BIG-IP devices discovered pre-upgrade

Component: BIG-IQ Web Application Security (ASM)

Symptoms:
After upgrading to BIG-IQ v8.1, BIG-IP rediscovery is failing for ASM - “Duplicate item. Key already exists: name : VIOL_WEB_SCRAPING”

Conditions:
ASM rediscovery fails only in following combinations of BIG-IP and BIG-IQ.

This issue occurs under 2 different ASM service discovery conditions:
1. During a BIG-IQ upgrade from 8.0 to 8.1 with BIG-IP version 16.0.0.1
2. When a BIG-IQ manages BIG-IP devices that were upgraded from version 16.0.0 to 16.0.0.1 or 15.0.1 to 15.0.1.1.

Impact:
Primary rediscovery attempt, following BIG-IQ upgrade, fails for ASM and results in error. This error occurs only when trying to discover ASM services on affected BIG-IP versions.

Workaround:
If this error occurs during first attempt post upgrade, try re-discovering BIG-IP with the ASM service.

Fix:
This issue is fixed.

1029321 : Renewal of certificates provided by Venafi or Let's Encrypt fails if the certificate is used in a profile

Links to More Info: BT1029321

Component: BIG-IQ Local Traffic & Management

Symptoms:
Trying to renew certificates signed by Let's Encrypt/Venafi from the GUI or API of BIG-IQ does not work. No error is returned in the BIG-IQ GUI, but a error is logged.

Conditions:
- The CSR was signed by Venafi or Let's Encrypt.
 
- The certificate and key are used by a profile or pinned to a managed BIG-IP device.

- You are trying to manually (or automatically) renew the certificate.

Impact:
Renewal fails with the following error in the logs:

[/cm/adc-core/external-ca/lets-encrypt/csr-request/<uuid>/worker LetsEncryptCertRequestTaskWorker] Error occurred while deleting key state with exception : /Common/<key name>.key is in use by Profile Client SSL '/Common/<profile name>'.

Workaround:
1- Un-pin the certificate and key from all BIG-IP devices that use it.

2- Change your SSL Profile configuration on BIG-IQ, and use a different cert/key pair.

Do not deploy these changes to your BIG-IP.

3- Manually renew the certificate.

4- Revert changes done in step #1 and #2.

5- Deploy changes to the BIG-IP.

Fix:
This issue is now fixed and certificates are renewed as expected.

1028669-9 : Python vulnerability: CVE-2019-9948

Links to More Info: K28622040, BT1028669

1028329 : Filter query in AFM rule search is slow

Links to More Info: BT1028329

Component: BIG-IQ Network Security

Symptoms:
When there are a high number of referenced items, BIG-IQ might take more time to filter for AFM rules in rule lists or policies in environments with high load.

Conditions:
1. Navigate to Configuration >> Security >> Network Security >> Rule Lists >> Rule List Item
2. In the "Filter" search area enter a search string and search

Impact:
Performance is degraded.

Workaround:
Filtering might be slow, but BIG-IQ provides the correct results.

Fix:
Filtering performance is now improved on BIG-IQ.

Fixed Versions:
8.1.0.1

1025165 : Error managing DNS objects from BIG-IQ with RADIUS authentication

Component: BIG-IQ Local Traffic & Management

Symptoms:
While managing DNS objects using RADIUS authentication for BIG-IQ, you might get an error similar to the following:

"Failed to xxxxx. Failed while self servicing resourceReference"

Conditions:
You are managing DNS objects on BIG-IQ after logging in using a RADIUS account.

Impact:
You are unable to manage DNS objects on BIG-IQ as a RADIUS user.

Workaround:
None.

Fix:
You can now use RADIUS for BIG-IQ authentication and successfully manage DNS objects.

Fixed Versions:
8.1.0.1

1021149 : Search and data collection can become unhealthy after adding a Data Collection Device (DCD)

Links to More Info: K68489751, BT1021149

Component: AppIQ

Symptoms:
Data Collection Devices (DCD) become unhealthy after adding a DCD and logs messages similar to the following in /var/log/elasticsearch/eslognode.log

2020-07-14T00:04:13,292][INFO ][o.e.d.z.ZenDiscovery ] [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx] master_left [{xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx}{xxxxxxxxxxx}{xxxxx}{192.0.2.3}{192.0.2.3:9300}{zone=default}], reason [failed to ping, tried [3] times, each with maximum [30s] timeout]
[2020-07-14T00:04:13,293][WARN ][o.e.d.z.ZenDiscovery ] [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx] master left (reason = failed to ping, tried [3] times, each with maximum [30s] timeout), current nodes:

Conditions:
There are long-lived TCP connections on port 9300 between BIG-IQ and data collection devices (DCD). Those connections send keepalive probes after being idle for 299 seconds. If an intermediate device idle-times out this TCP connection in less than 299 seconds, the ES cluster will experience stability problems.

Impact:
Data Collection Devices are not healthy.

Workaround:
On each BIG-IQ and DCD, perform the following steps:

1. Edit the file /etc/bigstart/scripts/elasticsearch and modify the following setting to meet the needs of the environment:

sysctl -w net.ipv4.tcp_keepalive_time=299 \
net.ipv4.tcp_keepalive_intvl=60 \

2. Set net.ipv4.tcp_keepalive_time to the number of seconds a connection is idle before keepalives are sent.

3. Set net.ipv4.tcp_keepalive_intvl to the interval between keepalives

Example values that work in most environments:

sysctl -w net.ipv4.tcp_keepalive_time=20 \
net.ipv4.tcp_keepalive_intvl=20 \
 
4. To set the values type the following commands:

Note: The values set in /etc/bigstart/scripts/elasticsearch take effect on the next boot, but will not persist after an upgrade.

sysctl -w net.ipv4.tcp_keepalive_time=20
sysctl -w net.ipv4.tcp_keepalive_intvl=20

See also https://support.f5.com/csp/article/K68489751

Fix:
This issue is fixed and the DCD no longer becomes unhealthy.

Fixed Versions:
8.1.0.1

1016473-2 : Collected-stats-internal-logging is disabled when BIG-IP is removed from BIG-IQ

Links to More Info: BT1016473

Component: AppIQ

Symptoms:
Collected-stats-internal-logging becomes disabled when a managed BIG-IP is removed from BIG-IQ.

Conditions:
1. Check what is the default setting on the BIG-IP AFM, and if the collected-stats-internal-logging is enabled.
2. Add BIG-IP to BIG-IQ, discover LTM and AFM services. After adding AFM device into BIG-IQ's managed devices, check if the setting still appears to be as follows:

   [root@bigip1:Active:Standalone] config # tmsh list security analytics settings collected-stats-internal-logging
   security analytics settings {
       collected-stats-internal-logging enabled
   }
3.Remove services from BIG-IP in BIG-IQ, then remove BIG-IP from BIG-IQ.
4. Check security analytics settings 'collected-stats-internal-logging' and 'collect-all-dos-statistic'.

Impact:
Collected-stats-internal-logging is disabled for BIG-IP AFM.

Fix:
Collected-stats-internal-logging status remains as expected, even after removing a managed BIG-IP from BIG-IQ.

1016169-5 : Installation of BIG-IQ version 8.0.0 EHF fails

Links to More Info: BT1016169

Component: REST Framework and TMOS Platform

Symptoms:
When installing BIG-IQ 8.0.0 EHF, the installation fails and BIG-IQ returns the following error in the /var/log/restjavad.0.log.

"...Failed to run pre upgrade hook due to 'mount: /dev/loop0 is write-protected, mounting read-only
mount: /dev/loop1 is write-protected, mounting read-only
Querying for em_rest_java RPM file in ISO failed."

Conditions:
Installation of BIG IQ 8.0.0 EHF build fails if EHF ISO does not have em_rest_java RPM as part of ISO.

Impact:
Installation fails.

Workaround:
N/A

Fix:
This issue is now fixed and package validation is done as per expectations.

Fixed Versions:
8.1.0.1

1011941-7 : Admin users can run arbitrary commands on the BIG-IQ configuration utility

Links to More Info: K06024431

• The F5 Networks Technical Support web site: http://www.f5.com/support/
• The AskF5 web site: https://support.f5.com/csp/#/home
• The F5 DevCentral web site: http://devcentral.f5.com/

Generated: Tue Apr 26 07:29:10 2022 PDT

Copyright F5 Networks (2022) - All Rights Reserved