913329-4 : BIG-IQ analytics data retention policy and data aggregation may not work as expected

Component: AppIQ

Symptoms:
AppIQ/postaggregator logs are filled with memory exceptions (java.lang.OutOfMemoryError)

Location of log: var/log/appiq/postaggregator.log

Sample trace in log:

ERROR c.f.a.a.TimeRangeAggregator [scheduling-1] Exception thrown while attempting to perform index aggregations java.lang.OutOfMemoryError: Java heap space

Conditions:
BIG-IQ configured to collect data.

Impact:
- The BIG-IQ disk space might fill up quickly, resulting in overall instability

Workaround:
Expand JVM memory allocated to both post-aggregator service and the Elasticsearch instance on the CM.

For Elasticsearch instance on the CM:

1. vi /etc/biq_daemon_provision.json

2. edit the restjavad memory setting:

"big_iq": {
    "restjavad": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "800m",
        "SYS_8GB": "3500m",
        "SYS_16GB": "6000m",
        "SYS_32GB": "12700m", -->>> change this to "9800m"
        "SYS_64GB": "20000m",
        "SYS_128GB": "20000m"
      },
      "new_ratio": {
        "SYS_32GB": "1"
      }
    },

3. edit the elasticsearch memory (the one under "big_iq")

 },
    "elasticsearch": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "100m",
        "SYS_8GB": "200m",
        "SYS_16GB": "500m",
        "SYS_32GB": "1600m", -->>> increase this to "4000m"
        "SYS_64GB": "3200m",
        "SYS_128GB": "6400m"
      }
    },

4. bigstart restart restjavad

5. bigstart restart elasticsearch



For post aggregator service:
for versions < 8.0.0:

1. On the CM console start a SSH session

2. Edit the post-aggregator service config file
       /etc/bigstart/scripts/appiqpostaggregator

3. Change below line
       Old: MAX_JVM_HEAP=200m
       New: MAX_JVM_HEAP=1000m

4. Save the config file

5. Restart post-aggregator service
       bigstart restart appiqpostaggregator

for versions >= 8.0.0
1. vi /etc/biq_daemon_provision.json
2. edit the appiqpostaggregator memory (the one under "big_iq")

"appiqpostaggregator": {
      "active": true,
      "memory_allocation": {
        "SYS_4GB": "100m",
        "SYS_8GB": "300m",
        "SYS_16GB": "500m",
        "SYS_32GB": "500m", -----> increase this to 1000m
        "SYS_64GB": "500m",
        "SYS_128GB": "500m"
      }
    },

3. Restart post-aggregator service
       bigstart restart appiqpostaggregator

903905-5 : BIG-IQ or BIG-IP devices experience a service disruption during certain circumstances

Component: REST Framework and TMOS Platform

Symptoms:
The TMM process might eventually run out of memory, which can result in a disruption in BIG-IP services.

Conditions:
-- BIG-IP device has been running for 8 weeks or longer without a TMM restart or system reboot.

-- The BIG-IP system's internal risk-policy subsystem (used by the security features) has not been configured to communicate with an external risk-policy server.

-- In a vCMP configuration, the BIG-IP 'host' instance is susceptible, since no security features can be configured in its context.

-- A BIG-IQ device running any BIG-IQ v8.x release prior to 8.1.0.1 is also susceptible.

Impact:
Traffic disrupted while TMM restarts.

Workaround:
None.

Fix:
Default configuration of security mechanism no longer causes memory leak in TMM.

1041689-1 : Restjavad logs IO Reactor errors and becomes unstable

Component: REST Framework and TMOS Platform

Symptoms:
restjavad stop functioning and logs errors similar to:

"I/O reactor has been shut down", "Client is not running", "too many files open" and "Restarting apache async client". Further restjavad process was unable to write logs to restjavad.0.log, async error logs, logs with text logs logged.

Conditions:
This occurs when Usage Data collection is enabled from the System > Usage Data screen.

Impact:
restjavad eventually stops functioning.

Workaround:
Disable usage data collection by navigating to System > Usage Data and deselect the Usage Data Collection check box.

1041677-1 : BIG-IQ unable to upload QKview to iHealth

Component: BIG-IQ Device Management

Symptoms:
Qkview upload to ihealth server fails.

Conditions:
Using the BIG-IQ GUI to upload a qkview to iHealth.

Impact:
Qkview upload from BIG-IQ fails. This includes a BIG-IQ device qkview or a BIG-IP device qkview.

Fix:
Fixed in 8.1.0.2

1040345-1 : IPSec service broken in BIG-IQ v8.0.x/8.1.x★

Solution Article: K63534030

Component: BIG-IQ Configuration - Network

Symptoms:
IPSec service no longer functions in BIG-IQ v8.0.x/8.1.x

When you run the following command from BIG-IQ
   # curl -sk -XGET -u admin:admin https://localhost:9200/_template | jq keys

..the output does not display the following entry
    "ipsec-event-logs_template"

Conditions:
This can be seen as soon as IPSec service is activated.

Impact:
IPSec service will not function.

Workaround:
You can follow steps in https://support.f5.com/csp/article/K63534030

Fix:
This issue is now fixed.

1038757 : Connectivity to BIG-IP devices after upgrading a BIG-IQ HA configuration to v8.1.0

Component: BIG-IQ Configuration - Infrastructure

Symptoms:
After upgrading a BIG-IQ HA configuration, authorization fails to the standby BIG-IQ which results in communication failure to the standby BIG-IQ to BIG-IP devices.

Conditions:
Upgrading BIG-IQ HA configuration to v8.1.0.

Impact:
The standby BIG-IQ cannot communicate with managed BIG-IP devices.

Workaround:
Remove and re-add the BIG-IP devices.

Fix:
This issue is fixed in BIG-IQ v8.1.0.1

1037853-1 : Users assigned to a role that is not Administration can see the option for scheduling a report, but cannot schedule it

Component: AppIQ

Symptoms:
Users assigned to a non-Administrative role can see the scheduling option for reports, but cannot create a schedule.

Conditions:
When a non-Administrative role is assigned to a user and they attempt to create a report schedule.

Impact:
Unable to schedule a report.

Fix:
Users assigned to a role that does not have permission to schedule reports will no longer see the report schedule option.

1035093-2 : Assigning a utility license to an unmanaged device

Solution Article: K41117246

Component: REST Framework and TMOS Platform

Symptoms:
When attempting to assign a utility license to an unmanaged BIG-IP device, BIG-IQ returns the following error:

"The system returned an unexpected error (400 bad request). Error while assigning license to unmanaged device."

Conditions:
When you attempt to assign a utility license to an unmanaged BIG-IP device.

Impact:
You cannot assign utility licenses to unmanaged BIG-IP devices.

Workaround:
None.

Fix:
This is fixed in BIG-IQ version 8.1.0.1. You can now assign utility licenses to unmanaged BIG-IP devices.

1034825-2 : Discovering and importing BIG-IP configurations with objects with the same name

Component: BIG-IQ Configuration - Infrastructure

Symptoms:
When adding/discovering BIG-IP devices, configured in a device service cluster, BIG-IQ returns an error similar to the following:

Failed to synchronize clustered devices; reason: Device sync failed from device bigip1.example.com (10.30.40.100) to device bigip2.example.com (10.30.40.101): working-config target object sender failed: Duplicate item. Key already exists: partition : Common, deviceReference : /mgmt/shared/resolver/device-groups/cm-adccore-allbigipDevices/devices/f9c47c54-a1d5-5a65-97f2-334457840585, name : HTTPServer.

Conditions:
This happens when:

1) You add a BIG-IP device that is configured in a device service cluster.
2) More than one object in the BIG-IP configuration has the same name.

For example, both an LTM virtual server and network route shared the name: HTTPServer

Impact:
BIG-IQ is unable to discover the BIG-IP device.

Fix:
BIG-IQ can now manage BIG-IP device objects with the same name in its configuration.

1032641-6 : BIG-IQ analytics performance is reduced when receiving high rate of DNS statistics from BIG-IP

Component: AppIQ

Symptoms:
When BIG-IQ receives a large number of DNS statistics from a BIG-IP (From appiqagent at DCD), BIG-IQ logs the following error:

2021-05-26 20:09:20,009 INFO c.f.a.a.r.ReactorPipeline [XNIO-2 task-45] Source buffer high watermark reached (available-capacity: 5905, max-capacity: 65536, usage: 90.99%, please retry later
2021-05-26 20:09:20,009 WARN c.f.a.a.c.RawDataAPIController [XNIO-2 task-45] Sending HTTP response 504 Gateway Timeout with Retry-After : 3434ms

Conditions:
One or more BIG-IP devices are configured to send DNS statistics.

Impact:
When the high-water mark level is too high, statistics are lost.

Fix:
This issue has been resolved and DNS statistics are properly reported.

1031321-4 : BIG-IQ analytics are unresponsive

Component: AppIQ

Symptoms:
BIG-IQ analytics become unresponsive and errors display in the /var/log/elasticsearch log.

Conditions:
BIG-IQ configured with Data Collection Devices (DCD)s.

Impact:
Analytics are not report as expected.

Workaround:
Perform the following steps on each DCD:

1. Edit /var/config/appiq/agentmanager/config/application-bigiq.yml

For"indexes-overflow-protection" change the "initial-delay" from 7200000 to 0.

2. Save the configuration.

3. Restart appiqagent to reload the configuration by typing:

bigstart restart appiqagent

1028329-3 : Filter query in AFM rule search is slow

Component: BIG-IQ Network Security

Symptoms:
When there are a high number of referenced items, BIG-IQ might take more time to filter for AFM rules in rule lists or policies in environments with high load.

Conditions:
1. Navigate to Configuration >> Security >> Network Security >> Rule Lists >> Rule List Item
2. In the "Filter" search area enter a search string and search

Impact:
Performance is degraded.

Workaround:
Filtering might be slow, but BIG-IQ provides the correct results.

Fix:
Filtering performance is now improved on BIG-IQ.

1021149-4 : Elasticsearch can become unhealthy after adding a Data Collection Device (DCD)

Solution Article: K68489751

Component: AppIQ

Symptoms:
Elasticsearch displays as unhealthy and logs messages similar to the following in /var/log/elasticsearch/eslognode.log

2020-07-14T00:04:13,292][INFO ][o.e.d.z.ZenDiscovery ] [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx] master_left [{xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx}{xxxxxxxxxxx}{xxxxx}{192.0.2.3}{192.0.2.3:9300}{zone=default}], reason [failed to ping, tried [3] times, each with maximum [30s] timeout]
[2020-07-14T00:04:13,293][WARN ][o.e.d.z.ZenDiscovery ] [xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx] master left (reason = failed to ping, tried [3] times, each with maximum [30s] timeout), current nodes:

Conditions:
There are long-lived TCP connections on port 9300 between BIG-IQ and data collection devices (DCD). Those connections send keepalive probes after being idle for 299 seconds. If an intermediate device idle-times out this TCP connection in less than 299 seconds, the ES cluster will experience stability problems.

Impact:
ElasticSearch Cluster is not healthy.

Workaround:
On each BIG-IQ and DCD device, perform the following steps:

1. Edit the file /etc/bigstart/scripts/elasticsearch and modify the following setting to meet the needs of the environment:

sysctl -w net.ipv4.tcp_keepalive_time=299 \
net.ipv4.tcp_keepalive_intvl=60 \

2. Set net.ipv4.tcp_keepalive_time to the number of seconds a connection is idle before keepalives are sent.

3. Set net.ipv4.tcp_keepalive_intvl to the interval between keepalives

Example values that work in most environments:

sysctl -w net.ipv4.tcp_keepalive_time=20 \
net.ipv4.tcp_keepalive_intvl=20 \
 
4. To set the values type the following commands:

Note: The values set in /etc/bigstart/scripts/elasticsearch take effect on the next boot, but will not persist after an upgrade.

sysctl -w net.ipv4.tcp_keepalive_time=20
sysctl -w net.ipv4.tcp_keepalive_intvl=20

See also https://support.f5.com/csp/article/K68489751

Fix:
This issue is fixed and the DCD no longer becomes unhealthy.

1016169-3 : Installation of BIG-IQ version 8.0.0 EHF fails

Component: REST Framework and TMOS Platform

Symptoms:
When installing BIG-IQ 8.0.0 EHF, the installation fails and BIG-IQ returns the following error in /var/log/restjavad.0.log.

"...Failed to run pre upgrade hook due to 'mount: /dev/loop0 is write-protected, mounting read-only
mount: /dev/loop1 is write-protected, mounting read-only
Querying for em_rest_java RPM file in ISO failed."

Conditions:
Installation of BIG IQ 8.0.0 EHF build fails if EHF ISO does not have em_rest_java RPM as part of ISO.

Impact:
Installation fails.

Workaround:
N/A

Fix:
This issue is now fixed and package validation is done as per expectations.

• The F5 Networks Technical Support web site: http://www.f5.com/support/
• The AskF5 web site: https://support.f5.com/csp/#/home
• The F5 DevCentral web site: http://devcentral.f5.com/

Generated: Mon Oct 11 08:30:35 2021 PDT

Copyright F5 Networks (2021) - All Rights Reserved