Applies To:
Show Versions
F5OS-A
- 1.4.0
F5OS-A Release Information
Version: 1.4.0
Build: 10698
Note: This content is current as of the software release date
Updates to bug information occur periodically. For the most up-to-date bug data, see Bug Tracker.
The blue background highlights fixes |
Known Issues in F5OS-A v1.4.x
Vulnerability Fixes
ID Number | CVE | Links to More Info | Description |
1141801-3 | CVE-2021-33060 | K12055286 | F5OS-A Intel CPU vulnerability CVE-2021-33060 |
1055789-2 | CVE-2021-40438 | K01552024 | Apache vulnerability CVE-2021-40438 |
Functional Change Fixes
ID Number | Severity | Links to More Info | Description |
1161557-1 | 1-Blocking | BT1161557 | BIG-IP tenants created before F5OS-C 1.5.1 or F5OS-A 1.3.0 may be allocated a smaller disk than required |
F5OS-A Fixes
ID Number | Severity | Links to More Info | Description |
1185369 | 1-Blocking | F5OS rSeries appliances will not launch tenants after upgrade to F5OS-A 1.3.0 | |
1148097 | 1-Blocking | rSeries r2xxx/r4xxx support for configuration of MAC block size per tenant | |
1135125-1 | 1-Blocking | Reading data from wrong socket leads to LACPD restart. | |
1252377-1 | 2-Critical | VXLAN-GPE and GENEVE are disabled by default when software is upgraded from F5OS-A 1.3.0 build to F5OS-A 1.4.0★ | |
1215917-1 | 2-Critical | webUI failed to load when downgrading from 1.4.0 to 1.3.1 with self-signed certificate with encrypted RSA key type | |
1211025-1 | 2-Critical | Firmware update interrupted during OS install★ | |
1210325 | 2-Critical | sys-host-config file size growing | |
1207741 | 2-Critical | LLDP crash when an LLDP interface is added or deleted | |
1204481-2 | 2-Critical | K000132166 | System may flap external links multiple times during startup or links may fail to come up at all |
1203641 | 2-Critical | STPD memory usage is increasing linearly | |
1196085 | 2-Critical | K000132166 | Disabling and re-enabling a port on rSeries can leave the port in a DOWN state |
1196073 | 2-Critical | K000132166 | Front panel port initialization failures can leave a port permanently DOWN |
1194537 | 2-Critical | Lacpd restarting due to incorrect interface name | |
1194261 | 2-Critical | CVE-2022-37601 - Prototype pollution vulnerability in function parseQuery in parseQuery.js in webpack | |
1194257 | 2-Critical | Cleartext Transmission of Sensitive Information in moment-timezone | |
1194253 | 2-Critical | Command Injection in moment-timezone before 0.5.35 | |
1194249 | 2-Critical | CVE-2022-29078 - The ejs (aka Embedded JavaScript templates) package 3.1.6 for Node.js | |
1194245 | 2-Critical | Arbitrary Code Injection vulnerability was found in ejs before 3.1.6. | |
1194073 | 2-Critical | After upgrade, trying to configure allowed-ip entry without and IP address causes system control to repeatedly crash and dump core | |
1190969 | 2-Critical | Memory leak in system-image-agent service | |
1185577 | 2-Critical | F5OS-A memory leak in ImageAgent process on rSeries hosts may affect tenant performance or lead to unexpected restarts of tenant or host | |
1173853-1 | 2-Critical | BT1173853 | Packet loss caused by failure of internal hardware bus |
1169365 | 2-Critical | Utils-agent coredump on file transfer | |
1169341-2 | 2-Critical | BT1169341 | Using MAC Masquerade in a BIG-IP tenant causes traffic issues when re-deploying the tenant |
1166277-1 | 2-Critical | System downgrade is not possible with tenants in deployed state. | |
1162609-1 | 2-Critical | F5 r2600/r2800/r4600/r4800 devices unable to establish LACP link or send LLDP to some switches | |
1145753-2 | 2-Critical | QKView obfuscation step can cause excessive disk usage | |
1144401-1 | 2-Critical | F5OS-A kubectl/docker related information missing in qkview | |
1141577-1 | 2-Critical | WebUI crashes when a new SSL/TLS private key is generated | |
1141137-1 | 2-Critical | BT1141137 | Qkview collects redundant log files |
1137121-2 | 2-Critical | Tenants are stuck in Pending state with status 0/1 nodes available after upgrading to F5OS-A 1.2.0 | |
1136597-2 | 2-Critical | LDAP user with admin and operator role gets only operator permissions | |
1136361-1 | 2-Critical | RJ45 interface links once at 1G | |
1135849 | 2-Critical | telemetry.db grew to 50G and caused error "database disk image is malformed" | |
1132733-2 | 2-Critical | BT1132733 | LDAP config tried to configure blank bind password |
1131993-1 | 2-Critical | Not able to set severity from CLI/webUI for some services. | |
1128765-2 | 2-Critical | BT1128765 | Data Mover lock-up causes major application traffic impact and tenant deploy failures |
1109525-2 | 2-Critical | K3s cluster is unhealthy when the system date or time is changed | |
1096729-1 | 2-Critical | BT1096729 | IP Fragments are disaggragated incorrectly |
1085925-2 | 2-Critical | SSH connection cannot be allowed/blocked based on source IP address | |
1055481 | 2-Critical | CVE-2021-39275 Buffer overrun in ap_escape_quotes | |
986773-2 | 3-Major | BT986773 | Disabling appliance mode without ConfD running |
974293 | 3-Major | Qkview availability | |
945537-3 | 3-Major | BT945537 | STP Validation for forward-delay, max-age, and hello-time fields |
1253713-2 | 3-Major | CVE-2020-15999 freetype: Heap-based buffer overflow due to integer truncation in Load_SBit_Png | |
1234049-1 | 3-Major | The Add/Edit tenant deployment screen on the r4600 webUI does not have the option for 12 vCPUs in the vCPUs dropdown | |
1230609-1 | 3-Major | Neighbor interface description is not updated in LLDP neighbor details | |
1229465 | 3-Major | QKView is not collecting core files in /var/crash | |
1226429-1 | 3-Major | "DEBUG cannot reply twice on the same call" log reporting repeatedly | |
1208825 | 3-Major | The default value of virtual disk size is 77GB and user is not allowed to have a tenant with disk size smaller than 77GB on the webUI | |
1205453 | 3-Major | Http-server package update to fix CVE-2019-0217 | |
1195993 | 3-Major | The 'docker pull' fails if docker registry is not set up properly | |
1195261 | 3-Major | Occasionally Selinux modules are getting corrupted when the system reboots | |
1194193 | 3-Major | sys-host-config is not properly handling the DNS search list | |
1190985-2 | 3-Major | WebUI server error when opening entry for added NTP server created with FQDN | |
1188821 | 3-Major | FIPS partition configuration is not removed even after the chassis partition is detached from tenant | |
1188265 | 3-Major | Unsupported PSU installed in rSeries system not reported as unsupported | |
1188141-1 | 3-Major | Tenant launch gets stuck due to un-initialization of VFs under one or more PF | |
1188101-1 | 3-Major | Incorrect LCD-UI after upgrade to 1.3.1 | |
1187189 | 3-Major | BT1187189 | Tenants fail to start after bare metal install |
1186173-1 | 3-Major | BT1186173 | Radius server secret-key should not be empty |
1186161-1 | 3-Major | BT1186161 | Radius server secret key should not be empty |
1186105 | 3-Major | K000132166 | rSeries logs multiple UP/DOWN link transitions during system start up. |
1186101 | 3-Major | K000132166 | Front panel interfaces are not disabled on system reboot |
1185557 | 3-Major | Upgrading to F5OS-A v1.3.1 requires increasing the minimum tenant Virtual Disk Size to 77GB from 76GB of to allow editing of existing tenants in the webUI | |
1184529 | 3-Major | BT1184529 | Intermittent ingress broadcast traffic failure for tenants on shared VLAN |
1183693 | 3-Major | Platform log has a high number of "Error" and "Fail" entries from diagnostic agent | |
1183489 | 3-Major | System generated events will be logged in platform.log, which is sent to remote logging | |
1183337 | 3-Major | The tmstat-agent and tmstat-merged error log messages | |
1182569 | 3-Major | After failed partial F5OS upgrade, system-manager will not start★ | |
1181721-1 | 3-Major | Add additional commands and files to QKView collection | |
1173653 | 3-Major | FIPS tenant table not able to display PCI_Device_ID, Partition_ID, Transaction_Count details | |
1167665 | 3-Major | CVE-2022-40674 - libexpat before 2.4.9 has a use-after-free | |
1166905-1 | 3-Major | Port speed is configurable from restconf for data port interfaces on Appliance devices. | |
1166201 | 3-Major | Opensource Updates | |
1165973-1 | 3-Major | Application error while using the CLI command "show components" | |
1161333 | 3-Major | K3S events logged even when K3S is in steady state on R2800/R4800/R5K/R10K platforms | |
1154573-1 | 3-Major | The "hdp_dmq_stat" table is missing data for several statistics | |
1154129-1 | 3-Major | Missing port-speed option for management interface on Appliance | |
1146181-1 | 3-Major | User logon/logoff logs in audit logs, to be sent via remote syslog | |
1143841-1 | 3-Major | BT1143841 | TACACS+ remote authentication for SSH does not work when server listens on non-default port |
1141661-1 | 3-Major | LDAP groups configurable with custom gidNumber to role mappings | |
1141293-1 | 3-Major | BT1141293 | F5OS will not import system images copied with WinSCP |
1140537-1 | 3-Major | BT1140537 | DMA-Agent system logs preserved through system reboots |
1138217 | 3-Major | "Allow IP" rule name does not have any length limit | |
1137669-1 | 3-Major | BT1137669 | Potential mis-forwarding of packets caused by stale internal hardware acceleration configuration |
1137637-1 | 3-Major | System is not configured to use user-specified NTP servers by default | |
1137601-1 | 3-Major | BT1137601 | Convey warning to user when user tries to change root user password with appliance mode enabled |
1137341 | 3-Major | LDAPS server group and StartTLS should be mutually exclusive | |
1137333-1 | 3-Major | Help text for LDAP TLS certificate check has been clarified | |
1136633-1 | 3-Major | Utils-agent "Failed to delete inactive download sessions" error on startup | |
1135865-2 | 3-Major | Remotely authenticated user who is a member of multiple roles that include invalid roles is not allowed to log in | |
1135861-2 | 3-Major | LDAP authentication mishandling | |
1135109-1 | 3-Major | AAA server group name and type are not displayed on ConfD | |
1134141-1 | 3-Major | Uploading qkview to iHealth may fail on long iHealth user names | |
1132973-2 | 3-Major | Live upgrade to F5OS-A-1.3.0 will not work if STP is not configured correctly. | |
1132473-1 | 3-Major | VELOS shows in the wording for "show services service " table on rSeries | |
1126677-1 | 3-Major | Inconsistencies with time zones displayed in controller and log files | |
1117645 | 3-Major | Customer security policy requires disabling basic authentication | |
1117577-1 | 3-Major | Management interface is not accessible if core system daemons are not running | |
1114405 | 3-Major | Currently allowed-ip profile name is in string format. There is no restriction while configuring profile name. | |
1107613-1 | 3-Major | BT1107613 | Enhance the LACP LAG data shown under the interface to take into account lacp state of the LACP LAG member |
1106881-4 | 3-Major | BT1106881 | F5OS with an AFM license provisioned may provide incorrect AFM stats to a BIG-IP tenant |
1102497 | 3-Major | Allow for encrypted key with passphrase | |
1086749-3 | 3-Major | Interface speeds are not reported correctly when linked at a slower speed | |
1084153-1 | 3-Major | Tenant deployment will fail when we move tenant (deployed with max vCPU) from provisioned to deployed | |
1080041 | 3-Major | Newly installed license is getting replaced with old license after performing config-restore | |
1075361-8 | 3-Major | Messages log has a very high number of "error" and "fail" entries | |
1062129-1 | 3-Major | Tenants are in pending state forever. | |
1044645-1 | 3-Major | openssl: Read buffer overruns processing ASN.1 strings | |
1232309-1 | 4-Minor | CVE-2020-10754: nmcli did not honour 802-1x.ca-path and 802-1x.phase2-ca-path settings | |
1225981 | 4-Minor | Files greater then 1000 MiB are truncated in QKView | |
1211861-2 | 4-Minor | BT1211861 | Configured input values of IP address fields reset to default upon switching the protocol |
1211777-2 | 4-Minor | BT1211777 | Configured input values of IP address fields reset to default upon switching the protocol |
1205485 | 4-Minor | Http-server package update to fix CVE-2017-15710. | |
1205481 | 4-Minor | Http-server package update to fix CVE-2017-15715 | |
1205477 | 4-Minor | Http-server package update to fix CVE-2018-1301. | |
1205461 | 4-Minor | Http-server package update to fix CVE-2018-1303. | |
1205457 | 4-Minor | Http-server package update to fix CVE-2018-1312 | |
1205441 | 4-Minor | Http-server package update to fix CVE-2019-0220. | |
1205401 | 4-Minor | Http-server package update to fix CVE-2019-10092. | |
1205393 | 4-Minor | CVE-2019-10098: mod_rewrite redirects may be sent to unexpected URL | |
1200973-1 | 4-Minor | Apache HTTPD vulnerability CVE-2020-1927 | |
1188761 | 4-Minor | Updates to openssl to resolve CVE-2019-1563 and CVE-2019-1547 | |
1188469 | 4-Minor | Updates to Openssl to resolve CVE-2020-1968 | |
1184821-1 | 4-Minor | Obscure crash in external authenticator | |
1167761 | 4-Minor | Directory Indexing enabled for management webUI | |
1156113-1 | 4-Minor | Appliance OMD repeatedly logs an obtuse error message every 10 seconds | |
1154625-1 | 4-Minor | BT1154625 | The Tenant Deployments column on the tenant images screen is not reflecting exact purpose of that column |
1137841 | 4-Minor | Configuring auth server-group and server requires duplicate name/address values | |
1137725-2 | 4-Minor | nslcd start/run script may fail or log alarming messages | |
1137689 | 4-Minor | BT1137689 | iHealth accepts QKView files to upload without any file extension |
1134657-1 | 4-Minor | USB information not available in QKView | |
1122829-1 | 4-Minor | Bash history does not include timestamps for commands | |
1108509-1 | 4-Minor | Unable to fetch appliance fan speed using SNMP | |
1137361-2 | 5-Cosmetic | Enabling LDAP may produce a log message with the usage help for the kill command |
Cumulative fix details for F5OS-A v1.4.0 that are included in this release
986773-2 : Disabling appliance mode without ConfD running
Links to More Info: BT986773
Component: F5OS-A
Symptoms:
When appliance mode is enabled (default behavior with setip wizard) and if CC ConfD does not come up (due to hardware or software bug, for example), there is no mechanism (other than to reimage the chassis) to disable appliance mode and recover the system.
Conditions:
Appliance mode is enabled (default behavior with setip wizard) and CC ConfD does not come up.
Impact:
There is no mechanism (other than to reimage the chassis) to disable appliance mode and recover the system.
Fix:
Appliance mode can be disabled via SSH command and serial console login scripts.
974293 : Qkview availability
Component: F5OS-A
Symptoms:
Under certain conditions, the qkview container stops responding.
Conditions:
Unspecified conditions.
Impact:
No qkview functionality.
Workaround:
N/A
945537-3 : STP Validation for forward-delay, max-age, and hello-time fields
Links to More Info: BT945537
Component: F5OS-A
Symptoms:
One or more forwarding-delay, max-age, or hello-time fields are configured and are not mirrored as operational data, or
One or more forwarding-delay, max-age, or hello-time fields are configured, and the configuration is not reflected in the spanning-tree BPDUs.
Conditions:
When configuring STP, use this formula for the forwarding-delay, max-age, and hello-time fields for STP, RSTP, and MSTP configurations:
2 * (hello-time + 1)) <= max-age && max-age <= (2 * (forwarding-delay - 1
Impact:
Any configuration that does not match the expected formula will not propagate to spanning tree BPDUs.
Workaround:
Configure the forward-delay, max-age, and hello-time fields using this formula:
2 * (hello-time + 1)) <= max-age && max-age <= (2 * (forwarding-delay - 1
Fix:
Fixed an issue where a user could configure the forward-delay, max-age, and hello-time fields for STP so that the expected formula was not met. Entering an invalid configuration displays an error.
1253713-2 : CVE-2020-15999 freetype: Heap-based buffer overflow due to integer truncation in Load_SBit_Png
Component: F5OS-A
Symptoms:
freetype processes PNG images embedded into fonts. A crafted TTF file can lead to heap-based buffer overflow due to integer truncation in Load_SBit_Png function.
Conditions:
N/A
Impact:
N/A
Workaround:
N/A
Fix:
freetype has been updated to 2.8-14.el7_9.1. to resolve the issue.
1252377-1 : VXLAN-GPE and GENEVE are disabled by default when software is upgraded from F5OS-A 1.3.0 build to F5OS-A 1.4.0★
Component: F5OS-A
Symptoms:
When r10000 or r5000 Series hardware is running with F5OS-A 1.3.0, the default settings for VXLAN-GPE and GENEVE are enabled, and hardware disaggregation support for these tunnel protocols is enabled without any explicit configuration.
If the software is upgraded from F5OS-A 1.3.0 to F5OS-A 1.4.0, these protocols will be disabled, and hardware disaggregation is disabled. It is required to enable these two protocols explicitly in the configuration to enabled them in the hardware.
Conditions:
If VXLAN-GPE and GENEVE tunnels are used in the deployment with F5OS-A 1.3.0 software version without any explicit enabled configuration for these two tunnels, and software upgraded to F5OS-A 1.4.0 or later.
Impact:
Hardware disaggregation support for VXLAN-GPE and GENEVE will be disabled if software is upgraded from F5OS-A 1.3.0 to F5OS-A 1.4.0 or later when these two tunnels are using default configuration to enable them.
Workaround:
Use explicit tunnel settings to enable VXLAN-GPE and GENEVE in F5OS-A 1.3.0, or enable these two protocols explicitly after software upgrade from F5OS-A 1.3.0.
Fix:
VXLAN-GPE and GENEVE are disabled in default global configuration and advised to use explicit tunnel configuration settings to enable hardware disaggregation support.
1234049-1 : The Add/Edit tenant deployment screen on the r4600 webUI does not have the option for 12 vCPUs in the vCPUs dropdown
Component: F5OS-A
Symptoms:
The vCPUs dropdown does not have 12 as an option in the Add/Edit tenant deployment screen on the r4600 webUI.
Conditions:
While adding or editing a tenant on the r4600 system via webUI.
Impact:
The user cannot add or edit a tenant with 12 vCPU cores on the webUI.
Workaround:
Users can add/edit a tenant with 12 vCPU cores from the CLI.
Fix:
The webUI will have an additional option for '12' in the vCPUs dropdown thus allowing the user to deploy a tenant with 12 vCPU cores.
1232309-1 : CVE-2020-10754: nmcli did not honour 802-1x.ca-path and 802-1x.phase2-ca-path settings
Component: F5OS-A
Symptoms:
It was found that nmcli, a command line interface to NetworkManager did not honour 802-1x.ca-path and 802-1x.phase2-ca-path settings, when creating a new profile. When a user connects to a network using this profile, the authentication does not happen and the connection is made insecurely.
Conditions:
N/A
Impact:
N/A
Fix:
The networkmanager and dependent packages are upgraded to NetworkManager-team-1.18.8.
1230609-1 : Neighbor interface description is not updated in LLDP neighbor details
Component: F5OS-A
Symptoms:
Port Description TLV is not displayed under LLDP interface neighbors.
Conditions:
1) enable LLDP on device and on switch
2) enable port description TLV
3) set port description on interface in switch side
Impact:
No impact.
Workaround:
N/A
Fix:
Fixed code to display port description.
1229465 : QKView is not collecting core files in /var/crash
Component: F5OS-A
Symptoms:
QKView was designed to collect core files in /var/core only. The operating system kernel can create core files in /var/crash. SEs need to know about these files.
Conditions:
OS kernel creates a core file.
Impact:
Core file not collected by QKView.
Workaround:
Core file can be manually copied from /var/crash.
Fix:
QKView takes a directory listing from /var/crash and collects core files in that directory.
1226429-1 : "DEBUG cannot reply twice on the same call" log reporting repeatedly
Component: F5OS-A
Symptoms:
When the snmpget operation is performed on IF-MIB, the message "DEBUG cannot reply twice on the same call" appears in /var/log/message. The issue is that the DEBUG is enabled in one of the services container so this DEBUG message is logging in /var/log/message.
Conditions:
One of the reproduction steps is to perform the snmpget operation on IF-MIB.
Impact:
No known impact on the functionality. They are DEBUG messages only.
Workaround:
No workaround. The debug messages stops when the snmpget operation is completed.
Fix:
Removed unwanted debug enable from the service container.
1225981 : Files greater then 1000 MiB are truncated in QKView
Component: F5OS-A
Symptoms:
QKView is unable to collect an untrunucated platform.log file that has been rotated.
Conditions:
Rotated copy of the platform.log file is greater than 1000 MiB.
Impact:
Logs are not complete in QKView making it difficult to troubleshoot issues.
Workaround:
Collect the log files manually.
1215917-1 : webUI failed to load when downgrading from 1.4.0 to 1.3.1 with self-signed certificate with encrypted RSA key type
Component: F5OS-A
Symptoms:
webUI fails to load.
Conditions:
If the self-signed certificate is enabled with encrypted-RSA/ECDSA, and the system is downgraded to lower versions than 1.4.0
Impact:
webUI fails to load.
Workaround:
Remove the self-signed encrypted certificate before downgrading to lower versions.
Fix:
Added code changes to restrict the downgrade to lower versions if encrypted RSA/ECDSA certificate is available.
1211861-2 : Configured input values of IP address fields reset to default upon switching the protocol
Links to More Info: BT1211861
Component: F5OS-A
Symptoms:
IP address fields are reset to default values.
Conditions:
When the user changes the protocol and switches back to the previously selected protocol via the "Address" field on the webUI.
Impact:
Values of IP address fields are lost as they are reset to default values.
Workaround:
Users can cancel or navigate away from the screen. When they visit the management interface screen again, the configured values will appear.
Fix:
The "Address" field in the Management Interface section has been removed, and all the IPv4 and IPv6 address fields are always displayed, thereby eliminating the need to switch protocols.
We have added 'X' buttons adjacent to the address fields that can be leveraged to put in default values in case the user does not want to configure the fields for a particular protocol.
1211777-2 : Configured input values of IP address fields reset to default upon switching the protocol
Links to More Info: BT1211777
Component: F5OS-A
Symptoms:
IP address fields are reset to default values.
Conditions:
When the user changes the protocol and switches back to the previously selected protocol via the "Address" field on the webUI.
Impact:
Values of IP address fields are lost as they are reset to default values.
Workaround:
Users can cancel or navigate away from the screen. When they visit the management interface screen again, the configured values will appear.
Fix:
The "Address" field in the Management Interface section has been removed, and all the IPv4 and IPv6 address fields are always displayed, thereby eliminating the need to switch protocols.
We have added 'X' buttons adjacent to the address fields that can be leveraged to put in default values in case the user does not want to configure the fields for a particular protocol.
1211025-1 : Firmware update interrupted during OS install★
Component: F5OS-A
Symptoms:
Firmware update can be interrupted by docker container issues.
Conditions:
Random container issue restarts all containers.
Impact:
If firmware is being updated in that moment, the firmware update will fail and it could cause problems to normal system operation.
Fix:
Docker container failure handles routine checks if firmware is being updated and waits until the update is done before handling the failure.
1210325 : sys-host-config file size growing
Component: F5OS-A
Symptoms:
File /var/F5/system/sys-host-config size is growing.
Conditions:
This issue is applicable to Appliance platforms.
When F5OS-A is installed, /var/F5/system/sys-host-config size growth will be observed.
Impact:
/var/F5/system/sys-host-config file size will grow with time and will occupy disk space.
Workaround:
/var/F5/system/sys-host-config can be removed if file size is high.
Fix:
Fixed code to avoid writing to file.
1208825 : The default value of virtual disk size is 77GB and user is not allowed to have a tenant with disk size smaller than 77GB on the webUI
Component: F5OS-A
Symptoms:
Depending on the tenant image type, the virtual disk requirements vary. Although the user can make necessary changes if the required disk size is greater than 77GB, they cannot make it lesser than 77GB on the webUI.
Conditions:
Deploying a tenant with image types that have virtual disk size requirement lesser than 77GB, such as T1 and T2 type images.
Impact:
The tenant will be not deployed with the virtual disk size as required for the tenant.
Workaround:
Users can edit the tenant from the CLI and update the virtual disk size as required, and can manage this tenant from the CLI.
Fix:
With the fix, there will be no inline validation for lower or upper limit on the Virtual Disk Size input field on the webUI form, and the default value is set to 0. If and when the configuration is saved with the default value, it will scale up to the minimum default Virtual Disk space required for that specific image.
1207741 : LLDP crash when an LLDP interface is added or deleted
Component: F5OS-A
Symptoms:
LLDP process crashes and generates a core file.
Conditions:
Add/del interfaces to LLDP.
This is a rare scenario coming due to thread synchronization.
Impact:
No impact on the functionality as LLDP service will restart and will update details correctly as per current LLDP configuration.
Workaround:
NA
Fix:
Thread synchronization is done to avoid crash.
1205485 : Http-server package update to fix CVE-2017-15710.
Component: F5OS-A
Symptoms:
No Impact on F5OS-A as we are not using AuthLDAPCharsetConfig config.
Conditions:
If Http-server configured with AuthLDAPCharsetConfig, uses the Accept-Language header value to lookup the right charset encoding when verifying the user's credentials. If the header value is not present in the charset conversion table, a fallback mechanism is used to truncate it to a two characters value to allow a quick retry (for example, 'en-US' is truncated to 'en'). A header value of less than two characters forces an out of bound write of one NUL byte to a memory location that is not part of the string.
Impact:
No Impact on F5OS-A as we are not using AuthLDAPCharsetConfig config.
Fix:
Http-server has been updated to an unaffected version.
1205481 : Http-server package update to fix CVE-2017-15715
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to CVE-2017-15715
Conditions:
The expression specified in <FilesMatch> could match '$' to a newline character in a malicious filename, rather than matching only the end of the filename.
Impact:
This will impact where uploads of some files are externally blocked, but only by matching the trailing portion of the filename.
Workaround:
N/A
Fix:
Apache Http-server package is updated to fix the issue.
1205477 : Http-server package update to fix CVE-2018-1301.
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to CVE-2018-1301
Conditions:
A specially crafted request could have crashed the Apache HTTP Server, due to an out of bound access after a size limit is reached by reading the HTTP header.
Impact:
Http-server could crash.
Workaround:
Restrict access to the management port to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1205461 : Http-server package update to fix CVE-2018-1303.
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to CVE-2018-1303
Conditions:
N/A
Impact:
A specially crafted HTTP request header could have crashed the Apache HTTP Server prior to version 2.4.30 due to an out of bound read while preparing data to be cached in shared memory. It could be used as a Denial of Service attack against users of mod_cache_socache. The vulnerability is considered as low risk since mod_cache_socache is not widely used, mod_cache_disk is not concerned by this vulnerability.
Workaround:
Restrict access to the Management Interface via appropriate network controls and limit access to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1205457 : Http-server package update to fix CVE-2018-1312
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to CVE-2018-1312
Conditions:
when generating an HTTP Digest authentication challenge, the nonce sent to prevent reply attacks was not correctly generated using a pseudo-random seed. In a cluster of servers using a common Digest authentication configuration, HTTP requests could be replayed across servers by an attacker without detection.
Impact:
HTTP requests could be replayed across servers by an attacker without detection.
Workaround:
Restrict access to the Management Interface via appropriate network controls and limit access to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1205453 : Http-server package update to fix CVE-2019-0217
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to:
CVE-2019-0217
Conditions:
N/A
Impact:
It could allow a user with valid credentials to authenticate using another username, bypassing configured access control restrictions.
Workaround:
Restrict access to the Management Interface via appropriate network controls and limit access to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1205441 : Http-server package update to fix CVE-2019-0220.
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to:
CVE-2019-0220
Conditions:
When the path component of a request URL contains multiple consecutive slashes ('/'), directives such as LocationMatch and RewriteRule must account for duplicates in regular expressions while other aspects of the servers processing will implicitly collapse them.
Impact:
Http-server get crash.
Workaround:
This flaw can be mitigation by replacing multiple consecutive slashes, used in directives that match against the path component of the request URL with regular expressions.
Fix:
Apache Http-server package is updated to fix the issue.
1205401 : Http-server package update to fix CVE-2019-10092.
Component: F5OS-A
Symptoms:
Apache http-server packages used in F5OS-A 1.0.0, F5OS-A 1.2.0, F5OS-A 1.2.1 F5OS-A 1.3.0, F5OS-A 1.3.1 were potentially susceptible to:
CVE-2019-10092
Conditions:
A limited cross-site scripting issue was reported affecting the mod_proxy error page.
Impact:
An attacker could cause the link on the error page to be malformed and instead point to a page of their choice. This would only be exploitable where a server was set up with proxying enabled but was misconfigured in such a way that the Proxy Error page was displayed.
Workaround:
Restrict access to the Management Interface via appropriate network controls and limit access to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1205393 : CVE-2019-10098: mod_rewrite redirects may be sent to unexpected URL
Component: F5OS-A
Symptoms:
In Apache HTTP server 2.4.0 to 2.4.39, Redirects configured with mod_rewrite that were intended to be self-referential might be fooled by encoded newlines and redirect instead to an unexpected URL within the request URL.
Conditions:
N/A
Impact:
N/A
Workaround:
Restrict access to the Management Interface via appropriate network controls and limit access to trusted users.
Fix:
Apache Http-server package is updated to fix the issue.
1204481-2 : System may flap external links multiple times during startup or links may fail to come up at all
Links to More Info: K000132166
Component: F5OS-A
Symptoms:
When the system boots up, the interfaces may flap (go up and down) several times in quick succession before coming up and stabilizing.
In some cases, the interfaces fail to come up at all.
If a peer switch is configured to detect excessive link flaps, it may put the port in an err-disable state and prevent the link from coming up.
Conditions:
-- r5000 or r10000 Series appliance
Impact:
If the peer switch triggers a link-flap detection feature, the ports may remain offline until an administrator manually recovers the port on that switch.
Workaround:
There is no workaround for this issue on the rSeries appliance.
An administrator can mitigate this issue by doing one of the following:
- configuring the peer switch to automatically try to recover ports that are disabled for excessive link flapping
- increasing the number of link flaps required in a certain interval before the port is put in a disabled state
Fix:
Disable sending of remote-fault signaling to peer device while the system is booting up.
1203641 : STPD memory usage is increasing linearly
Component: F5OS-A
Symptoms:
Increase in memory utilization of the STPD process.
Conditions:
=> Enable STP.
=> Configure STP on interfaces.
=> Monitor memory utilization of STP process using Top Command.
Impact:
STPD's memory utilization increases over time, affecting the system's overall memory availability.
Workaround:
None
Fix:
Fixed by deleting the data packets (BPDUs) after they are transmitted.
1200973-1 : Apache HTTPD vulnerability CVE-2020-1927
Component: F5OS-A
Symptoms:
See: https://my.f5.com/manage/s/article/K23153696
Conditions:
See: https://my.f5.com/manage/s/article/K23153696
Impact:
See: https://my.f5.com/manage/s/article/K23153696
Workaround:
See: https://my.f5.com/manage/s/article/K23153696
Fix:
See: https://my.f5.com/manage/s/article/K23153696
1196085 : Disabling and re-enabling a port on rSeries can leave the port in a DOWN state
Links to More Info: K000132166
Component: F5OS-A
Symptoms:
Disabling a port and then re-enabling it can result in the port staying DOWN.
Conditions:
Port Disable followed by Port Enable. The condition is aggravated when the port "enable" follows the "disable" too quickly. For example port "enable" within 15 seconds of the port "disable".
Impact:
Port stays DOWN and traffic is impacted.
Workaround:
There is no guaranteed workaround.
Sometimes disabling/re-enabling the port on the other side will bring the port back up.
Also, waiting 30-45 seconds before re-enabling the port minimizes the risk of this issue occurring.
Fix:
Improve port "disable" such that subsequent port "enable" does not leave the port DOWN.
1196073 : Front panel port initialization failures can leave a port permanently DOWN
Links to More Info: K000132166
Component: F5OS-A
Symptoms:
Failure of port initialization during system start-up or as a result of port re-initialization (port disable/enable).
Conditions:
Front panel port being initialized.
Impact:
Link stays DOWN and traffic is disrupted.
Workaround:
1) Disable then enable the link on the rSeries device.
2) Disable then enable the link on the peer device.
Fix:
Correct error handling of port initialization failures.
1195993 : The 'docker pull' fails if docker registry is not set up properly
Component: F5OS-A
Symptoms:
The 'docker pull' returns an error if the docker registry is not configured properly which leads to the wrong tag determination of the k3s pod.
Conditions:
At startup, there is a race condition between sw-mgmt creating all the registries, and appliance-orchestration-manager starting up.
Impact:
The k3s pod do not start due to the wrong tag.
Workaround:
Update the k3s pod manifest file with appropriate tag and recreate the k3s failed pod using the command 'kubectl'.
Fix:
None
1195261 : Occasionally Selinux modules are getting corrupted when the system reboots
Component: F5OS-A
Symptoms:
In rSeries appliances, if Selinux modules are corrupted:
-> Virt-handler pod crashes continuously
-> Tenant will be in pending state
-> Semodule file size is 0 in dir "/etc/selinux/targeted/active/modules/400/"
-> No or missing files under /etc/selinux/targeted/active/modules/ dir
Conditions:
If interruption happens during Selinux modules building on system bootup, the interruption can be an abrupt power off.
Impact:
-> Virt-handler pod is crashes continuously.
-> Tenant functionality is impacted.
Workaround:
Execute below commands:
-> cp -r /usr/etc/selinux/targeted/active/modules/* /etc/selinux/targeted/active/modules/
-> semodule -B
Fix:
Issue is fixed in F5OS-A 1.4.0 release.
1194537 : Lacpd restarting due to incorrect interface name
Component: F5OS-A
Symptoms:
Lacpd process exited while creating the lag.
Conditions:
When LACP interface name is not configured as per standard.
Impact:
Lacpd restarting continuously.
Workaround:
No workaround.
Fix:
Fixed by adding validation for LACP interface name.
1194261 : CVE-2022-37601 - Prototype pollution vulnerability in function parseQuery in parseQuery.js in webpack
Component: F5OS-A
Symptoms:
Prototype pollution vulnerability in function parseQuery in parseQuery.js in webpack loader-utils 2.0.0 via the name variable in parseQuery.js.
Impact:
F5OS-A 1.4.0 is affected by CVE-2022-37601
Workaround:
N/A
Fix:
The library which is dependant on loader-utils package is updated so that the recommended version of loader-utils is used.
1194257 : Cleartext Transmission of Sensitive Information in moment-timezone
Component: F5OS-A
Symptoms:
Cleartext Transmission of Sensitive Information in moment-timezone. (GHSA-v78c-4p63-2j6c)
https://github.com/moment/moment-timezone/security/advisories/GHSA-v78c-4p63-2j6c
Conditions:
N/A
Impact:
N/A
Workaround:
NA
Fix:
The dependency moment-timezone causing the vulnerability is updated to recommended version.
1194253 : Command Injection in moment-timezone before 0.5.35
Component: F5OS-A
Symptoms:
Command Injection in moment-timezone before 0.5.35. (GHSA-56x4-j7p9-fcf9)
https://github.com/moment/moment-timezone/security/advisories/GHSA-56x4-j7p9-fcf9
Conditions:
N/A
Impact:
F5OS-A 1.4.0 may be affected by this issue
Workaround:
N/A
Fix:
moment-timezone library is updated to recommended version.
1194249 : CVE-2022-29078 - The ejs (aka Embedded JavaScript templates) package 3.1.6 for Node.js
Component: F5OS-A
Symptoms:
The ejs (aka Embedded JavaScript templates) package 3.1.6 for Node.js allows server-side template injection in settings[view options][outputFunctionName]. This is parsed as an internal option, and overwrites the outputFunctionName option with an arbitrary OS command (which is executed upon template compilation).
Conditions:
N/A
Impact:
F5OS-A 1.4.0 may be affected by CVE-2022-29078
Workaround:
N/A
Fix:
The library that is dependant on ejs is updated so the recommended version of ejs is used.
1194245 : Arbitrary Code Injection vulnerability was found in ejs before 3.1.6.
Component: F5OS-A
Symptoms:
Arbitrary Code Injection vulnerability was found in ejs before 3.1.6. Caused by filename which isn't sanitized for display.
https://github.com/mde/ejs/commit/abaee2be937236b1b8da9a1f55096c17dda905fd
Conditions:
N/A
Impact:
N/A
Workaround:
N/A
Fix:
The library that is dependant on ejs is updated so the recommended version of ejs is used.
1194193 : sys-host-config is not properly handling the DNS search list
Component: F5OS-A
Symptoms:
When restarted or reloaded, the system_manager container duplicate entries for the DNS search list are seen in the logs.
Conditions:
During restart or reload of system_manager container.
Impact:
DNS config search path will not work as expected.
Workaround:
N/A
Fix:
Addressed code changes to remove duplicate entries from DNS search list and fetch the latest data as user updated in the search list.
1194073 : After upgrade, trying to configure allowed-ip entry without and IP address causes system control to repeatedly crash and dump core
Component: F5OS-A
Symptoms:
This issue occurred when there was an allowed-IP config without the IP address.
Conditions:
Goes into a crash state when it goes for a reboot with no IP address.
Impact:
System control docker container will not come up properly.
It impacts the allowed IP feature and the features dependent on system control.
Workaround:
If it is running F5OS-A version F5OS-A 1.3.1 or F5OS-A 1.3.0., or any version less than 1.4.0.
Please make sure to have the allowed IP profiles with proper IP addresses; remove the allowed IP profile with no IP address.
This issue is fixed in 1.4.0.
Fix:
This issue is fixed in F5OS-A 1.4.0.
1190985-2 : WebUI server error when opening entry for added NTP server created with FQDN
Component: F5OS-A
Symptoms:
When the user creates an NTP server with FQDN, the NTP server data table on the time settings screen shows the resolved IP address instead of the FQDN. If the user clicks on the hyperlinked IP address in order to launch the edit screen for the NTP server, the webUI throws an error as a record with the IP address is not found.
Conditions:
For an NTP server created with FQDN.
Impact:
The edit screen for the NTP server does not launch.
Workaround:
If the user replaces the IP address in the browser URL with the FQDN of the NTP server, they are able to view the Edit screen and make the required changes.
Fix:
WebUI will list the FQDN in the data table instead of the IP address.
1190969 : Memory leak in system-image-agent service
Component: F5OS-A
Symptoms:
Memory usage by system-image-agent on the rSeries host F5OS-A operating system is sometimes higher than expected.
When larger than approximately 2GB (r2xxx/r4xxx) or 4GB (other rSeries), this may create enough memory pressure to affect scheduling of tenant vCPU, causing various tenant symptoms that indicate lower performance. These may include (list is not exhaustive):
- dropping sporadic packets
- tmm reporting clock advanced in /var/log/ltm logs
- cores of tenant daemons
- unexpected restart of tenants
- restart of F5OS-A processes
- sluggish manageability of tenant or rSeries host
When the hypervisor layer is nearly out of memory, the Linux kernel may trigger the out-of-memory killer which may terminate processes, including those that are tenants. If this happens then OOM-killer logs showing ImageAgent with high RSS (~500,000 or more) will be present in host QKView logs in:
Files > Log > messages logs in iHealth view of rSeries host qkview
qkview/subpackages/host-qkview/qkview/filesystem/var/log/messages
eg
kernel: xxxx invoked oom-killer: ...
kernel: [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
kernel: [ 4321] 0 4321 696934 512846 1111 126261 0 ImageAgent
This indicates ImageAgent uses 512846 4KB pages in resident memory and 126216 4KB pages of swap (so approximately 2GB of resident and 0.5GB of swap). If not leaking, it should be very small.
Conditions:
When system-image-agent service is idle, there is a periodic memory leak. Rate of leak increases with the repeated image management related operations.
Impact:
Poor performance or unstable tenants: possible restarts, including of host rSeries.
Workaround:
While there is no workaround, the issue can be mitigated. If the leaking ImageAgent process can be restarted before it gets too big, it should be possible to avoid symptoms. It is best to restart it before it reaches 1GB in resident memory use (RES or RSS, depending on utility).
On iHealth you can view this in a host QKView under Commands, open system_image_agent folder and click on top. Look at the value under RES column for a row with command of /confd/bin/ImageAgent
Restarting the process should not affect traffic service.
To restart the system image agent, log into the host rSeries system as root and run:
docker restart system_image_agent
(Note underscores, not hyphens)
After this, there will be various log messages from image-agent in /var/F5/system/log/platform.log:
image-agent[10]: priority="Notice" version=1.0 msgid=0x2001000000000001 msg="Image Agent starting". <---
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000005 msg="DB is not ready".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000005 msg="DB is not ready".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000006 msg="DB state monitor started".
image-agent[10]: priority="Info" version=1.0 msgid=0x2005000000000001 msg="Image file added" FILE="BIGIP-15.1.6.1-0.0.10.ALL-F5OS.qcow2.zip.bundle".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000003 msg="DB state is now Active". <---
Fix:
Leak scenarios have been fixed.
1188821 : FIPS partition configuration is not removed even after the chassis partition is detached from tenant
Component: F5OS-A
Symptoms:
FIPS partition configuration is not removed even after the FIPS partition is detached from the tenant.
Conditions:
If the tenant is created again with the same name.
Impact:
appliance-1# show tenants
tenants tenant synctenant1
state unit-key-hash wtlsoWPAXj/CxXQJleNTR4aYuaQ10qNulxBWnppQmEOebQLphkh6nPN1ogkFpSSsoiNdoy9zMzwfmV8cCeZWAQ==
state type BIG-IP
state image BIGIP-tmos-tier2-17.1.0-0.0.1574.ALL-F5OS.qcow2.zip.bundle
state mgmt-ip 10.238.160.66
state prefix-length 24
state gateway 10.238.160.254
state cryptos enabled
state vcpu-cores-per-node 30
state memory 108032
state storage size 82
state running-state deployed
state appliance-mode disabled
state status Running
state primary-slot 1
state image-version "BIG-IP 17.1.0 0.0.1574"
state mac-data base-mac 14:a9:d0:15:3d:24
state mac-data mac-pool-size 1
MAC
-------------------
14:a9:d0:15:3d:24
state cpu-allocations cpu-allocation 1
cpus [ 6 7 8 11 12 13 14 15 17 18 19 20 21 22 23 30 31 32 35 36 37 38 39 41 42 43 44 45 46 47 ]
state instances instance 1 synctenant1-1
instance-id 1
phase Running
creation-time 2022-11-11T06:39:22Z
ready-time 2022-11-11T06:39:49Z
status "Started tenant instance"
4. Check ENV details after removing FIPS partition from tenant
>> [root@appliance-1 IMAGES]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system kube-multus-ds-amd64-75ckg 1/1 Running 2 107m
kube-system kube-flannel-ds-t2llf 1/1 Running 2 107m
kube-system coredns-6786db45bd-5r2j4 1/1 Running 2 107m
kube-system klipper-lb-vknsd 2/2 Running 4 107m
kube-system local-path-provisioner-5d99c5c99b-9vpxr 1/1 Running 2 107m
kube-system pause-7b5dc98878-vwb7m 1/1 Running 2 107m
kube-system metrics-server-68fddb5cb7-bxdq4 1/1 Running 2 107m
kubevirt virt-operator-7b8b67d494-2c9nx 1/1 Running 2 106m
kubevirt virt-operator-7b8b67d494-rp9kv 1/1 Running 2 106m
kubevirt virt-controller-6b8c7d95dd-sjw8g 1/1 Running 2 105m
kubevirt virt-api-6f779c8f89-7vn2c 1/1 Running 2 104m
kubevirt virt-controller-6b8c7d95dd-9wbmf 1/1 Running 2 105m
kubevirt virt-handler-j5xfv 1/1 Running 2 106m
kubevirt virt-api-6f779c8f89-q2977 1/1 Running 2 104m
default virt-launcher-synctenant1-1-2cg6r 1/1 Running 0 2m20s
>> kubectl describe vmi synctenant1-1
Name: hnet-conf-synctenant1
Env:
Name: TMM_DESCSOCK_SVC_ID
Value: 10
Name: TENANT_ID
Value: 2
Name: KVM_OPERATION
Value: 4
Name: KVM_MEMORY
Value: 113279762432
Name: TENANT_OP
Value: BIGIP
Name: HA_IP
Value: BIGIP
Name: HA_MASK
Value: BIGIP
Name: FIPS_PARTITION >> we didnt attach FIPS partition to tenant, still its there in config
Value: syncpart1
Name: TENANT_NAME
Value: synctenant1
Name: TENANT_TYPE
Value: BIGIP
Name: TENANT_SIZE
Value: 82
Name: VLOG_INFO
Value: sevbound
Name: SLOT_NUM
Value: 1
Actual Results: FIPS partition configuration is not removing even after de-attach the FIPS partition from tenant
Expected Results: FIPS partition name should not display in describe command.
Workaround:
N/A
Fix:
Fips_partition is cleared when the tenant is deleted before the delete thread handles it.
1188761 : Updates to openssl to resolve CVE-2019-1563 and CVE-2019-1547
Component: F5OS-A
Symptoms:
For CVE-2019-1563 see https://my.f5.com/manage/s/article/K97324400
For CVE-2019-1547 see https://my.f5.com/manage/s/article/K73422160
Conditions:
N/A
Impact:
N/A
Workaround:
N/A
Fix:
N/A
1188469 : Updates to Openssl to resolve CVE-2020-1968
Component: F5OS-A
Symptoms:
The Raccoon attack exploits a flaw in the TLS specification which can lead to an attacker being able to compute the pre-master secret in connections which have used a Diffie-Hellman (DH) based ciphersuite. In such a case this would result in the attacker being able to eavesdrop on all encrypted communications sent over that TLS connection. The attack can only be exploited if an implementation re-uses a DH secret across multiple TLS connections. Note that this issue only impacts DH ciphersuites and not ECDH ciphersuites. This issue affects OpenSSL 1.0.2 which is out of support and no longer receiving public updates. OpenSSL 1.1.1 is not vulnerable to this issue. Fixed in OpenSSL 1.0.2w (Affected 1.0.2-1.0.2v).
Conditions:
Usage of ciphersuites with Diffie Hellman (DH) enabled. Ciphersuites using Elliptic Curve Diffie Hellman (ECDH).
Impact:
Potential confidentiality impact to the pre-master secret.
Workaround:
N/A
Fix:
Openssl has been updated to 1.0.2zc
1188265 : Unsupported PSU installed in rSeries system not reported as unsupported
Component: F5OS-A
Symptoms:
An unsupported PSU installed in an rSeries r5000/r10000 system may not be reported as unsupported.
The GE CSAC0250BZ PSU (from an iSeries i2000/i4000 system) is mechanically compatible with an rSeries r5000/r10000 system but is not a supported PSU. It does not provide sufficient power for an rSeries system. When mistakenly installed in an rSeries system, it is not properly identified and reported as unsupported.
Conditions:
A GE CSAC0250BZ PSU from an iSeries i2000/i4000 system is mistakenly installed in an rSeries r5000/r10000 system.
Impact:
The GE CSAC0250BZ PSU is not reported as unsupported in an rSeries r5000/r10000 system.
Workaround:
Remove the unsupported GE CSAC0250BZ PSU from the rSeries system and install a PSU supported by rSeries.
Fix:
The AOM on rSeries r5000/r10000 systems can now identify the GE CSAC0250BZ PSU and properly report it as unsupported.
1188141-1 : Tenant launch gets stuck due to un-initialization of VFs under one or more PF
Component: F5OS-A
Symptoms:
On r2x00/r4x00 based systems, tenant launch gets stuck with an error in ConfD tenant status leaf:
"error adding container to network \"sriov-net3-tenant1\": SRIOV-CNI failed to load netconf: LoadConf(): failed to get VF information: \"lstat /sys/bus/pci/devices/0000:ec:00.7/physfn/net: no such file or directory"
The VFs(aka, SR-IOV Based Virtual Functions) were not seen under a PF(aka, SR-IOV based Physical Function) when run following the command.
Command: `ip link show <PF>`
PF can be, `x557_1`, `x557_2`, `x557_3`, `x557_4`, `sfp_5`, `sfp_6`, `sfp_7`, `sfp_8`.
For example, the faulty PF(x557_4 in this case) has no VFs listed compared to the healthy PF(x557_1 in this case),
# ip link show x557_4
18: x557_4: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 14:a9:d0:01:56:8a brd ff:ff:ff:ff:ff:ff
# ip link show x557_1
15: x557_1: <NO-CARRIER,BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether 14:a9:d0:01:56:87 brd ff:ff:ff:ff:ff:ff
vf 2 MAC 00:00:00:00:11:02, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:11:02, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:11:02, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:11:02, spoof checking on, link-state auto, trust off
Conditions:
On r4x00 or r2x00 based systems:
1. ConfD tenant status leaf reports "LoadConf(): failed to get VF information".
2. The VFs were not created under one or more PFs.
3. One of the files from "x557_1", "x557_2", "x557_3", "x557_4", "sfp_5", "sfp_6", "sfp_7", "sfp_8" missed from "/sys/class/net" directory.
For suppose when x557_4 is a faulty PF(aka, SR-IOV based Physical Function), then `/sys/class/net` shouldn't list x557_4 in its directory.
[root@appliance-1 ~]# ls /sys/class/net/x557_4
ls: cannot access /sys/class/net/x557_4: No such file or directory
[root@appliance-1 ~]#
Impact:
Tenant launch will be unsuccessful and is not able to connect to the tenant console or over tenant's management connection.
Workaround:
Workaround #1
===============
1. Move the tenant(s)' running-state in ConfD to provisioned.
2. Run "/usr/omd/scripts/config_ice_vfs.sh" script when "/sys/class/net" starts to show missing PF from the list above.
3. Run "kubectl rollout restart daemonset kube-sriov-device-plugin-amd64 -n kube-system".
4. Move the tenant(s)' running-state in ConfD to deployed.
Workaround #2 (only when second step takes too long)
==================================================
1. From second step in Workaround #1, if the PF wasn't detected in "/sys/class/net" even after a 20 minute duration, reboot the host to trigger the device probing.
Fix:
The workarounds should fix the tenants' statuses and move them to a running state.
1188101-1 : Incorrect LCD-UI after upgrade to 1.3.1
Component: F5OS-A
Symptoms:
An upgrade to F5OS-A 1.3.1 entails LCD-UI upgrade. This is not reflected correctly in platform inventory where the version displayed is the one before LCD-UI upgrade.
Conditions:
The issue is seen when AFU triggers LCD-UI upgrade. No issue when LCD-UI is not upgraded.
Impact:
LCD-UI is upgraded successfully to the desired version but the version displayed as part of platform inventory ("show components component properties property") is not correct.
Workaround:
The next reboot will update platform inventory with the correct LCD-UI firmware version.
Fix:
N/A
1187189 : Tenants fail to start after bare metal install
Links to More Info: BT1187189
Component: F5OS-A
Symptoms:
If the system reboots without an ISO imported after bare metal install, it will try to set up the port 2000 registry as a mirror of itself, which is not correct and results in the registry being empty. Because of this, the tenants are not being deployed.
Conditions:
In bare metal install ISO is not imported.
Impact:
Tenants will not deploy.
Workaround:
Import the version of the ISO that matches the version of the image used to perform the bare metal installation.
Fix:
When patch release is being installed, the correct 'active version of services' is taken which makes the registry not empty. Hence the tenants will be deployed without any issue.
1186173-1 : Radius server secret-key should not be empty
Links to More Info: BT1186173
Component: F5OS-A
Symptoms:
On add server screen of radius server group, the secret key field is not a mandatory field, which allows the user to add a server without any secret key.
Conditions:
User created a server group of type radius and tries to add a server in that group.
Impact:
If no secret key is provided by user as the field is not mandatory, the timeout value is read as a secret key for server, which is not correct.
Workaround:
User can provide a secret key even though it is not mandatory on the webUI.
Fix:
Secret key field is made a mandatory field now, so the user must enter a secret key before saving the form.
1186161-1 : Radius server secret key should not be empty
Links to More Info: BT1186161
Component: F5OS-A
Symptoms:
When setting up a Radius server without specifying a secret key, an entry for the server is made that has a missing secret key, thus it creates an invalid configuration file.
Conditions:
Normal
Impact:
The system has an invalid configuration.
Workaround:
Don't make an entry without a secret key.
Fix:
In the fixed version, if no secret key is entered, then the configuration files are not generated.
1186105 : rSeries logs multiple UP/DOWN link transitions during system start up.
Links to More Info: K000132166
Component: F5OS-A
Symptoms:
As the rSeries platform starts up and initializes its front-panel interfaces, multiple UP/DOWN link transitions are logged.
Conditions:
rSeries system startup.
Impact:
Confusing log messages regarding link transitions.
Workaround:
None
Fix:
Improve logging so only one DOWN/UP transition is logged at start-up.
1186101 : Front panel interfaces are not disabled on system reboot
Links to More Info: K000132166
Component: F5OS-A
Symptoms:
Peer device will not see its links go DOWN until the system or blade starts to reboot.
Conditions:
-- r5000, r10000 series appliance
-- CX410 chassis
Impact:
Unwanted traffic could egress the system unexpectedly.
Workaround:
There is no workaround for this issue.
Fix:
Detect that the system is rebooting and proactively disable the front-panel interfaces.
1185577 : F5OS-A memory leak in ImageAgent process on rSeries hosts may affect tenant performance or lead to unexpected restarts of tenant or host
Component: F5OS-A
Symptoms:
A memory leak exists in the ImageAgent process on the F5OS-A host hypervisor layer of rSeries devices.
This process manages the software images on the system.
When larger than approximately 2GB, this may create enough memory pressure to affect scheduling of tenant vCPU causing various tenant symptoms that indicate lower performance. These may include (list is not exhaustive):
- dropping sporadic packets
- tmm reporting Clock advanced in /var/log/ltm logs
- cores of tenant daemons
- unexpected restart of tenants
- restart of F5OS-A processes
- sluggish manageability of tenant or rSeries host
When the hypervisor layer is nearly out of memory the Linux kernel may trigger the out of memory killer which may terminate processes including those that are tenants. If this happens then oom-killer logs showing ImageAgent with high rss (~500,000 or more) will be present in host qkview logs in:
Files > Log > messages logs in iHealth view of rSeries host qkview
qkview/subpackages/host-qkview/qkview/filesystem/var/log/messages
eg
kernel: xxxx invoked oom-killer: ...
kernel: [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name
kernel: [ 4321] 0 4321 696934 512846 1111 126261 0 ImageAgent
This indicates ImageAgent uses 512846 4KB pages in resident memory and 126216 4KB pages of swap. So about 2GB of resident and 0.5GB of swap. Typically it will be very small.
Conditions:
rSeries host running affected F5OS-A version before 1.4.0.
Install of software using PXE.
Impact:
Poor performance or unstable tenants: possible restarts, including of host rSeries.
Workaround:
While there is no workaround, the issue can be mitigated. If the leaking ImageAgent process can be restarted before it gets too big it should be possible to avoid symptoms. It would probably be best to restart it before it reaches 1 GB in resident memory use (RES or RSS depending on utility).
On iHealth you can view this in a host qkview under Commands, open system_image_agent folder and click on top. Look at the value under RES column for row with command of /confd/bin/ImageAgent
Restarting the process should not affect traffic service.
To restart the system image agent, log into the host rSeries system as root and run:
docker restart system_image_agent
(N.B. underscores, not hyphens)
After this, there will be various log messages from image-agent in /var/F5/system/log/platform.log:
image-agent[10]: priority="Notice" version=1.0 msgid=0x2001000000000001 msg="Image Agent starting". <---
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000005 msg="DB is not ready".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000005 msg="DB is not ready".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000006 msg="DB state monitor started".
image-agent[10]: priority="Info" version=1.0 msgid=0x2005000000000001 msg="Image file added" FILE="BIGIP-15.1.6.1-0.0.10.ALL-F5OS.qcow2.zip.bundle".
image-agent[10]: priority="Info" version=1.0 msgid=0x6602000000000003 msg="DB state is now Active". <---
1185557 : Upgrading to F5OS-A v1.3.1 requires increasing the minimum tenant Virtual Disk Size to 77GB from 76GB of to allow editing of existing tenants in the webUI
Component: F5OS-A
Symptoms:
After upgrading to F5OS-A v1.3.1 from an earlier version, when you attempt to edit the attributes and parameters of an existing tenant, the Save button on the screen will not become selectable.
Conditions:
Applies to upgrading from an earlier F5OS-A version to F5OS-A v1.3.1 and preexisting configured, provisioned, or deployed tenants are present.
Impact:
If the Virtual Disk Size for any of the preexisting tenants is not increased to a minimum Virtual Disk Size of 77GB you will be unable to edit and save the tenant configuration via the webUI.
Workaround:
Increase the minimum tenant Virtual Disk Size to 77GB on the Add/Edit Tenant screen in addition to any other configuration elements and the Save button will become enabled. Alternatively, tenants can be edited via the CLI interface.
Fix:
Subsequent versions of F5OS will provide an inline validation warning that will be displayed near the Virtual Disk Size webUI element calling attention that the Virtual Disk Size minimum is insufficient if it is set to any value below 77GB.
1185369 : F5OS rSeries appliances will not launch tenants after upgrade to F5OS-A 1.3.0
Component: F5OS-A
Symptoms:
After an upgrade to F5OS-A 1.3.0, the system will not be able to deploy tenants. Even if the system software is reverted to the previous version, the issue remains.
The system may report a tenant status such as the following:
- Tenant deployment failed - Server is not responding
- 0/1 nodes are available: 1 node(s) didn't match Pod's node affinity/selector.
The "show cluster cluster-status" command will report that the cluster is not ready:
cluster cluster-status summary-status "1 Appliance is NOT ready, K3S cluster is NOT ready."
There will be error messages in /var/log/messages that mention "x509: certificate signed by unknown authority", for instance:
k3s: E1102 16:50:48.340717 44106 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5ba7aa29305335ce0b6a87b48a570b292f90e1f42f2a2b4ae4fff90a96a55df7\": Multus: [kube-system/klipper-lb-8cht8]: error getting pod: Get \"https://[100.75.0.1]:443/api/v1/namespaces/kube-system/pods/klipper-lb-8cht8?timeout=1m0s\": x509: certificate signed by unknown authority" pod="kube-system/klipper-lb-8cht8"
Conditions:
- F5OS rSeries appliance
- System upgraded to F5OS-A 1.3.0 for the first time
Impact:
The system is unable to deploy tenants. Even if the system is reverted to the previous software version, the issue remains and the system will be unable to launch tenants.
Workaround:
Once a system is affected, the fix is to reinstall the Kubernetes cluster. This procedure will take about 10 minutes and will not affect the configuration or data of the tenants.
1. Log in to the rSeries appliance CLI with the root account.
2. To identify if the setup is in an error state, check for the string “x509: certificate signed by unknown authority” in /var/log/messages, or K3S cluster is not healthy and running.
3. Change all deployed tenants to a provisioned state.
4. Stop the appliance_orchestration_manager service by running the following command:
systemctl stop appliance_orchestration_manager_container
5. Uninstall K3S by running the following commands:
k3s-uninstall.sh
rm /var/omd/* /tmp/omd/tokens/* /tmp/omd/appliance-ansible-host
6. Start the appliance_orchestration_manager service by running the following command:
systemctl start appliance_orchestration_manager_container
7. Wait about 10 minutes.
8. From the F5OS CLI (log in as admin), check the cluster status:
show cluster install-status ; show cluster cluster-status
The cluster-status should be "K3S cluster is initialized and ready for use".
From a root shell, check that "kubectl get pods -A" shows running containers in both the "kube-system" and "kubevirt" namespaces.
Fix:
N/A
1184821-1 : Obscure crash in external authenticator
Component: F5OS-A
Symptoms:
An unexpected sequence of characters in the username or password of an external login could cause a crash in the external authenticator.
Conditions:
Certain malformed usernames or passwords being used for external authentication.
Impact:
The crash in these circumstances would prevent successful login. After analysis, it was deemed there was no security risk or exposure.
Workaround:
Use usernames and passwords for authentication via SSH or webUI that conform to the device username/password requirements.
Fix:
The bug was fixed and a crash no longer occurs.
1184529 : Intermittent ingress broadcast traffic failure for tenants on shared VLAN
Links to More Info: BT1184529
Component: F5OS-A
Symptoms:
Traffic on the affected VLAN does not function correctly.
Affected tenants are able to respond to ARP requests or other broadcast traffic. Pings may intermittently fail.
On an affected F5OS rSeries appliance, even though the VLAN is shared between multiple tenants, the VLAN is missing from the software rebroadcaster, as observed by looking at:
docker exec system_tmstat_zmq tmctl -Sd blade rbcast_vlan_stat
Conditions:
-- Multiple tenants configured with access to the same VLAN.
-- The VLAN is assigned to multiple interfaces, and then removed from one interface.
Impact:
Traffic on the affected VLAN does not function correctly. Inbound broadcast traffic is not delivered to tenants.
Workaround:
If a system is already affected, deleting and re-adding the VLAN to an interface or trunk will resolve the issue.
Fix:
This issue no longer occurs.
1183693 : Platform log has a high number of "Error" and "Fail" entries from diagnostic agent
Component: F5OS-A
Symptoms:
While diagnostic agent service is getting started, it is logging some errors that are not having any impact.
Conditions:
Seen during diagnostic agent service start up.
Impact:
Logs will have more error or fail messages from diagnostic agent service at start up.
Fix:
Diagnostic agent service start up logs are cleaned.
1183489 : System generated events will be logged in platform.log, which is sent to remote logging
Component: F5OS-A
Symptoms:
All the system-generated events are displayed using "show system events" and these events were not sent to remote logging.
Conditions:
N/A
Impact:
None
Workaround:
All the system events are logged and displayed using "show system events" and the system alarms are displayed using "show system alarms".
Fix:
The system events are logged in platform.log. The platform.log has the capability to send it for remote logging.
1183337 : The tmstat-agent and tmstat-merged error log messages
Component: F5OS-A
Symptoms:
Both the tmstat-agent and tmstat-merged containers are logging few error messages that are not errors.
Conditions:
- On startup.
Impact:
Additional messages that are generated on startup, there is no functional impact to the customer.
Workaround:
Ignore the messages on startup.
Fix:
Suppressed internal conditions on startup that caused error messages to be logged. Removed unused configuration file entries and created directory that can also be created by other daemons.
1182569 : After failed partial F5OS upgrade, system-manager will not start★
Component: F5OS-A
Symptoms:
The primary system management daemon docker container (system_manager) is not running.
"docker logs system_manager" or the systemd journal or /var/log/message contains an error similar to the following:
"Bad configuration: cdb/confd.conf:677: Element snmpEngineID is invalid: \"80:00:2f:f4:03:Invalid\" is not a valid value.\n"
Conditions:
The F5OS-A services are upgraded to F5OS-A 1.2.0 while the OS remains at an earlier version. This might occur as a result of ID1181929: https://cdn.f5.com/product/bugtracker/ID1181929.html
Impact:
The system remains inoperative.
Workaround:
After the system F5OS OS and service versions match, remove /var/F5/system/cdb/engine-id and then reboot.
1181721-1 : Add additional commands and files to QKView collection
Component: F5OS-A
Symptoms:
There is no change in functionality. The fix adds new commands and files to QKView collection.
Conditions:
Additional commands and files are added to the QKView collection and they will be collected whenever QKView is requested.
Impact:
Additional commands and files are added to the QKView collection.
Workaround:
Only new commands and files will not be collected as part of QKView collection. Old commands and files will get collected in QKView.
Fix:
Additional commands and files are added to the QKView collection.
1173853-1 : Packet loss caused by failure of internal hardware bus
Links to More Info: BT1173853
Component: F5OS-A
Symptoms:
All or 50% of from-network packets arriving at a front panel port are dropped in hardware prior to delivery to tenant(s) running on the CPU. Packet loss is caused by CRC errors on an internal bus connecting two hardware components leading to eventual failure of the bus.
Conditions:
Issue occurs randomly, but is most commonly seen soon after bootup when packets first start to be handled by fastL4 hardware acceleration, hardware per-virtual server syn cookie protection, or AFM hardware protection.
Impact:
Total loss of from-network to CPU packets on r5900, r5800, and r5600 appliances, and either total loss or loss of 50% of from-network to CPU packets on r10900, r10800, and r10600 appliances. The r4800, r4600, r2800, and r2600 appliances are unaffected.
Workaround:
Reboot the appliance and disable fastL4 acceleration, per-virtual syn cookie hardware protection, and AFM hardware protection before re-enabling ingress traffic.
Fix:
This issue has been corrected.
1173653 : FIPS tenant table not able to display PCI_Device_ID, Partition_ID, Transaction_Count details
Component: F5OS-A
Symptoms:
FIPS tenant table is unable to display PCI_Device_ID, Partition_ID, or Transaction_Count details.
Conditions:
When a tenant is created with FIPS partition.
Impact:
appliance-1# show fips tenants
PCI
DEVICE PARTITION TRANSACTION
NAME PARTITION ID ID CNT
--------------------------------------------------------
fipstenant2 part2 - - -
Workaround:
N/A
Fix:
As a part of this fix, PCI device ID is displayed, and partition ID and transaction count are removed as they are not required.
1169365 : Utils-agent coredump on file transfer
Component: F5OS-A
Symptoms:
utils-agent coredump is generated under path /var/shared/core/container/. This is intermittent.
Conditions:
On file upload or download from appliances.
Impact:
File operations may not be complete. It wont block any functionality
Workaround:
Service utils-agent restart and operator able to perform file operations.
Fix:
curl global context initialisation and termination is done wrongly in utils-agent. Correctected in the latest builds.
1169341-2 : Using MAC Masquerade in a BIG-IP tenant causes traffic issues when re-deploying the tenant
Links to More Info: BT1169341
Component: F5OS-A
Symptoms:
If the tenant has configured MAC Masquerade, when the tenant is moved to a Configured or Provisioned state, then back to Deployed, the tenant may experience loss of traffic.
Conditions:
The tenant has configured MAC Masquerade and redeploys the tenant.
Impact:
The tenant may experience loss of datapath traffic.
Workaround:
N/A
Fix:
Using MAC Masquerade in a BIG-IP tenant no longer causes traffic issues.
1167761 : Directory Indexing enabled for management webUI
Component: F5OS-A
Symptoms:
Directory Indexing is enabled for management webUI.
Conditions:
When the management IP is followed by the name of any directory that is contained in the webUI, the build directories and file contents are visible on the browser.
Impact:
The webUI build directories and file contents are visible on the browser.
Workaround:
http-server config can be updated to disable directory indexing.
Steps:
1. Log in as root user into the system
2. Enter inside the http-server docker container and update the config file:
[root@appliance-1 ~]# docker exec -it http-server bash
bash-4.2# cd /etc/httpd/conf.d
bash-4.2# vi velocity.conf
Replace "Options Indexes FollowSymLinks"
with "Options FollowSymLinks"
Fix:
Disabled directory indexing.
1167665 : CVE-2022-40674 - libexpat before 2.4.9 has a use-after-free
Component: F5OS-A
Symptoms:
libexpat before 2.4.9 has a use-after-free in the doContent function in xmlparse.c.
Conditions:
N/A
Impact:
F5OS-A-1.4.0 is affected by CVE-2022-40674
Workaround:
Do not upload untrusted files onto the system.
Fix:
The expat rpm has been upgraded to expat-2.1.0 to resolve the issue.
1166905-1 : Port speed is configurable from restconf for data port interfaces on Appliance devices.
Component: F5OS-A
Symptoms:
User can set port speed for data ports using restconf.
This is applicable for only Appliance devices.
Conditions:
This is user configuration, and restconf access is required to configure the system.
Impact:
There is no impact, but the user is not advised to configure port speed as it is internal to the system.
Workaround:
N/A
Fix:
N/A
1166277-1 : System downgrade is not possible with tenants in deployed state.
Component: F5OS-A
Symptoms:
Tenants stuck in pending phase or tenant pods are missing.
Conditions:
Tenants deployed in 1.3.0 will be stuck in the pending phase when the system is downgraded to 1.2.0.
Impact:
Tenants will not be in a running state.
Workaround:
Move tenants to configured/provisioned state.
1166201 : Opensource Updates
Component: F5OS-A
Symptoms:
Opensource libraries used in previous versions were potentially susceptible to:
CVE-2016-4658
CVE-2017-18342
CVE-2018-25032
CVE-2019-15605
CVE-2019-17498
CVE-2019-20044
CVE-2020-10531
CVE-2020-12321
CVE-2020-24489
CVE-2020-25710
CVE-2020-8625
CVE-2021-20233
CVE-2021-20271
CVE-2021-2388
CVE-2021-25214
CVE-2021-25217
CVE-2021-27219
CVE-2021-27803
CVE-2021-30465
CVE-2021-3156
CVE-2021-3538
CVE-2021-3621
CVE-2021-4034
CVE-2021-42574
CVE-2021-43527
CVE-2021-44142
CVE-2022-1227
CVE-2022-1271
CVE-2022-23852
CVE-2022-24407
CVE-2022-24903
CVE-2022-2526
CVE-2022-2738
CVE-2022-29154
CVE-2022-34169
CVE-2022-40674
CVE-20919-8696
Conditions:
This addresses different problems. Multiple common vulnerabilities are fixed.
Impact:
Strengthens System Security
Fix:
Multiple common vulnerabilities are fixed to make system more secure.
RPM libraries have been upgraded to the following versions.
rpm-4.11.3-48.el7_9.x86_64
rpm-build-libs-4.11.3-48.el7_9.x86_64
rpm-libs-4.11.3-48.el7_9.x86_64
rpm-python-4.11.3-48.el7_9.x86_64
rsync-3.1.2-11.el7_9.x86_64
java-1.8.0-openjdk-headless-1:1.8.0.342.b07-1.el7_9.x86_64
tzdata-2022a-1.el7.noarch
tzdata-java-2022a-1.el7.noarch
systemd-219-78.el7_9.7.x86_64
systemd-libs-219-78.el7_9.7.x86_64
systemd-sysv-219-78.el7_9.7.x86_64
podman-1.6.4-36.el7_9.x86_64
runc-1.0.0-69.rc10.el7_9.x86_64
expat-2.1.0-15.el7_9.x86_64
1165973-1 : Application error while using the CLI command "show components"
Component: F5OS-A
Symptoms:
The user receives an error message using the CLI (show components -> Error: application error) when there is a faulty sensor in the hardware.
Conditions:
When the system has the faulty sensor.
Impact:
Application error seen in the ConfD CLI while trying to execute "show components". The webUI is affected as well.
Workaround:
N/A
Fix:
We have added a check at diag-agent to not throw the application error; it will show data for the healthy components.
1162609-1 : F5 r2600/r2800/r4600/r4800 devices unable to establish LACP link or send LLDP to some switches
Component: F5OS-A
Symptoms:
LACP and LLDP messages transmitted from an F5OS r2x00/r4x00 appliance to a peer switch have an incorrect length, and are ignored by some switches.
This can result in LACP aggregate links configured between an F5OS appliance and peer switch to fail to establish.
For example, Extreme Networks switches may produce a message similar to this:
<Erro:LACP.RxPDUSizExcd> Slot-2: Received PDU LACP size exceeded. Incoming Port: 1:1 PDU size: 132 required size: 128
Juniper hardware may produce messages similar to this:
kernel: xe-1/1/1: received pdu - length mismatch for lacp : len 128, pdu 124 like 1
LLDP packets sent from the F5OS device may not be accepted or correctly interpreted by the connected switch.
Conditions:
-- rSeries r2600/r2800/r4600/r4800-series appliance
-- LACP trunk (aggregate link) configured
(or)
-- LLDP advertising enabled
Impact:
Unable to establish an LACP trunk between the F5OS r2600/r2800/r4600/r4800 and a network switch.
Workaround:
Configure the LAG using a static configuration (that is, no LACP) on both sides, if possible.
Fix:
Fixed code to trim extra 4 bytes going in the BPDUs.
1161557-1 : BIG-IP tenants created before F5OS-C 1.5.1 or F5OS-A 1.3.0 may be allocated a smaller disk than required
Links to More Info: BT1161557
Component: F5OS-A
Symptoms:
If the BIG-IP tenant disk space is fully used by creating multiple software volumes within the tenant, it will generate disk errors.
Conditions:
- A tenant originally deployed from an “ALL-F5OS” tenant image (i.e., BIGIP-15.1.6.1-0.0.10.ALL-F5OS.qcow2.zip.bundle) originally created from one of the following:
-- 14.1.5 or above in the 14.1.x branch of code
-- 15.1.6.1 or above in the 15.1.x branch of code
- The tenant is configured to use 76G of disk space (the default)
Impact:
Software installs within the tenant may fail.
Workaround:
Beginning in F5OS-A 1.3.0, the system detects the minimum size of a disk created from a tenant image file, and enforces that minimum on newly-created tenants.
If a customer has a tenant affected by this issue and upgrades their system to F5OS-A 1.3.0 or later, set the tenant to "configured", and then deploy the tenant again.
If the disk size is not right, the system will show the minimum size, then adjust the tenant disk size to what is advised by the system or larger.
From 1.4.0, user does not need to adjust the size unless the user needs a bigger size.
The right/minimum size will be auto-allocated when the state is changed.
Fix:
The tenant disk size will be detected and auto-allocated.
Behavior Change:
There are two behaviors.
1.3.x: If the disk size is smaller than it has to be, it warns the user and doesn't start the tenant until the user specifies the right/minimum size.
1.4.0: It auto increases the size to the right/minimum size if the user didn't specify the disk size.
1161333 : K3S events logged even when K3S is in steady state on R2800/R4800/R5K/R10K platforms
Component: F5OS-A
Symptoms:
K3S events shall be logged into //var/F5/system/log/k3s_events.log even when K3S is in steady state on R2800/R4800/R5K/R10K platforms.
Conditions:
K3S events are logged even when K3S is in steady state.
Impact:
/var/F5/system/log/k3s_events.log file will be filled with all the events logged.
1156113-1 : Appliance OMD repeatedly logs an obtuse error message every 10 seconds
Component: F5OS-A
Symptoms:
When tenants fail to deploy/start for any reason, Appliance OMD would periodically log an obtuse error message every 10 seconds.
Appliance OMD is allowing and error message to be logged while the data is partially populated.
Conditions:
This issue is observed when tenants fail to deploy/start for any reason.
Impact:
Obtuse error logs are observed every 10 seconds.
Workaround:
Obtuse logging is fixed in the Appliance OMD to log the data only if the data is properly populated.
1154625-1 : The Tenant Deployments column on the tenant images screen is not reflecting exact purpose of that column
Links to More Info: BT1154625
Component: F5OS-A
Symptoms:
The Tenant Deployments column on the tenant images screen on the VELOS chassis partition webUI is currently showing the comma separated slot numbers on which the tenant is deployed. If the image is not used for any deployment, a string 'Not In Use' is shown.
Conditions:
There should be a deployed tenant using the tenant image we are viewing information for.
Impact:
The name "Tenant deployments" on the column is confusing the user, as it indicates that the column will show the count of tenants that are deployed using that image.
Workaround:
NA
Fix:
The column name is changed to "In Use" and the column will now show value "True" or "False" indicating if a tenant is deployed using that image.
1154573-1 : The "hdp_dmq_stat" table is missing data for several statistics
Component: F5OS-A
Symptoms:
The TMCTL "hdp_dmq_stat" table is missing data for the following counters:
hdp_dmq_stat/tpg_txpkts
hdp_dmq_stat/tpg_badifh_drop_cnt
hdp_dmq_stat/tpg_unsup_tag_drop_cnt
Conditions:
Always.
Impact:
No valid data for the affected counters.
Workaround:
None
Fix:
Correctly fill in the table columns with the counter values.
1154129-1 : Missing port-speed option for management interface on Appliance
Component: F5OS-A
Symptoms:
There is no option to change the port speed on the management interface of the Appliance through the CLI. An error displays when you attempt to disable auto-negotiation or when you try to change the port speed from the webUI (after disabling from
the CLI).
Conditions:
Always
Impact:
Port speed cannot be configured for the management interface on Appliance.
Workaround:
No workaround.
Fix:
Current schema changes allow port speed to be configured for management interface on Appliance.
1148097 : rSeries r2xxx/r4xxx support for configuration of MAC block size per tenant
Component: F5OS-A
Symptoms:
Prior to F5OS-A 1.4.0, r2xxx/r4xxx each BIG-IP tenant network interface uses the same MAC address.
Conditions:
For cases like inline L2, tenants require each BIG-IP tenant interface to have a unique MAC address.
Impact:
Cases like inline L2 require MAC Data/MAC Block Size
to be greater than one.
Workaround:
Upgrade to 1.4.
Fix:
When deploying a tenant, selecting MAC Data/MAC Block Size value of small represents a block of 8 MACs. When this value is used, the tenant gets a block of 8 contiguous MACs.
When using LAGs, tenants require that trust-mode is set to true
in order to adjust tenant interface MAC addresses.
1146181-1 : User logon/logoff logs in audit logs, to be sent via remote syslog
Component: F5OS-A
Symptoms:
The user logon/logoff logs were not sent via remote syslog.
Conditions:
Releases prior to version 1.6.0.
Impact:
The user logon and logoff logs will not be sent to remote syslog.
Workaround:
NA
Fix:
audit.log is included in remote syslog, so all the user logon and logoff logs can be sent to remote syslog.
1145753-2 : QKView obfuscation step can cause excessive disk usage
Component: F5OS-A
Symptoms:
QKView performs the obfuscation steps for capturing files, which can create temporary files the same size as the captured files. If a sufficiently large file is captured, this may cause a disk full error.
Conditions:
QKView captures a very large file and obfuscates it.
Impact:
System may be unusable.
Workaround:
Before executing QKView, scan the system for extraordinarily large log files and delete them. One example is telemetry.db.
Fix:
This bug fix truncates the file to a maximum size of 0.5 GB (or a size defined by the maxfilesize argument) before performing obfuscation. This limits the chance for a disk full error.
1144401-1 : F5OS-A kubectl/docker related information missing in qkview
Component: F5OS-A
Symptoms:
Kubectl/docker information on the system is not collected as part of qkview.
Conditions:
Kubectl/docker information on the system is missed in qkview whenever qkview is triggered on the system.
Impact:
Kubectl/docker information on the system is not collected as part of qkview.
Workaround:
No workaround.
Fix:
Kubectl/docker information on the system will be collected as part of qkview.
1143841-1 : TACACS+ remote authentication for SSH does not work when server listens on non-default port
Links to More Info: BT1143841
Component: F5OS-A
Symptoms:
If remote authentication is configured to use TACACS+ and the servers use a port other than 49 (the default port for TACACS), users will not be able to authenticate via SSH.
SELinux errors in /var/log/audit/audit.log similar to the following:
type=AVC msg=audit(1660923433.566:3728): avc: denied { name_connect } for pid=20995 comm="sshd" dest=4949 scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:munin_port_t:s0 tclass=tcp_socket permissive=0
Conditions:
-- rSeries appliance running F5OS-A, or VELOS system controller; this issue does not affect VELOS chassis partitions
-- TACACS+ remote authentication
-- TACACS+ server listening on a port other than 49
Impact:
Unable to authenticate when connecting via SSH.
Workaround:
Configure the SELinux policy to allow this traffic. On a VELOS chassis, these instructions must be run on each system controller separately.
1. Connect to the F5OS system via SSH as root.
2. Examine the SELinux audit denials and confirm all of them are associated with traffic that should be allowed:
grep denied.*name_connect.*sshd /var/log/audit/audit.log > /root/ssh-audit-denials.log
cat /root/ssh-audit-denials.log
Remove entries from the file /root/ssh-audit-denials.log that you do not want to allow.
3. After confirming the contents of the file /root/ssh-audit-denials.log, run the following commands to create and install an SELinux policy to allow that traffic:
audit2allow -M sshd.allowtacacs < /root/ssh-audit-denials.log
semodule -i sshd.allowtacacs.pp
1141801-3 : F5OS-A Intel CPU vulnerability CVE-2021-33060
Links to More Info: K12055286
1141661-1 : LDAP groups configurable with custom gidNumber to role mappings
Component: F5OS-A
Symptoms:
In prior releases, the group ID number representing authentication roles was hard-coded to certain values. This could cause problems since an external authentication system (for example, LDAP) may have conflicting group IDs.
Conditions:
External authentication system (e.g. LDAP, AD, or radius) where a group ID number conflicts with the hard-coded role IDs (for example, 9000).
Impact:
This could cause difficulty configuring a user with specific role assignments in an external authentication system.
Workaround:
Reconfigure group IDs in external system such that the hard-coded group ID numbers match the role numbers required by the F5 system.
Fix:
Added configuration to allow the administrator to specify the group ID number in use by the external system to identify user roles. The external number will be mapped to the F5 role based on this setting.
1141577-1 : WebUI crashes when a new SSL/TLS private key is generated
Component: F5OS-A
Symptoms:
The webUI crashes when a new SSL/TLS certificate is created in the Certificate Management tab.
The HTTP server has to restart to read the newly-created private keys (encrypted or un encrypted) from a configuration file. Before the HTTP server restarts, all active client connections will be closed. This will cause the webUI to crash, and the server will be unreachable temporarily.
Conditions:
No configuration changes required.
Impact:
The webUI crashes and the TCP connection with the HTTP server will be closed.
Workaround:
The user has to reestablish the connection to the server after waiting a few seconds.
Fix:
No fix required.
1141293-1 : F5OS will not import system images copied with WinSCP
Links to More Info: BT1141293
Component: F5OS-A
Symptoms:
F5OS will not import system images copied into /var/import/staging/ using WinSCP. The file will be present on the filesystem, but the system will not process and validate them.
On older software versions (prior to F5OS-C 1.3.0 and F5OS-A 1.1.0), the image will remain stuck in an "In Queue" state.
Conditions:
Importing F5OS system images (F5OS-C controller and chassis partition images and F5OS-A system images) to /var/import/staging/.
Impact:
The images cannot be used for F5OS software installs.
Workaround:
After importing the images, log in to the F5OS device as root and run touch against the newly-uploaded files. For instance:
touch /var/import/staging/F5OS-C-1.4.0-4112.CONTROLLER.iso
Fix:
F5OS will correctly import system images copied with WinSCP.
1141137-1 : Qkview collects redundant log files
Links to More Info: BT1141137
Component: F5OS-A
Symptoms:
Qkview collects most log files as part of its main collection, but some containers have been specified to collect log files specific to the operation of that container.
Conditions:
Execute
system diagnostics qkview capture
Impact:
Redundant log files collected use extra storage space and bandwidth for transmission.
Fix:
Redundant log files have been scrubbed from container collection.
1140537-1 : DMA-Agent system logs preserved through system reboots
Links to More Info: BT1140537
Component: F5OS-A
Symptoms:
The dma-agent log file is deleted and recreated every time the system is rebooted, this makes investigating dma-agent related issues difficult if the system had restarted since the problem occurred.
Conditions:
Accessing dma-agent system logs.
Impact:
Difficult in investigating or debugging dma-agent system logs.
Workaround:
Do not reboot the system in which dma-agent logs need to be investigated.
1138217 : "Allow IP" rule name does not have any length limit
Component: F5OS-A
Symptoms:
Allowed IP profile name does not have any length restrictions. A user is able to create a profile name with long strings.
Conditions:
When trying to create the allowed-ip rule and give the profile name more than 50 characters.
Impact:
confd need to store a rule name of any length in cdb.
1137841 : Configuring auth server-group and server requires duplicate name/address values
Component: F5OS-A
Symptoms:
When configuring a server-group from the CLI, users were forced to enter the config name and/or the config address multiple times to successfully configure the server-group.
Conditions:
User is attempting to configure an auth server-group using the CLI.
Impact:
Configuration was unnecessarily complex and error prone.
Workaround:
Explicitly enter the config name or config address to complete the configuration of the server-group.
Fix:
Server-group only requires the config name and/or config address to be entered once to successfully configure the object.
1137725-2 : nslcd start/run script may fail or log alarming messages
Component: F5OS-A
Symptoms:
The script that watches and restarts the nslcd process could sometimes fail to do so, and would sometimes log messages that appeared alarming.
Conditions:
Changing authentication settings that affect nslcd.
Impact:
The messages were benign, but the occasional failure to restart nslcd on config change could cause authentication changes to fail to propagate to the running process.
Workaround:
Restarting the name-service-ldap container is likely to solve the issue.
Fix:
The nslcd start/run script was rewritten to minimize alarming log messages and reliably start and restart the process when expected.
1137689 : iHealth accepts QKView files to upload without any file extension
Links to More Info: BT1137689
Component: F5OS-A
Symptoms:
QKView files without any extension failed to upload into iHealth.
Conditions:
If the QKView files are generated without any extension.
Impact:
iHealth report invalid file extension.
Workaround:
Generate QKView files with extension.
Fix:
Allow to upload QKView files to iHealth without any extension.
1137669-1 : Potential mis-forwarding of packets caused by stale internal hardware acceleration configuration
Links to More Info: BT1137669
Component: F5OS-A
Symptoms:
Because configuration entries added to the internal ePVA hardware acceleration tables may become stuck, packets arriving from front panel ports may be handled by stale entries resulting in unexpected forwarding behavior. The stale entries may also prevent TMM from offloading new connections to ePVA.
Conditions:
The most likely cause for entries to become stuck is either a reboot of tenant or restart of TMM while it has active connections offloaded to ePVA without also rebooting the entire appliance.
Impact:
Packets may be forwarded to unexpected destinations, and/or new connections are unable to be offloaded to ePVA.
Workaround:
Don't reboot or restart TMM without also rebooting the entire appliance.
Fix:
Packets are behaving as expected.
1137637-1 : System is not configured to use user-specified NTP servers by default
Component: F5OS-A
Symptoms:
The default configuration of a VELOS system is for external NTP to be disabled. This means that even if user-specified NTP servers are configured, they will not be used until the overall NTP feature is enabled. Additionally, the overall NTP enablement value is not reflected in the output of 'show running-config system ntp' in controller ConfD.
Conditions:
System is running with default NTP configuration and user configures an external NTP server.
Impact:
Behavior is confusing or concerning to users who do not realize that they need to enable NTP for their configuration to take effect.
Workaround:
To work around the issue, users must enable external NTP:
syscon-1-active(config)# system ntp config enabled
syscon-1-active(config)# commit
Commit complete.
Fix:
System is configured to use user-specified NTP servers by default, and reports NTP enablement value in running config.
1137601-1 : Convey warning to user when user tries to change root user password with appliance mode enabled
Links to More Info: BT1137601
Component: F5OS-A
Symptoms:
There are no issues in functionality. This is to show extra information whenever the user tries to change the root user password on Users screen on the webUI, and the appliance mode is enabled. The CLI shows a message, and webUI will also show the same message in the form of popup.
Conditions:
Whenever the user tries to change the root user password on the Users screen on the webUI while appliance mode is enabled, a warning popup shows up with the same information.
Impact:
This does not impact the functionality.
Fix:
Enable appliance mode and change the root user password on the Users screen, a popup shows up with the information "The password has changed but appliance mode is enabled that blocks root login."
1137361-2 : Enabling LDAP may produce a log message with the usage help for the kill command
Component: F5OS-A
Symptoms:
If the nslcd process is being restarted but was not previously running, this message could be issued.
Conditions:
The nslcd process is being restarted because of a configuration change but was not previously running.
Impact:
Alarming log messages. Potential failure to restart nslcd, resulting in failures in remote authentication.
Workaround:
Restarting the name-service-ldap container is likely to resolve the issue.
Fix:
The nslcd run/start script was rewritten to make it more robust, while reducing the chance for unnecessarily alarming log messages.
1137341 : LDAPS server group and StartTLS should be mutually exclusive
Component: F5OS-A
Symptoms:
If an LDAPS server group is specified with LDAP settings that specify StartTLS, LDAP authentication will not function.
Conditions:
LDAPS server group configured with LDAP settings specifying StartTLS.
Impact:
LDAP authentication will not function.
Workaround:
Do not configure an LDAP server group as type "LDAP over SSL" and also set the LDAP "TLS" setting to "StartTLS".
Set the LDAP server group to "LDAP over TCP" if the LDAP "TLS" setting is "StartTLS".
1137333-1 : Help text for LDAP TLS certificate check has been clarified
Component: F5OS-A
Symptoms:
The help text for LDAP tls_reqcert was not clear. This has been rectified.
appliance(config)# system aaa authentication ldap tls_reqcert
Possible completions:
allow Session proceeds with or without server certificate, including a bad one.
demand Session terminates immediately if a bad or no certificate is provided.
hard This keyword is equivalent and semantically the same as demand.
never The client will not request or check any server certificate.
try Equivalent to allow, but the session is terminated if a bad certificate is provided.
Impact:
Help text was confusing.
Fix:
Help text has been improved.
1137121-2 : Tenants are stuck in Pending state with status 0/1 nodes available after upgrading to F5OS-A 1.2.0
Component: F5OS-A
Symptoms:
The system is unable to start tenants, and the tenant reports a status of "Insufficient f5.com/qat".
Conditions:
Might occur after an F5OS-A software upgrade or after reinstalling K3s.
Impact:
Tenants will not start and are unusable.
Workaround:
To work around this issue, perform one of these actions:
1. Reboot the rSeries appliance.
or
2. Restart the qat-plugin process by logging into the appliance as root, and running "pkill qat-plugin".
Fix:
Fixed an issue with the qat-plugin process that prevented the system from starting tenants.
1136633-1 : Utils-agent "Failed to delete inactive download sessions" error on startup
Component: F5OS-A
Symptoms:
After booting a device, an error message of "Failed to download inactive download session" is observed in the error logs.
Conditions:
Immediately after booting.
Impact:
Error logs are observed after booting.
Workaround:
N/A
Fix:
The log is fixed in the utils-agent to log the data only if the data is not deleted properly.
1136597-2 : LDAP user with admin and operator role gets only operator permissions
Component: F5OS-A
Symptoms:
An LDAP user configured with groups for both admin and operator roles only receives operator permissions.
Conditions:
LDAP user configured with gidNumber assignments for both admin and operator roles.
Impact:
A user with this config would be assigned only operator permissions.
Workaround:
Only configure the gidNumber for the desired role in LDAP for the user. Do not configure multiple roles for the same user.
Fix:
There was an error in the NACM rules for ConfD config. The role logic has been fixed.
1136361-1 : RJ45 interface links once at 1G
Component: F5OS-A
Symptoms:
The RJ45 interfaces on F5 r2000 and r4000 platforms link at 1G only once. If the link goes down, the interfaces cannot reestablish a link at 1G.
Conditions:
When an RJ45 interface that is 10G/1G capable is connected to a 1G port on F5 r2000 and r4000 platforms.
Impact:
The RJ45 interface won't achieve a link.
Workaround:
To clear the no-link condition, reboot or power cycle the platform. The RJ45 link will then come up at 1G, but only once.
Fix:
The RJ45 interfaces on F5 r2000 and r4000 platforms are now able to re-establish a 1G link.
1135865-2 : Remotely authenticated user who is a member of multiple roles that include invalid roles is not allowed to log in
Component: F5OS-A
Symptoms:
Users on systems have a role assigned to them. This role is one of a predefined set which includes the admin role. A remote user with multiple roles, some of which are not in this predefined set, is configured on a remote authentication server (LDAP, tacplus or RADIUS). Such a user was treated different based on mode of access (GUI or ssh) and the remote authentication method. Sometimes the user can log in, sometimes not.
Conditions:
A user has to configured on a remote authentication server (LDAP, tacplus or RADIUS) with multiple group IDs, some of which are not assigned to any role in our system.
That remote authentication method has to be configured as an authentication method on our system.
User supplies the correct password and tries to log in. The user may or may not be allowed into the system, depending on method of access and remote authentication method.
Impact:
When a remote user has multiple roles which include invalid roles, the behavior of the system was inconsistent.
Workaround:
Removing the invalid group ID from the remote server will fix the issue.
Fix:
When a remote user belongs to multiple roles, some of which are invalid ones, only the valid roles are considered for authorization. Also, this is consistently done across methods of access (GUI, ssh, etc.) and across all remote authentication methods (LDAP, tacplus, RADIUS, etc.).
1135861-2 : LDAP authentication mishandling
Component: F5OS-A
Symptoms:
Under certain circumstances when LDAP authentication is configured, a remote user may not be authorized correctly when logging into the command line.
Conditions:
An improperly configured user profile.
LDAP configured on F5OS.
Impact:
Authorization does not occur as expected.
Workaround:
Restrict access to the management port to trusted users.
Fix:
LDAP authorization works as expected.
1135849 : telemetry.db grew to 50G and caused error "database disk image is malformed"
Component: F5OS-A
Symptoms:
As we received multiple RAS events continuously while monitoring, the telemetry.db size grew to 50G.
Conditions:
If the hardware is in issue state, we can see more events getting generated, which will increase the telemetry.db size.
Impact:
File system will not be accessible as telemetry.db is consuming more space.
Workaround:
Delete the telemetry.db file and restart the platform-monitor service.
Fix:
This fix truncates the telemetry.db to a size of 500 MB or less.
1135125-1 : Reading data from wrong socket leads to LACPD restart.
Component: F5OS-A
Symptoms:
Reading an update from the ConfD subscription socket
leads to LACPD container restart.
Conditions:
Reading an update from the ConfD subscription socket.
Impact:
This issue leads to LACPD container restart.
Workaround:
N/A
Fix:
Read data from read socket, not from subscription socket.
1135109-1 : AAA server group name and type are not displayed on ConfD
Component: F5OS-A
Symptoms:
When a server group is created on an appliance, "show system aaa server-groups" does not display the name and type of the server group.
Conditions:
When a AAA server group is created (LDAP/RADIUS/TACACS).
Impact:
appliance-1# show system aaa server-groups
NAME NAME TYPE
------------------------
ldap-group - - ----> Name and type are not displayed
Workaround:
N/A
Fix:
Published the name and type of the server group created.
1134657-1 : USB information not available in QKView
Component: F5OS-A
Symptoms:
USB information was not available in QKView.
Impact:
SEs do not have sufficient data to diagnose USB issues.
Workaround:
Execute the lsusb command and record results.
Fix:
The lsusb command is now executed as part of QKView collection.
1134141-1 : Uploading qkview to iHealth may fail on long iHealth user names
Component: F5OS-A
Symptoms:
When an iHealth username/email is entered into the configuration for the iHealth upload feature, if it is sufficiently long (over 16 characters), there may be an authentication error when attempting to upload.
Conditions:
iHealth username/email exceeds 16 characters.
Impact:
Unable to upload to iHealth.f5.com via F5OS-A or F5OS-C webUI.
Workaround:
Use the file export feature to download the qkview file from the device to a PC, and then use the PC to upload the qkview file to iHealth.f5.com.
Fix:
Feature has been fixed in F5OS-C 1.6.0 and F5OS-A 1.3.0.
1132973-2 : Live upgrade to F5OS-A-1.3.0 will not work if STP is not configured correctly.
Component: F5OS-A
Symptoms:
System database compatibility checks will fail with STP misconfigurations.
Conditions:
Live upgrades to F5OS-A-1.3.0 will not work if STP is not configured correctly.
Impact:
System database compatibility checks will fail.
Workaround:
STP cannot be enabled on individual LAG members. To perform a live upgrade to F5OS-A-1.3.0, the user must correct the STP configurations by removing the STP from the interface which is assigned to aggregation-id.
1132733-2 : LDAP config tried to configure blank bind password
Links to More Info: BT1132733
Component: F5OS-A
Symptoms:
When using the CLI or older webUI, it was possible to enter an "empty" password. This would cause nslcd to be incorrectly configured.
Conditions:
LDAP configured. Blank LDAP bind password entered:
system aaa authentication ldap bindpw ""
Impact:
A blank password was highly unlikely to be the intended result and would fail to work correctly when configuring authentication or talking to the LDAP server.
Workaround:
Explicitly set the bind password to unset:
no system aaa authentication ldap bindpw
Fix:
Fixed authentication so any form of "empty" password results in the password being unset.
1132473-1 : VELOS shows in the wording for "show services service " table on rSeries
Component: F5OS-A
Symptoms:
When the user runs "show services service" on rSeries:
appliance-1# show services service
Possible completions:
9 Service id is unique and generated by Network Manager
displaylevel Depth to show
| Output modifiers
<cr>
Possible match completions:
ipv6-prefix-length Networking mask used by disaggregator algorithms
tenant_name Tenant name associated with each Service
tier1_dag_profile sDAG on VELOS <--
tier2_dag_profile eDAG on VELOS <--
You can see "VELOS" in the description; this text is incorrect. It should either say "rSeries" or no platform at all.
Conditions:
Run "show service service".
Impact:
No functional impact.
Workaround:
N/A
1131993-1 : Not able to set severity from CLI/webUI for some services.
Component: F5OS-A
Symptoms:
Not able to set severity for following services from CLI/webUI in F5OS-A.
R5R10: - Utils-agent, partition-common(system-common), tcam-manager
R2R4: - Utils-agent, partition-common.
Conditions:
Try to change severity for following services:
R5R10: Utils-agent, partition-common(system-common), tcam-manager
R2R4: Utils-agent, partition-common.
Impact:
Not able to change severity as these services are not listed in ConfD CLI as well as webUI.
Workaround:
N/A
Fix:
Severity can now be set from CLI/webUI for all services.
1128765-2 : Data Mover lock-up causes major application traffic impact and tenant deploy failures
Links to More Info: BT1128765
Component: F5OS-A
Symptoms:
Major impact to BIG-IP tenant virtual server traffic. PoolMember health monitors fluctuate up and down, or remain down. LACP LAGs may go down.
Depending on which Data Mover (DM) is impacted, a subset of the BIG-IP tenant TMMs will no longer transmit packets. The LACP daemon will be unable to transmit its PDUs.
/var/F5/partition<n>/log/velos.log contains messages like these at the time the problem started:
blade-1(p1) dma-agent[10]: priority="Alert" version=1.0 msgid=0x4201000000000129 msg="Health monitor detected DM Tx Action Completion ring hung." ATSE=0 DM=2 OQS=3.
blade-1(p1) dma-agent[10]: priority="Info" version=1.0 msgid=0x4201000000000135 msg="Health monitor DM register dump requested.".
blade-1(p1) dma-agent[10]: priority="Info" version=1.0 msgid=0x4201000000000137 msg="Health monitor DM register dump complete." FILE="agent-dump-1666310215.txt".
In the BIG-IP tenant, the tmctl sep_stats table shows high counts for tx_send_drops2 or tx_send_drops3 (over 10,000). In the output below, all of the TMMs with SEP devices on DM 2 are impacted, unable to transmit packets.
# tmctl sep_stats --select=iface,dm,sep,atse_socket,tx_send_drops2,tx_send_drops3
iface dm sep atse_socket tx_send_drops2 tx_send_drops3
------ -- --- ----------- -------------- --------------
1/0.1 2 0 0 1180470 <-- 80068 <--
1/0.10 2 9 0 0 33046 <--
1/0.11 0 10 0 0 0
1/0.2 0 1 0 0 0
1/0.3 1 2 0 0 0
1/0.4 2 3 0 0 33714 <--
1/0.5 0 4 0 0 0
1/0.6 1 5 0 0 0
1/0.7 2 6 0 0 32980 <--
1/0.8 0 7 0 0 0
1/0.9 1 8 0 0 0
In the F5OS Partition CLI, the following command will show a high count of tx-action-ring-full drops. In the output below, DM 2 on blade-1 is impacted:
default-1# show dma-states dma-state state dm-packets dm-packet * 2-3 tx-action-ring-full
TX ACTION
NAME DM QOS RING FULL
--------------------------------
blade-1 0 2 0
0 3 0
1 2 0
1 3 0
2 2 65890377811 <--
2 3 328664822594 <--
merged 0 2 0
0 3 0
1 2 0
1 3 0
2 2 65890377811 <--
2 3 328664822594 <--
After encountering this, subsequent attempts to deploy a tenant may fail until the blade is recovered, since the locked-up Data Mover is unable to free the memory it is holding for the impacted tenants.
Conditions:
Although the exact conditions are unknown, the problem is more likely to occur when standard virtual servers are configured to mirror traffic to the peer BIG-IP.
While L7 connection mirroring increases the risk, it is not a necessary condition.
Impact:
Significant or total loss of application traffic for BIG-IP tenant instances running on the affected blade. This impact could also affect tenant instances on other blades if the LACP LAGs are marked down.
Subsequent attempts to launch a new tenant or to stop and then start an existing one may fail.
Workaround:
To recover a device, determine which blade is affected by looking at the start the following dma-agent log message in /var/F5/partition<n>/log/velos.log:
blade-1(p1) dma-agent[10]: priority="Alert" version=1.0 msgid=0x4201000000000129 msg="Health monitor detected DM Tx Action Completion ring hung." ATSE=0 DM=2 OQS=3.
^^^^^^^
Then, reboot the blade. This will shut down all tenant instances on the blade. Once the blade boots up, the tenants should run and pass traffic normally.
If the blade cannot be rebooted immediately, it may be possible to mitigate the problem for a multi-slot tenant by disabling the impacted slot to steer traffic to the remaining slots that are still healthy:
# An example of disabling BIG-IP tenant slot 1
tmsh modify sys cluster default members { 1 { disabled } }
Reducing the use of connection mirroring, especially for standard virtual servers, should reduce the likelihood of encountering this issue.
Fix:
No fix exists yet.
1126677-1 : Inconsistencies with time zones displayed in controller and log files
Component: F5OS-A
Symptoms:
System logs on F5OS systems are logged in a mix of the user's configured time zone (when available: controller/appliance) and UTC, depending on which log file you look at.
Conditions:
If user has a time zone configured that is different from UTC, the logs may show different times for log messages.
Impact:
Troubleshooting and tracing issues can be difficult, as the time zones used in different logs do not match.
Workaround:
N/A
Fix:
Fixed all controller, partition, and blade docker images to be cognizant of the relevant configured time zone for either the chassis or the partition. When a partition is created, it defaults to the configured chassis time zone, but is independently configurable thereafter.
1122829-1 : Bash history does not include timestamps for commands
Component: F5OS-A
Symptoms:
Bash 'history' does not include timestamps.
Conditions:
User is logged into a bash shell and runs the 'history' command.
Impact:
It is unclear when bash commands in 'history' were run.
Workaround:
N/A
Fix:
Bash history now includes timestamps for commands.
1117645 : Customer security policy requires disabling basic authentication
Component: F5OS-A
Symptoms:
F5OS by default enables basic authentication, meaning it allows users to perform create/modify/delete Restconf operations using basic authentication.
Conditions:
This is observed when the user tries to perform Restconf operations(except initial login) on F5OS using a username/password (basic authentication).
Impact:
This basic authentication violates some of the customer security policies.
Workaround:
N/A
1117577-1 : Management interface is not accessible if core system daemons are not running
Component: F5OS-A
Symptoms:
If the system management daemon (confd) is not able to run when the system starts up, the system will not configure its management IP address and will not have network connectivity.
Conditions:
rSeries appliance
Impact:
Management connectivity is lost, and the only way to access the system is via serial console.
Workaround:
An administrator can configure an IP address and default route for an rSeries appliance when logged in from the serial console using the "ip" command.
For instance, the following commands temporarily assign a management IP address of 198.51.100.100 to the appliance, and create a default route via a gateway of 198.51.100.254.
ip addr add 198.51.100.100/24 dev mgmt0-system
ip route add default via 198.51.100.254
Fix:
Configure IP workaround.
1114405 : Currently allowed-ip profile name is in string format. There is no restriction while configuring profile name.
Component: F5OS-A
Symptoms:
Able to create allowed IP profile name with any character (accepting all symbols and characters).
Conditions:
rSeries appliance
Impact:
allowed-ip profile name can be created with any symbols and characters.
Fix:
Added code to restrict allow list profile name to alphanumeric characters and special characters such as "-", "." and "_".
1109525-2 : K3s cluster is unhealthy when the system date or time is changed
Component: F5OS-A
Symptoms:
When the system date is changed, some of the k3s cluster certificates becomes invalid, and pods enter into an unknown/non-operational state.
Once the system date and time are made current, most pods will be recovered.
Some of the virt-controller/virt-operator/virt-api kubevirt pods are in a failed state but tenant functionality is not affected.
Conditions:
System date and time is changed back and forth.
Impact:
Some of the k3s pods go into a failed/non-operational state.
Workaround:
Re-spinning the certificates will restore the pods.
Delete the pods to trigger a re-spin of certificates that are in a terminating or crashed state.
The orchestration manager will start the pod with a new certificate.
Command to delete the pod:
#kubectl delete pod <pod-name> -n <name-space>
Fix:
Only change the system date and time when necessary.
1108509-1 : Unable to fetch appliance fan speed using SNMP
Component: F5OS-A
Symptoms:
Unable to get appliance fan RPMs using SNMP (for example, snmpget/snmpwalk).
Conditions:
Appliance with management IP and allowlist configuration.
Impact:
User cannot fetch fan RPMs using SNMP; an SNMP walk will fail.
Workaround:
Fan speed can be fetched using CLI.
Fix:
Support to fetch fan details is added to the appliance code in 1.3.0, and data can now be fetched using SNMP.
1107613-1 : Enhance the LACP LAG data shown under the interface to take into account lacp state of the LACP LAG member
Links to More Info: BT1107613
Component: F5OS-A
Symptoms:
The data shown under the /interfaces/interface oper-state and lag-speed does not take into account the lacp state for LACP LAGs.
Prior to this change, the oper-status and speed were computed only based on port oper-data.
This was the original design, with the assumption that the user will collect data from lacp state and aggregate the two outputs.
Conditions:
Configuration of LACP LAGs.
Impact:
User has to collect output from two different sources to get accuracy on LACP LAG speed and oper status.
Workaround:
n/a
Fix:
After the fix, the LACP LAG data shown under the interfaces/interface takes into account both oper-status and lacp_state when computing the speed and LAG oper-status.
1106881-4 : F5OS with an AFM license provisioned may provide incorrect AFM stats to a BIG-IP tenant
Links to More Info: BT1106881
Component: F5OS-A
Symptoms:
This is an intermittent problem where the affected BIG-IP tenant may receive incorrect statistics from the F5OS platform. This can cause the BIG-IP tenant to drop DNS traffic that should not be dropped.
Typically, the BIG-IP tenant will have periods of time where it receives the correct stats, and periods where it receives incorrect stats.
Conditions:
All of the below must be true:
-- Two or more BIG-IP tenants are deployed either on the same node in a partition or on the same appliance.
-- An AFM license is installed on the F5OS platform.
-- At least one tenant is receiving malformed DNS traffic.
Impact:
Clients that send DNS traffic to the affected BIG-IP tenant will not receive DNS responses when they should.
Workaround:
When AFM is provisioned for the system, deploying tenants on different nodes on a chassis based system or one tenant per appliance avoids the issue.
Fix:
BIG-IP tenants receive the correct platform statistics regardless of the node in which they are deployed.
1102497 : Allow for encrypted key with passphrase
Component: F5OS-A
Symptoms:
Currently all OpenSSL keys are of type unencrypted which means that no passphrase is needed to use them. Although the keys are encrypted in ConfD, there is an insecure element in that the keys reside on the filesystem in the clear.
Conditions:
Always.
Impact:
There is no support for encrypted keys with a passphrase.
Fix:
With this new option added, keys with a passphrase are supported.
1096729-1 : IP Fragments are disaggragated incorrectly
Links to More Info: BT1096729
Component: F5OS-A
Symptoms:
IP fragments are all sent to TMM0. They should be distributed to all TMMs.
Conditions:
IP fragment traffic.
Impact:
Higher than normal amount of traffic being sent to TMM0.
Fix:
Fixed in code.
1086749-3 : Interface speeds are not reported correctly when linked at a slower speed
Component: F5OS-A
Symptoms:
RSeries 2xxx/4xxx interfaces support linking at certain speeds slower than the portgroup speed, but the interface speed is reported as higher.
For example:
-- A portgroup in 25G mode accepts a 10G SFP and link at 10G. The interface speed is reported as 25G.
-- A portgroup in 25G mode can link at 1G. The interface speed is reported as 25G.
-- A portgroup in 10G mode can link at 1G. The interface speed is reported as 10G.
Conditions:
This occurs when using an SFP that only supports a slower speed, or when connecting a 10G copper port to a 1G capable device.
Impact:
The interface speed reported in the webUI/CLI is higher than the actual link speed.
Workaround:
You can determine the actual link speed using ethtool, for example:
-- For port 1.0, use ethtool x557_1.
-- For port 5.0, use ethtool sfp_5.
Fix:
Now reports correct interface speeds.
1085925-2 : SSH connection cannot be allowed/blocked based on source IP address
Component: F5OS-A
Symptoms:
There is no command in F5OS-A or F5OS-C that can be used to allow SSH connection only from specific (or range) IP addresses.
SSH connections are allowed from all source IP addresses.
Conditions:
F5 rSeries or VELOS platform
Impact:
Malicious users might be able to connect (SSH) to F5OS-A or F5OS-C device.
Workaround:
None
Fix:
The existing command "system allowed-ips allowed-ip ..." is enhanced to support SSH. The command can be used to specify source IP addresses that can establish SSH connection.
1084153-1 : Tenant deployment will fail when we move tenant (deployed with max vCPU) from provisioned to deployed
Component: F5OS-A
Symptoms:
Tenant deployment will fail when moved (deployed with max vCPU) from provisioned to deployed while the old resources are still terminating in the system.
Conditions:
When the same tenant is redeployed immediately, the appliance cannot allocate resources as the old resources were not released to the system yet. This issue is observed only on r2k/r4k but not on r5k/r10k.
Impact:
Tenant deployment will be stuck in a pending state forever.
Workaround:
Move the tenant to provisioned state and wait for the tenant resources to terminate completely in the system and then move it to a deployed state.
1080041 : Newly installed license is getting replaced with old license after performing config-restore
Component: F5OS-A
Symptoms:
When database config restore is performed, the system license was getting replaced with a license that is present in the backed up database file.
Conditions:
Config-restore is overwriting the system license.
Impact:
The system license will be removed and replaced with different license.
Workaround:
N/A
Fix:
The database operations like config-restore and reset-to-default do not remove the system license. When the database config-backup is performed, the license file won't be backed up into the backup file.
1075361-8 : Messages log has a very high number of "error" and "fail" entries
Component: F5OS-A
Symptoms:
During system bring up/reboot, various fail and error logs are seen from multiple software components.
Conditions:
During system boot up or if we perform multiple reboots we may see various errors/failures in log messages.
Impact:
User will see error/fail messages, while System bring up/reboot.
Workaround:
N/A
Fix:
Fixed the error/fail logs for few components.
1062129-1 : Tenants are in pending state forever.
Component: F5OS-A
Symptoms:
Tenants never enter into running state.
Conditions:
If a tenant request contains more vCPUs greater than available vCPUs on the system.
Impact:
-- Tenants go into pending state forever.
-- Empty CPUs are listed under tenants state in confd.
Workaround:
Always follow defined product License capability to configure vCPUs for a tenant.
1055789-2 : Apache vulnerability CVE-2021-40438
Links to More Info: K01552024
1055481 : CVE-2021-39275 Buffer overrun in ap_escape_quotes
Component: F5OS-A
Symptoms:
ap_escape_quotes() may write beyond the end of a buffer when given malicious input. No included modules pass untrusted data to these functions, but third-party / external modules may. This issue affects Apache HTTP Server 2.4.48 and earlier.
Conditions:
N/A
Impact:
N/A
Workaround:
N/A
Fix:
The httpd and dependency packages rpms has been upgraded to the httpd-2.4.6-97.el7_9.1.
1044645-1 : openssl: Read buffer overruns processing ASN.1 strings
Component: F5OS-A
Symptoms:
The openssl assumes that the ASN.1 strings are terminated by the null character in the end. When the malicious actor sends a crafted ASN.1 string without a null character through one of the openssl's public API. This can result in the application crash and denial of service.
Conditions:
A crafted message having a ASN.1 string without a null terminated character. This message can be sent through the openssl's public API functions.
Impact:
This can result in application crash, causing a Denial of Service, or possibly memory disclosure. The threat from this vulnerability is data confidentiality and system availability.
Workaround:
No workaround available.
Fix:
The vulnerability is known to affect the openssl versions 1.0.2y and below. We have updated our openssl versions to 1.0.2Zc which has the fix for the vulnerability mentioned
Known Issues in F5OS-A v1.4.x
F5OS-A Issues
ID Number | Severity | Links to More Info | Description |
1250901-1 | 1-Blocking | On rSeries FIPS system live upgrade, cavium_n3fips (liquidsec_pf_vf driver) may fail to come into operational state | |
1226505-1 | 1-Blocking | Average transactions per second impacted in certain cases | |
1184441-1 | 1-Blocking | VXLAN-GPE and GENEVE tunnel support | |
1144005-1 | 1-Blocking | BT1144005 | TPS drop of ~14% from F5OS-A 1.1.0 and later on r10000 series platforms |
1273221-1 | 2-Critical | On rSeries FIPS system, operations which involve reboot, may result in FIPS device failure. | |
1273025-2 | 2-Critical | Once TACACS server-group is configured on a non-default port; on downgrade virt-handler pod getting into crash loopbackoff state because of selinux corruption | |
1270473 | 2-Critical | On firmware upgrade from CLI, wrong console message displayed | |
1267253-1 | 2-Critical | LDAP shadowExpire attribute not honored | |
1251989-1 | 2-Critical | Changing the system Date/time back and forth using NTP server brings the system to abnormal state | |
1249773-1 | 2-Critical | QKView may fail to collect all files for platform-monitor container | |
1217169-1 | 2-Critical | Disk full: Latest ISO is not getting imported★ | |
1211853-2 | 2-Critical | Hardware offload features may affect packets destined for unrelated tenants | |
1211413-1 | 2-Critical | On R4K/R2K system live upgrade, NETCONF transactions made by CNI fail rarely. | |
1184917 | 2-Critical | On rSeries, the MAC masquerade feature is only supported on BIG-IP tenants 15.1.6 and later | |
1168573-1 | 2-Critical | Tenants failing to come up with error address already in use | |
1146093 | 2-Critical | r2000/r4000 control plane starves for CPU when appliance is fully loaded | |
1273845-2 | 3-Major | Removing or manually adding TLS Certificate & Key on webUI removes whole TLS configuration | |
1256897-1 | 3-Major | Deleting an ECDSA curve using the CLI takes a while to restart the http-server with the default RSA certificate. | |
1231889-1 | 3-Major | Deleting default VLANs and creating them in a partition other than common partition is not supported on BIG-IP tenants running on R2800/R4800 platforms | |
1222721-1 | 3-Major | Deletion of STP configuration using "no stp" is failing | |
1217197 | 3-Major | FIPS partition reuse | |
1196017 | 3-Major | Kube-flannel stuck in ImagePullBackOff status due to wrong port or tag | |
1196005 | 3-Major | K3S pods version is shown incorrect★ | |
1190965 | 3-Major | Several unused fields are displayed for BIG-IP tenants | |
1185701-1 | 3-Major | BT1185701 | 'system aaa' command in ConfD to fail with "Error: application communication failure" |
1136765-1 | 3-Major | Error when trying to perform FIPS HSM initialization through API on F5OS-A | |
1136557-2 | 3-Major | F5OS config restore fails if .iso or components vary between two devices. | |
1127393-1 | 3-Major | Error message is not displayed when user configures more than 3 DNS servers in ConfD CLI or webUI | |
1110181-1 | 3-Major | Downgrade from F5OS-A 1.3.0 or later to any release before F5OS-A 1.3.0 with tenants having more than one service can cause redirects | |
1080405-1 | 3-Major | BT1080405 | Tenant management IP addresses are not validated against other IPs in use by the system |
Known Issue details for F5OS-A v1.4.x
1273845-2 : Removing or manually adding TLS Certificate & Key on webUI removes whole TLS configuration
Component: F5OS-A
Symptoms:
Manually adding TLS Certificate & Key on webUI instead of storing through Self-Signed Certificate or removing Certificate & Key removes TLS configuration, which includes verify-client and verify-client-depth.
Conditions:
- When there is an existing Verify Client and Client Depth configuration, and user tries to remove TLS Certificate & Key on Certificate Management screen on webUI.
- When user fills the TLS Certificate & Key on Certificate Management screen on webUI.
Impact:
Verify Client and Client Depth configuration of Certificate Authentication will be changed to default values which disables verification of httpd client certificates.
Workaround:
- Remove or Add TLS Certificate & Key through CLI.
- While creating Self-Signed Certificate, set "Store TLS" field as true
- Re-add Verify Client and Client Depth after removing or manually adding TLS Certificate & Key.
1273221-1 : On rSeries FIPS system, operations which involve reboot, may result in FIPS device failure.
Component: F5OS-A
Symptoms:
After reboot of the F5OS-A rSeries system in any operations (for example, live upgrade, reboot), FIPS HSM card may not become operational and tenants that were running earlier may not come into a running state. This is due to the handshake failure between the liquid security driver and the HSM card. The driver gets stuck in SAFE_STATE instead of coming into SECURE_OPERATIONAL_STATE.
The driver state can be checked with the below command on the host system.
[root@appliance-1 ~]# cat /proc/cavium_n3fips/driver_state
HSM 0:SECURE_OPERATIONAL_STATE
[root@appliance-1 ~]#
Conditions:
The issue may occur in a live software upgrade or any situation that involves a reboot of the rSeries FIPS system with F5OS-A.
The below logs will be observed in dmesg repeatedly for every retry of the hand shake between driver and HSM card.
[ 964.113688] liquidsec_pf_vf_driver 0000:ca:00.0: We might have a link issue... resetting
[ 964.113688] liquidsec_pf_vf_driver 0000:ca:00.0: RESETTING FIRMWARE... CAUTION
Impact:
FIPS HSM is not operational in the system, which results in FIPS tenants deployed on the F5OS rSeries host do not work as expected. They do not change to a RUNNING state.
Workaround:
As the driver is stuck in "HSM 0:SAFE_STATE", a power reboot will resolve the issue.
Below are the steps to follow:
1. Power off
2. Wait for 5 minutes
3. Power on
1273025-2 : Once TACACS server-group is configured on a non-default port; on downgrade virt-handler pod getting into crash loopbackoff state because of selinux corruption
Component: F5OS-A
Symptoms:
Virt-handler pod is crashing upon downgrading from F5OS-A 1.4.0 to F5OS-A 1.3.2, and a tenant is stuck in pending state.
Conditions:
Configure TACACS server-group on a non-default port (that is, other than port 49) and downgrade from F5OS-A 1.4.0 to F5OS-A 1.3.2.
Impact:
Tenant becomes stuck in pending state.
Workaround:
1. cp selinux module from /usr
cp /usr/etc/selinux/targeted/active/modules/400/f5_appliance to /etc/selinux/targeted/active/modules/400/f5_appliance
2. Reboot the device
reboot
1270473 : On firmware upgrade from CLI, wrong console message displayed
Component: F5OS-A
Symptoms:
When the firmware upgrade command from ConfD CLI is executed, on success it displays the below message:
Result FIPS firmware has been set successfully. Please reset HSM to reflect the update!
The HSM reset does a factory reset and wipes the HSM.
Conditions:
On firmware upgrade from ConfD CLI, the wrong console message is displayed to the user.
Impact:
If HSM resets, it factory resets the HSM and wipes it.
Workaround:
Do not reset HSM; instead reboot the system to get the new firmware reflected.
1267253-1 : LDAP shadowExpire attribute not honored
Component: F5OS-A
Symptoms:
When using LDAP authentication, usage of the shadowExpire and related attributes will not enforce expiration on the F5 device.
Conditions:
LDAP authentication is configured. LDAP shadowExpire, shadowMax, and related attributes are set such that the user should be expired.
Impact:
User with expired attributes can log into F5 device.
Workaround:
Either remove the user from groups with roles that allow access to the F5 device (for example, F5OS admin role gidNumber) or delete the user.
1256897-1 : Deleting an ECDSA curve using the CLI takes a while to restart the http-server with the default RSA certificate.
Component: F5OS-A
Symptoms:
After setting a valid ECDSA curve type:
prime256v1 X9.62/SECG curve over a 256 bit prime field
secp384r1 NIST/SECG curve over a 384 bit prime field
and storing into tls the self-signed certificate the GUI will show the certificate info for this URL.
Going into the CLI and deleting the key and certificate:
su admin
config
no system aaa tls config certificate
no system aaa tls config key
commit
removes the ecdsa certificate and key and http-server is restarted with the default created rsa key and certificate.
However, the GUI still has the deleted certificate and continues to use it despite doing a refresh or attempting to log in from another browser window.
Looking at what happens under the covers, it shows that the ecdsa key and certfiicate are deleted and that httpd was restarted (all have new PID's).
The problem seems to happen with ecdsa curves only and might be explained by either of the following:
On linux operating systems, a file isn't completely deleted until the last referring program releases it.
The browser caches the certificate if it's type ecdsa and does not release that cache right away.
We notice that using the default rsa key and certificate seems to fail when the ecdsa is deleted, but after a 60 second timeout, the http-server recovers and everything seems back to normal. I could take a couple timeouts, meaning that two minutes must go by.
Conditions:
After selecting an ECDSA key type (for curve type prime256v1 or secp384r1) and connecting successfully, the key and certificate are deleted from ConfD, resulting in having the http-server use a default created RSA key and certificate.
Impact:
This can be a bit concerning, in that one expects the certificate to be replaced immediately once the key and certificate are removed. From an operational perspective, the flow does not seem to be affected as the webUI continues to work. Eventually the certificate type will no longer be the ECDSA type, but this can take a few minutes, perhaps longer.
Workaround:
To hasten the fix, one can do: docker restart http-server, which usually fixes the issue right away, or a reboot will also accomplish this.
1251989-1 : Changing the system Date/time back and forth using NTP server brings the system to abnormal state
Component: F5OS-A
Symptoms:
Upon changing the system date following things can be observed in the appliance
1. K3S cluster pods go into an errored state.
2. Cannot bring up the tenant on the Cluster
Conditions:
Either by using an NTP server or by using CLI date/time can be changed.
Changing the date forward and moving back to the original date.
Impact:
The K3S cluster does not come UP properly and eventually it brings down the tenant
Workaround:
Workaround:
1. Identify the pods which are having certificate issues.
2. In the case of the K3S cluster and kubevirt pods, It can be recovered by deleting the pods.
1250901-1 : On rSeries FIPS system live upgrade, cavium_n3fips (liquidsec_pf_vf driver) may fail to come into operational state
Component: F5OS-A
Symptoms:
After a reboot of the system in live upgrade, tenants that were running earlier might not change to a running state. This is due to the HSM board driver stuck in SAFE_STATE instead of OPERATIONAL_STATE.
In some cases, the driver changes to an operational state after some amount of time (approximately 10 minutes). But this time might vary upon detection of reset/link failure in the hardware. In some other systems, the driver becomes stuck in SAFE_STATE indefinitely.
Conditions:
Live upgrade/reboot of the rSeries FIPS system with F5OS-A.
You may observe the below logs in dmesg-
[ 964.105021] liquidsec_pf_vf_driver 0000:ca:00.0: We might have a link issue... resetting
[ 964.113688] liquidsec_pf_vf_driver 0000:ca:00.0: RESETTING FIRMWARE... CAUTION
Impact:
Running tenants goes to pending state when this issue occurs in a live upgrade.
Workaround:
Check contents of cavium_n3fips file as shown below.
[appliance]# cat /proc/cavium_n3fips/driver_state
HSM 0:OPERATIONAL_STATE
If the driver changes to an operational state, perform
"docker restart fips-support-pod" to help in recovering.
But if the driver state is still "HSM 0:SAFE_STATE", you may need to perform a power cycle reboot (but this will not guarantee recovery).
1249773-1 : QKView may fail to collect all files for platform-monitor container
Component: F5OS-A
Symptoms:
Very occasionally, QKView view will have a conflict collecting round-robin database (RRD) files in the platform monitor container. The qkview-collect routine may terminate unexpectedly as a result.
Conditions:
QKView capture request happens coincidentally to round-robin database update.
Impact:
RRD files may not be collected.
Workaround:
Rerun QKView.
1231889-1 : Deleting default VLANs and creating them in a partition other than common partition is not supported on BIG-IP tenants running on R2800/R4800 platforms
Component: F5OS-A
Symptoms:
VLANs created upon BIG-IP tenant bring-up are considered to be default VLANs and they are not supposed to be deleted and created in a different partition other than the common partition. When a VLAN that is in the common partition is deleted and created in a different partition, the subsequent default VLANs will not have a default VLAN-member associated to it.
Conditions:
When VLANs created upon tenant bring-up are deleted and created in different partitions other than the common partition.
Impact:
Partitions other than the common partition cannot have default VLANs. VLANs created in other partitions will not be operational in the data path.
Workaround:
Workaround is to create the VLAN-member for the default VLANs pushed from platform post moving a VLAN from common to another partition.
1226505-1 : Average transactions per second impacted in certain cases
Component: F5OS-A
Symptoms:
There is a reduction in http/https average transactions per second for some file sizes when ASM is configured on BIG-IP tenant on R2000 series.
Conditions:
BIG-IP config: virtual server with asm_rw policy attached to it; virtual server with profiles http, tcp, and websecurity attached to it (visual snippet is at the end of high level details).
CPU: 95-97%
simulated users: 1536
The traffic involved in testing ASM is close to real world traffic conditions.
Impact:
Reduction in average transactions per second when traffic is run for a specified duration with 1536 simulated users.
Impact is seen for http traffic specific to 32kb and 5kb file sizes.
Workaround:
N/A
1222721-1 : Deletion of STP configuration using "no stp" is failing
Component: F5OS-A
Symptoms:
"no stp" is failing with below error
Aborted: 'stp rstp config' : IEEE Std 802.1Q-2018: A Bridge shall enforce the following relationships:
Due to this, user cannot delete/disable STP with a single comamnd.
Conditions:
In case of VELOS paltforms, "no stp" will fail with error.
Impact:
User will not be able to delete/disable STP configuration with single command "no stp".
Workaround:
Except below configurations, all other configurations can be deleted.
1)no stp rstp config
2)no stp stp config
3)no stp mstp config
1217197 : FIPS partition reuse
Component: F5OS-A
Symptoms:
HSM partition should not be reused for another tenant. Delete the partition and create a new one and use it for another tenant.
Conditions:
When a FIPS tenant deleted the HSM partition associated with it used to attach to another tenant.
Impact:
The HSM partition may show a wrong state with old information.
Workaround:
Not a defect. Workaround is to reinitiate the HSM partition after attaching to new tenant. Or delete the HSM partition and create with same name and use it for new tenant.
1217169-1 : Disk full: Latest ISO is not getting imported★
Component: F5OS-A
Symptoms:
Not able to import images because /var/export/chassis LVM went to read only mode when the memory usage of this LVM is reached by more than 50%.
This LVM is created as vdo (virtual data optimizer) volume, twice the size of the physical partition size, so 50% of LVM size is equal to 100% size of underlying physical device(partition), on which this LVM is being created.
When the LVM usage reaches more than 50% of the metadata, LVM is corrupted, causing this issue.
Conditions:
The issue is seen when usage of the LVM /var/export/chassis is reached around 50% by importing more than 12 F5OS-A images on an rSeries low device.
Impact:
Not able to import images once the LVM /var/export/chassis went to read only mode.
Workaround:
The workaround is to delete older images from /var/export/chassis/import/iso/ or deport images which are not in use, using command below before importing/copying new images.
appliance-1(config)# system image remove iso <old/unused iso version>
1211853-2 : Hardware offload features may affect packets destined for unrelated tenants
Component: F5OS-A
Symptoms:
When a tenant requests that hardware assist be enabled for an L4 connection, syn cookie protection, DDoS protection, or allowlist/denylist, it is possible that packets destined for other tenants on the same VLAN will be affected by the hardware assist entry.
Conditions:
Hardware assist must have been activated for a specific flow or DDoS profile, and packets must be present for unrelated tenants that are on the same VLAN and contain the same IP destination and/or IP source address as the hardware assist activation.
Impact:
Packets destined for unrelated tenants may receive unexpected handling as a result of hardware assist matching those packets. For example, packets for an unrelated tenant on the same VLAN might be unexpectedly dropped if they have the same IP destination address as the activated DDoS hardware assist.
Workaround:
Ensure that tenants all use unique VLANs or that tenants that share a VLAN use unique IP source/destination addresses for their traffic.
1211413-1 : On R4K/R2K system live upgrade, NETCONF transactions made by CNI fail rarely.
Component: F5OS-A
Symptoms:
After the reboot of the system on live upgrade, tenant state, which was running earlier, would stay in pending state and with error log, "status Not ready: containers with unready status: [compute]".
During this time, "kubectl get events -A" would report the error log as follows - "SRIOV-CNI failed to load netconf: LoadConf(): the VF 0000:ec:15.2 does not have a interface name or a dpdk driver".
And also ice driver reports VF transaction issues, that can be seen in "dmesg" as "[39926.671860] ice 0000:ec:00.4: MAC 14:a9:d0:07:a4:0c does not exist for VF 3".
Conditions:
Live upgrade of the system (R4K/R2K) with F5OS-A.
Impact:
Running tenant goes to pending state when this issue occurs on a live upgrade of the system.
Workaround:
Reboot the host system to recover.
1196017 : Kube-flannel stuck in ImagePullBackOff status due to wrong port or tag
Component: F5OS-A
Symptoms:
On the rSeries appliance, the tenant deployment fails as the kube-flannel is stuck in ImagePullBackOff status due to the wrong port or tag.
Conditions:
The exact conditions are unclear but it is observed after the upgrade from F5OS-A 1.1.1.
Impact:
Tenant deployment fails.
Workaround:
1) Check the expected registry port for the current running image.
Ex: On an appliance running with F5OS-A 1.4.0 registry port found as 2004 (I could change on other devices)
[root@appliance-1 ~]# ls -l /var/docker/config/platform.yml
lrwxrwxrwx. 1 root root 52 Feb 13 13:11 /var/docker/config/platform.yml -> /var/docker/config/appliance/1.4.0-7488/platform.yml
[root@appliance-1 ~]#
[root@appliance-1 ~]# mount | grep "1.4.0" | grep "registry"
/var/export/chassis/import/.mounts/services/R5R10/1.4.0-7488/volume.img on /var/export/chassis/import/.volumes/appliance-services-registry-2004-volume type squashfs (ro,relatime,context=system_u:object_r:container_file_t:s0)
2) Check the expected flannel tag under the identified registry port
Ex:
[root@appliance-1 ~]# crictl images | grep flannel | grep 2004
localhost:2004/appliance-flannel 0.13.1 0a69e5ee8f6ef 20.7MB
[root@appliance-1 ~]#
3)Check the kube-flannel port under the DaemonSet in /tmp/omd/scripts/kube-flannel.yml
Ex:
[root@appliance-1 ~]# grep -i "image:" /tmp/omd/scripts/kube-flannel.yml
image: localhost:2004/appliance-flannel:0.13.1
image: localhost:2004/appliance-flannel:0.13.1
[root@appliance-1 ~]#
[root@appliance-1 ~]# kubectl get ds -A
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system kube-flannel-ds 1 1 1 1 1 <none> 40d
kube-system klipper-lb 1 1 1 1 1 <none> 40d
kube-system kube-multus-ds-amd64 1 1 1 1 1 kubernetes.io/arch=amd64 40d
kubevirt virt-handler 1 1 1 1 1 kubernetes.io/os=linux 40d
[root@appliance-1 ~]#
[root@appliance-1 ~]# kubectl describe ds kube-flannel-ds -n kube-system | grep -i "image"
Image: localhost:2004/appliance-flannel:0.13.1
Image: localhost:2004/appliance-flannel:0.13.1
[root@appliance-1 ~]#
4) Edit the DaemonSet of kube-flannel and correct the registry port or tag as applicable
Ex: # kubectl edit ds kube-flannel-ds -n kube-system
5) After editing the port or tag on the flannel DaemonSet, reboot the system and check whether the pod is starting or not.
1196005 : K3S pods version is shown incorrect★
Component: F5OS-A
Symptoms:
In rSeries r4000 and r2000 devices with v1.1.1, all K3S services have an incorrect tag (string 'message') instead of the actual number, due to an unknown issue with the docker registry at that time.
Conditions:
Live upgrade.
Impact:
Tenant deployment fails.
Workaround:
Live upgrade to the release after 1.2.0.
1190965 : Several unused fields are displayed for BIG-IP tenants
Component: F5OS-A
Symptoms:
Several unused fields are displayed for BIG-IP tenants when issuing the 'show tenants' command. These unused fields are ha-state, mac-ndi-set, and floating-address.
Conditions:
The unused fields will be displayed anytime 'show tenants' is issued.
Impact:
Display impact only.
Workaround:
No workaround.
1185701-1 : 'system aaa' command in ConfD to fail with "Error: application communication failure"
Links to More Info: BT1185701
Component: F5OS-A
Symptoms:
System fails to change password and renders system in a degraded state where user management no longer works.
System fails to provide proper user feedback to the user about failed password changes.
Conditions:
This policy option is causing the problem:
system aaa password-policy config retries 5
Impact:
F5OS user password cannot be changed.
Workaround:
Do not change the configuration from default.
system aaa password-policy config retries 5
1184917 : On rSeries, the MAC masquerade feature is only supported on BIG-IP tenants 15.1.6 and later
Component: F5OS-A
Symptoms:
The MAC masquerade feature is only supported on BIG-IP tenant versions 15.1.6 and later. Using the feature in an HA pair can cause traffic to fail over incorrectly between the pair.
Conditions:
MAC masquerade is used on rSeries with BIG-IP tenant versions other than 15.1.6 and later.
Impact:
Traffic may be degraded on a failover between an HA pair.
Workaround:
Upgrade BIG-IP tenant version to 15.1.6 or later.
1184441-1 : VXLAN-GPE and GENEVE tunnel support
Component: F5OS-A
Symptoms:
VXLAN-GPE and GENEVE tunnel support can cause host-generated UDP frames with destination ports matching system configured destination ports for VXLAN-GPE or GENEVE to be treated as VXLAN-GPE or GENEVE traffic even if the underlying frame is not VXLAN-GPE or GENEVE. Frames fitting this characteristic may have a bad UDP checksum forced onto the frame if frame fails basic VXLAN-GPE or GENEVE protocol checks.
Conditions:
Administrator configures VXLAN-GPE and/or GENEVE tunnel support.
Impact:
Minimal.
Workaround:
Tunnels are disable by default. This issue is only observed if tunnels are enabled.
1168573-1 : Tenants failing to come up with error address already in use
Component: F5OS-A
Symptoms:
rSeries tenants fail to come up with an error address already in use when tenants are deployed, without waiting for the system to complete the downgrade process from 1.3.0 to 1.2.0.
Conditions:
rSeries appliance takes up to 8 minutes to properly downgrade from 1.3.0 to 1.2.0.
The user will observe this issue if they deployed the tenant before the downgrade procedure.
Impact:
Tenants will land in an error state (address already in use) and cannot be recovered unless the system is rebooted.
Workaround:
User should wait at least 8 minutes for the downgrade to complete before deploying the tenants.
1146093 : r2000/r4000 control plane starves for CPU when appliance is fully loaded
Component: F5OS-A
Symptoms:
When an r2000/r4000 appliance is running to maximum capacity, and the CPU load is 100%, the Control Plane does not get enough cycles to perform operations.
Conditions:
Always
Impact:
During this time, the user cannot have the same experience as on a normal appliance.
- Appliance webUI takes upwards of one minute to load a home page.
- An incorrectly entered password takes over 5 seconds to process.
- A "show system | nomore" command takes about 5 minutes to complete.
1144005-1 : TPS drop of ~14% from F5OS-A 1.1.0 and later on r10000 series platforms
Links to More Info: BT1144005
Component: F5OS-A
Symptoms:
A TPS drop of approximately 12-14% was observed when running 512KB L7 HTTP tests on r10000 series platforms.
Increased CPU usage, and larger tcp_lro receive packet sizes and some packet drops were observed when it is running with full capacity.
Conditions:
Upgrade F5OS-A software version from 1.0.0 to any later software version, including 1.1.0, 1.2.0, 1.3.0, and 1.4.0.
Impact:
If F5OS software is upgraded from F5OS-A 1.0.0 to any later version, including F5OS-A 1.1.0, there will a maximum drop of 14% in TPS from device actually supported in F5OS-A 1.0.0.
Workaround:
No mitigation currently available.
1136765-1 : Error when trying to perform FIPS HSM initialization through API on F5OS-A
Component: F5OS-A
Symptoms:
On F5OS-A, if you try to perform a FIPS initialization of the HSM through the restconf API, a 502 error appears due to a timeout.
Conditions:
F5OS-A 1.4.0 on FIPS platforms using API endpoints for FIPS HSM initialization.
Impact:
FIPS initialization fails through restconf.
Workaround:
Use ConfD CLI for FIPS initialization.
1136557-2 : F5OS config restore fails if .iso or components vary between two devices.
Component: F5OS-A
Symptoms:
If the .iso or components in the backup file do not match the ones in the restore file, the restore operation fails with admin access denied error:
Error: Database config-restore failed.
Conditions:
Take a config backup from one device and restore it on another device on where .iso or components vary.
Impact:
Configuration restore fails.
Workaround:
Ensure that .iso and components match when performing backup and restore between devices.
1127393-1 : Error message is not displayed when user configures more than 3 DNS servers in ConfD CLI or webUI
Component: F5OS-A
Symptoms:
When user tries to configure more than 3 DNS server entries in F5OS-A using command "system dns servers server" or from webUI, no error message is displayed. System allows only 3 DNS servers, but user will be allowed to configure more than 3.
Conditions:
Configure DNS server in F5OS-A using ConfD CLI or webUI.
Impact:
No impact. Even though user configures more than 3, system will take only 3 entries.
Workaround:
NA
1110181-1 : Downgrade from F5OS-A 1.3.0 or later to any release before F5OS-A 1.3.0 with tenants having more than one service can cause redirects
Component: F5OS-A
Symptoms:
From F5OS-A 1.3.0 or later releases have new DAG capability to enable "dag-adjust" and "ipv6-prefix-length" settings in combination with BIG-IP tenant version 15.1.8 and later.
If we downgrade F5OS-A 1.3.0 or later to older releases before F5OS-A 1.3.0 and have a BIG-IP tenant software version 15.1.8 or later, the platform creates duplicate service entries. Due to a mismatch in the DAG profile in the platform and tenant, there will be packet redirects.
Conditions:
Downgrading F5OS-A 1.3.0 or later to any older release before F5OS-1.3.0 with BIG-IP tenant software version 15.1.8 or later.
Impact:
Performance degrade due to packet redirects.
Workaround:
Workaround steps:
1. Backup the tenant's configuration that experienced this issue -> https://support.f5.com/csp/article/K13132
2. Copy the configuration off the tenant to some other host
3. Take note of the affected tenant's partition configuration -> show running-config tenants tenant <name>
4. Delete the affected tenant in the partition
5. Recreate the tenant with the same configuration noted in step #3
6. Copy the tenant config backup taken in step #1 back to the tenant and reload the configuration
1080405-1 : Tenant management IP addresses are not validated against other IPs in use by the system
Links to More Info: BT1080405
Component: F5OS-A
Symptoms:
Tenant management IP addresses are not validated against other IP addresses in use by the system. This means that a user may configure a tenant management IP that is used elsewhere in the system and/or network causing a duplicate address. It is expected that the user is managing the IP allocation on their management network, and will not configure duplicates.
Conditions:
The user configures a tenant with a management IP that is already used by another tenant or interface in the system.
Impact:
This will cause issues accessing the tenant via the management IP if a duplicate is configured, as well as causing issues accessing the component/interface already using the address.
Workaround:
The workaround is to configure the management IP of the tenant to be an address that is not already in use.
★ This issue may cause the configuration to fail to load or may significantly impact system performance after upgrade
For additional support resources and technical documentation, see:
- The F5 Networks Technical Support web site: http://www.f5.com/support/
- The AskF5 web site: https://support.f5.com/csp/#/home
- The F5 DevCentral web site: http://devcentral.f5.com/