Setting up F5 Guided Configuration for SSL Orchestrator in High Availability

Setting up F5 Guided Configuration for SSL Orchestrator in high availability

Overview: Setting up F5 Guided Configuration for SSL Orchestrator in high availability

This section describes how to deploy F5 SSL Orchestrator high availability (HA). SSL Orchestrator HA configuration and deployment ensures a decrease in downtime and eliminates single points of failure. The deployment of SSL Orchestrator’s HA works with the BIG-IP device groups support to sync the SSL Orchestrator specific configuration items, and is transparent to the user.

The deployment occurs after completing a configuration change and selecting Deploy. The deployment request is first routed to one of the devices in the HA device group. This first device configures the device where the request is received. After successful deployment on that device, the request is repeated on other BIG-IP devices.

With SSL Orchestrator installed onto a dedicated system with failover, it automatically takes over in case of system failure. Data is synchronized between the two systems, ensuring high availability and consistent protection.

Note: When upgrading SSL Orchestrator in HA, make sure to read all of the applicable notes and warnings regarding the procedure. The HA upgrade process will break your current HA setup to prevent the other device from performing an auto upgrade.

Note: SSL Orchestrator high availability deployment is supported for use only with SSL Orchestrator versions 2.1 and later.

Setting up BIG-IP ISO with F5 Guided Configuration for SSL Orchestrator in high availability

To ensure that your F5 SSL Orchestrator high availability (HA) deployment succeeds, it is critical that you closely follow each deployment step, as well as the assumptions and dependencies, for both devices in the device group. In addition, you should adhere to all prerequisites. If the systems in the device group are not configured consistently, the deployment synchronization process might suffer errors or fail.

To install and setup the BIG-IP ISO with SSL Orchestrator Guided Configuration in HA, you will perform the following tasks to ensure your HA deployment succeeds:

• Install the BIG-IP ISO and SSL Orchestrator Guided Configuration.
• Configure the network for HA.
  • Configure the ConfigSync and Failover IP address.
  • Add a device to the local trust domain.
  • Create a Sync-Failover device group.
• Synchronize the device group.
• Set up a basic configuration for deployment.

Assumptions and dependencies

To ensure that your SSL Orchestrator HA deployment succeeds, it is critical that you closely review and follow all assumptions and dependencies.

• HA Setup: BIG-IP HA (CMI) must be set to Active-Standby mode with network failover. See the BIG-IP Device Service Clustering: Administration document for detailed information on Active-Standby HA mode.
• HA Setup: If the deployed device group is not properly synced or RPM packages are not properly syncing, make sure your HA self IP (for example, ha_self) Port Lockdown setting is not set to Allow None. On the Main tab, click Network > Self IPs and click your ha_self. If Port Lockdown is set to Allow Custom, check that the HA network port 443 is open on self IP.
• BIG-IP HA Devices: Only manual sync is supported.
• BIG-IP HA Devices: Devices in each BIG-IP HA pair must be the same model and run the same version of TMOS® (including any hotfixes). Except for the management interface, you must configure both devices to use the same arrangement of network interfaces, trunks, VLANs, self IPs (address and subnet mask), and routes. For example, if one BIG-IP device is connected to a specific VLAN/subnet using interface 1.1, the other BIG-IP device must also be connected to that VLAN/subnet using interface 1.1. If the BIG-IP device configurations do not match, this implementation will not deploy correctly, and HA failover will not work.
• User Experience: Deployment must be initiated from the active HA BIG-IP device.
• User Experience: If the environment is changed from non-HA to HA, or from HA to non-HA, the application must be redeployed.
• User Experience: You can refresh the SSL Interception Rules screen ( SSL Orchestrator > Interception Rules) for each peer device in order to see all modified changes.

Prerequisites

Before configuring the network for HA, make sure these prerequisites are in place:

• The information used to configure your devices is identical on both devices. Without identical information on both devices, the HA deployment process can suffer from errors or fail.

• The latest version of BIG-IP SSL Orchestrator is successfully installed on the first device (the Active device). See the section Installing and Upgrading F5 SSL Orchestrator to ensure that this prerequisite has been properly completed.

• Successfully set up an HA ConfigSync device group prior to starting the configuration. See the section Configuring the network for high availability and its subsections to ensure that this prerequisite has been properly completed. For additional information, refer to the BIG-IP Device Service Clustering: Administration document, section Managing Configuration Synchronization.

• SSL Orchestrator is installed with the appropriate license information using the SSL Orchestrator Setup Utility (or the CLI) and made sure your device setup information is identical on both devices:

  • While using the SSL Orchestrator Setup Utility, you have noted the details used for NTP and DNS setup and made sure they will be identical on both devices. To verify duplication, on the Main tab, click System > Configuration > Device and select NTP or DNS.
  • Ensure that any certificates used in the configuration are copied to all devices.
  • Ensure that information is identical on all devices. This information should include any of the following that are needed:
    • Client network
    • External network
    • Decrypt zone network
    • Decrypt zone control network
    • Networks providing access to ICAP devices and Receive-only devices
  • Ensure that the log publishers are configured and named the same.
  • Ensure that all systems use the same interfaces for any services. (If interface 1.1 is used to send traffic to an inline Layer 2 device on system A, then interface 1.1 must also be used on systems B, C, and D.) Note: Do not attempt to duplicate the configuration by saving and restoring a user configuration set (UCS) file from one machine to the other, or any other cloning approach. There are several IDs that must be unique that will also be duplicated, causing additional problems.

  Note: For more detailed information on using the SSL Orchestrator setup utility, see the Using the SSL Orchestrator setup utility section.

Installing the BIG-IP ISO and F5 Guided Configuration for SSL Orchestrator

Create a backup of your current configuration to ensure your settings are not lost if the update fails.

Having the latest version of F5 BIG-IP SSL Orchestrator establishes the version that later appears on your other BIG-IP HA peer device. After downloading the latest version of the SSL Orchestrator from downloads.F5.com, return to your SSL Orchestrator deployment settings.

To install the latest version, refer to Installing the new BIG-IP ISO image section in this guide or follow the steps shown below.

1. Go to https://downloads.f5.com and click Downloads. The Downloads Overview screen opens.

2. Click Find a Download. The Select a Product Line screen opens.

3. In the F5 Product Family column, find the Security section.

4. In the Product Line column, click SSL Orchestrator. The Select a Product Version and Container for SSL Orchestrator screen opens.

5. Select 15.1.0 from the list of BIG-IP version numbers and then click SSL Orchestrator. The Software Terms and Conditions screen opens.

6. Click I Accept. The Select a Download screen appears.

7. Click the appropriate filename to download BIG-IP SSL Orchestrator.

8. To install BIG-IP SSL Orchestrator, on the Main tab, click System > Software Management > Image List. The Images List screen opens.

9. From the Available Images section, select the check box next the to BIG-IP 15.1.0 ISO image.

10. Click Install. The Install Software Image popup screen opens.

11. In the Volume set name list, type a Boot Location name or number.

12. Click Install. The Images List screen opens.

    Note: If necessary, click the browser Refresh button if the BIG-IP version 15.1.0 image does not appear in the Installed Images list.

13. The BIG-IP installation is complete once the Install Status column for version 15.1.0 indicates complete.

See the section Installing and Configuring the System for F5 SSL Orchestrator in this guide for more additional detailed installation instructions that may be required after completing this set of installation tasks.

Note: Make sure to install SSL Orchestrator on the active system only. That system will copy it to the other systems in the ConfigSync group.

Later, after a successful SSL Orchestrator HA deployment, you should verify that the same version appears on the BIG-IP HA peer device.

Configuring the network for high availability

You can specify the settings for VLAN HA and self IP addresses on the active device to configure your network for high availability. If needed, you can configure all devices involved in the high availability group for HA.

Note: This network connects the various devices and must be a common Layer-2 network between all devices.

1. On the Main tab, click Network > VLANs.

   The VLAN List screen opens.

2. Click Create.

   A New VLAN screen opens where you can configure your new VLAN.

3. In the Name field, type the name (for example, ha_vlan).

4. For the Interfaces setting:

   1. From the Interface list, select an interface number.

   2. From the Tagging list, select Tagged for traffic for that interface to be tagged with a VLAN ID.

   3. Click Add.

      The interface you selected appears in the Interfaces list as a tagged service.

5. Click Finished.

   Next to the F5 logo, your device status appears showing ONLINE (ACTIVE) and Standalone with green indicators showing their status as up and running.

6. On the Main tab, click Network > Self IPs.

   The Self IP List screen opens.

7. Click Create.

   A New Self IP screen opens where you can configure your new self IP.

8. In the Name field, type the self IP name (for example, ha_self).

9. In the IP Address field, type the IP address for the device.

10. In the Netmask field, type the netmask for the device.

11. From the VLAN/Tunnel list, select the VLAN name (ha_vlan).

12. Click Finished.

Configuring ConfigSync and failover IP addresses

Before creating the device group, you should configure the configuration synchronization (ConfigSync) and Failover IP addresses for each BIG-IP system in the device group. The ConfigSync address is the IP address that the system uses when synchronizing configuration with peer devices, and the failover address is the IP address that the system uses for network failover.

1. On the Main tab, click Device Management > Devices.

   The Devices List screen opens.

2. Click your device in the device list.

   The properties screen for the device opens.

3. Click ConfigSync.

   The screen shows the ConfigSync Configuration area, with the local address of the device.

4. From the Local Address list, select the VLAN address (ha_vlan).

5. Click Update.

6. Click Failover Network, and then click Add.

   The New Failover Unicast Address screen opens.

7. In the Address field, make sure that the VLAN address (ha_vlan) is present.

8. Click Repeat.

9. After the screen refreshes, from the Address list, select the Management Address.

   Note: Connection Mirroring is not supported.

10. Click Finished.

    The Failover Unicast Configuration area lists both the VLAN HA (ha_vlan) and Management Address devices.

Adding a device to local trust domain

Any BIG-IP devices that you intend to add to a device group must first be members of the same local trust domain. When a BIG-IP device joins the local trust domain, it establishes a trust relationship with peer BIG-IP devices that are members of the same trust domain. For example, if you are creating a device group with two members, you must log in to one of the devices and join the other device to that system’s local trust domain. The devices can then exchange their device properties and device connectivity information.

1. On the Main tab, click Device Management > Device Trust.

   The Device Trust screen opens.

2. On the menu bar, click Device Trust Members to view peer and subordinate device settings.

   The Device Trust Members screen opens.

3. Click Add.

   The Device Trust screen opens, showing Retrieve Device Credentials (Step 1 of 3).

4. From the Device Type list, select Peer.

5. In the Device IP Address field, type the IP address of your device.

6. Click Retrieve Device Information.

   The screen shows Verify Device Certificates (Step 2 of 3).

7. Click Device Certificate Matches.

   The screen shows Add Device (Step 3 of 3).

8. In the Name field, type the name of the device you are adding.

9. Click Add Device.

   At the upper right, next to the F5 logo, the status of your device should show ONLINE (ACTIVE) and Connected, with a green indicator next to them showing its active and connected status.

Creating a sync-failover device group

For an HA configuration, you need to establish failover capability between two or more BIG-IP devices. Then, if an active device in a sync-failover device group becomes unavailable, the configuration objects fail over to another member of the device group, and traffic processing is unaffected. You perform this task on any one of the authority devices within the local trust domain.

1. On the Main tab, click Device Management > Device Groups.

   The Device Group List screen opens.

2. Click Create.

   The New Device Group screen opens.

3. In the General Properties area, name your new device group and select the group type.

   1. In the Name field, type the name of your device group.

   2. From the Group Type list, select Sync-Failover.

4. For the Configuration setting, retain the Basic configuration type, and then select members and define the sync type.

   1. In the Members setting, select available devices from the Available list and add them to the Includes list.

   2. From the Sync Type list, select Manual with Incremental Sync.

      Note: You must do a manual sync. If you select Automatic with Incremental Sync, your HA deployment will fail.

5. Click Finished.

The Device Groups list screen opens, listing your new device group. The ConfigSync Status column will indicate waiting Initial Sync.

Synchronizing the device group

For an HA configuration, you need to synchronize the BIG-IP® configuration data from the local device to the devices in the device group. This synchronization ensures that devices in the device group operate properly. When synchronizing self IP addresses, the BIG-IP system synchronizes floating self IP addresses only.

1. Next to the F5 logo, click Awaiting Initial Sync.

   On the Main tab, you can also click Device Management > Overview.

   The Device Management Overview screen opens, showing your Device Groups.

2. In the Sync Issues area, select ha to expand the Devices and Sync Options areas of the screen.

3. In the Devices area, select the device showing Changes Pending.

4. In the Sync Options area, select Push the selected device configuration to the group.

5. Click Sync.

You have now completed your F5® SSL Orchestrator™ HA deployment. Next, set up a basic configuration for deployment on your active device.

Setting up a basic configuration for deployment

You must create identical information on each device before deploying the configuration.

You can now setup a basic configuration for deployment on your active device.

1. On the Main tab, click SSL Orchestrator > Deployment Settings.

   The Deployment Settings screen opens.

2. Refer to the Configuring deployment settings section for complete instructions.

   After you deploy your configuration on the active device, the system automatically synchronizes the configuration with all of the other devices in the device group. Since some errors may not be apparent, it is critical that you thoroughly test and diagnose the success or failure of the deployment. Refer to Task summary for diagnosing and fixing high availability deployment for steps to test and verify your HA deployment.

Upgrading BIG-IP ISO with F5 Guided Configuration for SSL Orchestrator in high availability

Note: When upgrading SSL Orchestrator in High Availability (HA), the upgrade process will break your current HA setup to prevent the other device from performing an auto upgrade.

Before you begin, create a backup of your current configuration to ensure your settings are not lost if the update fails.

Use the following tasks to upgrade your BIG-IP ISO with SSL Orchestrator Guided Configuration in HA:

• Verify that gossip is working between the HA device pair.
• Upgrade the BIG-IP ISO to a major or point release version for SSL Orchestrator Guided Configuration.
• Upgrade the BIG-IP ISO from 14.1.x to Hotfix-BIGIP-14.1.x.x.x.x.x for SSL Orchestrator Guided Configuration.

This section applies to the following upgrade scenarios:

• BIG-IP ISO upgrade from point version (14.1.x.x) release to major version (15.x.x) release.
• BIG-IP ISO upgrade from 14.1.x to Hotfix-BIGIP-14.1.x.x.x.x.x (SSL Orchestrator RPM May be same or different as packaged with HF ISO).
• BIG-IP ISO upgrade from major version (14.1.0) release to point version (14.1.x.x.x) release.
• BIG-IP ISO upgrade from point version (14.1.x.x) release to another point version (14.1.x.x.x) release.

Assumptions and dependencies

To ensure that your SSL Orchestrator HA upgrade succeeds, it is critical that you closely review and follow all assumptions and dependencies.

• HA Setup: This chapter only addresses the SSL Orchestrator module upgrade.

• HA Setup: This chapter guides you through the upgrade process using the management IP and the SSL Orchestrator Guided Configuration UI.

• HA Setup: You must ensure you have adequate space for installing the new ISO.

• BIG-IP HA Devices: If any other modules are used and configured on the BIG-IP, refer to their respective upgrade guide before starting the upgrade. Some of the modules may require specific pre-upgrade and post-upgrade procedural steps.

• BIG-IP HA Devices: Existing traffic will be impacted during upgrade.

• BIG-IP HA Devices: Existing configurations are copied to the new partition during installation of a new ISO and not when booting into the new partition. In the new partition, the user will see the configuration which was present at the time of the ISO installation.

  Note: F5 recommends you do not perform any SSL Orchestrator configurations after installing the new software image.

• BIG-IP HA Devices: SSL Orchestrator will only successfully upgrade if the ISO upgrade is successful. If there are any issues with the new partition or software installation in the new partition, the SSL Orchestrator upgrade may fail or behave in an unexpected manner.

• User Experience: Do not click on the SSL Orchestrator menu, or sub menu, until both HA devices are successfully upgraded and running the same version of ISO.

• User Experience: Do not use the iAppsLx menu for upgrade or modify any SSL Orchestrator iAppsLx instances.

• User Experience: Only active and standby modes are supported in HA mode (assuming there will be only two devices in HA).

Prerequisites

Before upgrading the network for high availability, make sure these prerequisites are in place:

• You must have access to the SSL Orchestrator > Configuration UI through the management IP with administrator access privileges.
• You must have access privileges to SSH a device.
• Both devices should be synchronized.
• Gossip should be in a working stage.
• The HA pair should be in a “good state”. Check the logs (restnoded, restjavad, TMM) for any errors before starting the upgrade. Also make sure to perform a CMI synchronization.
• When HA breaks with an HA Layer 2 wired device setup, it may cause a Layer 2 loop that may result in network failure. To ensure there is no network failure in this instance, make sure the STP is properly configured.

Verifying gossip is working between the high availability device pair

1. On the Main tab, click Device Management > Device Groups to check the HA sync failover group settings:

   1. If the group is not present, then you must create one.

   2. The Group type should display Sync-Failover.

   3. All HA devices should be present in the Includes field.

   4. The sync type should display Manual with Incremental Sync.

2. Using any web browser, open the URL https://<Management IP>/mgmt/shared/gossip for all the HA devices. Change the <Management IP> with the management IP belonging to the HA devices and verify that the status shows as ACTIVE.

3. Using any web browser, open the URL https://<Management IP>/mgmt/tm/cm/device for all the HA devices. Change the <Management IP> with the management IP belonging to the HA devices and verify the following:

   1. Verify that the item’s count, returned on both the devices, are same.

   2. Verify that the attribute configsyncIp is present for all of the devices and that they match the HA VLAN IP of the corresponding device.

   3. Verify for each device that the attribute unicastAddress is present and that the configsyncIp value is an entry of unicastAddress. In addition, the management IP should also be an entry of unicastAddress.

4. Using any web browser, open the URL https://<Management IP>/mgmt/shared/resolver/device-groups/tm-shared-all-big-ips/devices for all of the HA devices in the browser. Change the <Management IP> with the management IP belonging to the HA devices. This will give the list of devices participating in the gossip so you can validate the following:

   1. Validate that all of the devices are present and are consistent with the results shown in step 2 and step 3.

   2. Validate that the attribute address is present and it matches the corresponding device HA configsyncIp.

5. Check the port Lockdown settings on all the HA devices by performing the following:

   1. Use any browser and log in to all HA devices using the management IP.

   2. On the Main tab, click Network > Self IPs and then select the Self IP used with HA VLAN.

   3. Verify that the Port Lockdown displays Allow All or Allow Default.

6. Using any web browser, open the URL https://<<Management IP>>/mgmt/tm/shared/bigip-failover-state for all of the HA devices. Change the <Management IP> with the management IP of the HA devices and verify the following:

   1. Verify that the attribute failoverState is set to Active for active device and Standby for the standby device.

   2. Verify that the attribute networkFailoverDeviceGroup matches the corresponding HA device.

7. If there is still an error or verification is not working correctly, perform the following to troubleshoot issues:

   1. Check whether the currently installed .iso and .rpm versions of each device are identical.

   2. Ping device B’s configsyncIp at device A.

      If this does not work, check if the VLAN for the HA setup is bound to correct network interface, and/or the selfIP (used as configsyncIp) of this VLAN is set correctly.

   3. Check that the NTP and DNS settings of each device are identical.

   4. On the Main tab, click Device Management > Overview and review the resulting status. Make sure there are no warnings or errors.

Upgrading BIG-IP ISO to a major or point release version with F5 Guided Configuration for SSL Orchestrator in high availability

Verify all Prerequisites before starting the upgrade.

Note: When upgrading SSL Orchestrator in High Availability (HA), the upgrade process will break your current HA setup to prevent the other device from performing an auto upgrade.

This section applies to the following upgrade scenarios:

• Upgrading from main release to point release (From 14.1.x to 14.1.x.x)
• Upgrading from main release to next main release (From 14.1.x to 15.x.x)
• Upgrading from main release to next point and main release (From 14.1.x to 15.x.x.x)
• Upgrading from point release to next main release or point release (From 14.1.x .x to 15.x.x.x or 15.x.x)

1. If your HA device pair are not in sync, perform a manual configuration sync and take a UCS backup on to both devices.

2. Download the target ISO from https://downloads.f5.com/esd/product.jsp?sw=BIG-IP&pro=big-ip_v14.x and select 14.1.0 from the list to see all the available downloads for the 14.1.x version of BIG-IP.

3. To upload the new ISO on both devices, perform the following steps (this process can be initiated on both of the devices in parallel and in any sequence):

   1. Using the management IP, on the Main tab, click System > Software Management > Image List and click Import.

   2. Click Choose File and select the newly downloaded ISO and click Import.

4. Update the standby box

5. Remove all devices from the device group by performing the following steps (this will break HA and prevent the other box from performing an auto upgrade):

   1. Log in to the StandBy device using the management IP and on the Main tab, click Device Management > Device Group.

   2. Click on the Include Self device.

   3. Select all the device members and move it to the Available section and click Update.

6. To install the new ISO, perform the following:

   1. From the Main tab, click System > Software Management > Image List.

   2. Select the check box next to the software image to be installed and click Install.

      Note: Wait until the image installation is complete. This page will continue to refresh to display the install status.

7. After the image installation successfully completes, perform the following steps to boot into the new partition:

   1. On the Main tab, click System > Software Management > Boot Locations.

   2. Click on the Boot Location where the new software image is installed. The General Properties screen appears.

   3. Click Activate and OK on the confirmation dialog and wait until the device is fully booted into the new partition.

      Note: This step is not applicable to the Viprion chassis.

8. After the standby device successfully boots, verify that the software upgrade was successful by checking the logs (restnoded, restjavad, ltm) for errors.

   Note: At this time, do not click on any part of the SSL Orchestrator menu or submenu in UI.

9. After you have verified that ISO upgrade was successful, on the Main tab, click SSL Orchestrator > Configuration.

   1. This will automatically upgrade SSL Orchestrator with the new RPM that is packaged with the ISO.

      Note: Once the SSL Orchestrator screen loads, do not click on any tab or link on the menu, submenu, or screen.

10. On the Main tab, click SSL Orchestrator > Configuration and validate the SSL Orchestrator RPM Version number showing in the top right banner. You can also hover your mouse over the information icon for more detailed information.

11. On the Main tab, click iApps > Application Services > Applications LX and validate that the system is still deploying a configuration. If system is deploying a configuration then wait on this screen until all deployments are complete before moving to next step.

12. In case of an error, perform the following troubleshooting steps:

    1. Correct the configuration and deploy again.

    2. If the above step does not resolve the issue, delete all SSL Orchestrator configurations and restore the UCS backup by restarting restnoded.

    3. If the UCS backup does not resolve the issue, open a support case to debug.

13. Update the active device by installing the new ISO

14. On the Main tab, click System > Software Management > Image List and select the software image that needs to be installed by selecting its corresponding checkbox.

    1. Click Install and wait until the image installation completes. This page will continue to refresh to display the install status.

15. After the image install is complete, perform the following steps to boot into the new partition:

    1. On the Main tab, click System > Software Management > Boot Locations.

    2. Select the Boot Location the new software image is installed (this will appear on the General Properties screen).

    3. Click Activate and OK and wait until the device fully boots into the new partition.

16. After the active device successfully boots, verify the software successfully upgrades on the active device and review the logs for errors.

    Note: Do not click on any tab or link on the SSL Orchestrator menu, submenu, or screen.

17. After verifying a successful ISO upgrade, on the Main tab, click SSL Orchestrator > Configuration to auto upgrade SSL Orchestrator with the new RPM.

    Note: Once the SSL Orchestrator screen loads, do not click on any tab or link on the menu, submenu, or screen.

18. On the Main tab, click SSL Orchestrator > Configuration and validate the SSL Orchestrator RPM Version number showing in the top right banner. You can also hover your mouse over the information icon for more detailed information.

19. On the Main tab, click iApps > Application Services > Applications LX to validate the system is deploying a configuration. If system is deploying a configuration, wait on this screen until all deployments are complete.

20. In case of an error, perform the following troubleshooting steps:

    1. Correct the configuration and deploy again.

    2. If the above step does not resolve the issue, delete all SSL Orchestrator configurations and restore the UCS backup by restarting restnoded.

    3. If the UCS backup does not resolve the issue, open a support case to debug.

21. To add devices to a device group, perform the following steps:

    1. On the Main tab, click Device Management > Device Group and click on the Include Self device.

    2. Select all the device group members and move them to the Includes section and click Update.

    3. Verify that the Observer Box status at the top left of the screen changes from Active and Standby mode.

22. If any changes are pending after the upgrade, initiate a configuration sync.

23. After a successful upgrade, to install a different version of the SSL Orchestrator RPM, perform the following steps:

    1. On the Main tab of an active box, click SSL Orchestrator > Configuration and click on the Upgrade SSL Orchestrator link.

    2. Select a new RPM and click Upload and install.

    3. Wait until the installation completes and the screen is refreshed.

       Note: Before continuing with any further configurations, log in to the standby device and validate that the standby device also upgraded to the new RPM version.

Upgrading BIG-IP ISO from 14.1.x to Hotfix-BIGIP-14.1.x.x.x.x.x with F5 Guided Configuration for SSL Orchestrator in high availability

Verify all Prerequisites before starting the upgrade.

Note: When upgrading SSL Orchestrator in High Availability (HA), the upgrade process will break your current HA setup to prevent the other device from performing an auto upgrade.

The SSL Orchestrator RPM may be the same as or different than the one packaged with the HF ISO.

1. If your HA device pair are not in sync, perform a manual configuration sync and take a UCS backup on to both devices.

2. Download the target HF ISO from https://downloads.f5.com.

3. To upload the EHF ISO on both devices, perform the following steps (this process can be initiated on both of the devices in parallel and in any sequence):

   1. Using the management IP, on the Main tab, click System > Software Management > Hotfix List and click Import.

   2. Click Choose File and select the newly downloaded ISO and click Import.

4. Update the standby box

5. Remove all devices from the device group by performing the following steps (this will break HA and prevent the other box from performing an auto upgrade):

   1. Log in to the StandBy device using the management IP and on the Main tab, click Device Management > Device Group.

   2. Click on the Include Self device.

   3. Select all the device members and move it to the Available section and click Update.

6. To install the new EHF ISO, perform the following:

   1. From the Main tab, click System > Software Management > Image List.

   2. Select the check box next to the software image to be installed and click Install.

      Note: Wait until the image installation is complete. This page will continue to refresh to display the install status.

7. After the image installation successfully completes, perform the following steps to boot into the new partition:

   1. On the Main tab, click System > Software Management > Boot Locations.

   2. Click on the Boot Location where the new HF software image is installed. The General Properties screen appears.

   3. Click Activate and OK on the confirmation dialog and wait until the device is fully booted into the new partition.

      Note: This step is not applicable to the Viprion chassis.

8. After the standby device successfully boots, verify that the software upgrade was successful by checking the logs (restnoded, restjavad, ltm) for errors.

   Note: At this time, do not click on any part of the SSL Orchestrator menu or submenu in UI.

9. After you have verified that ISO upgrade was successful, on the Main tab, click SSL Orchestrator > Configuration.

   1. This will automatically upgrade SSL Orchestrator with the new RPM that is packaged with the ISO.

      Note: Once the SSL Orchestrator screen loads, do not click on any tab or link on the menu, submenu, or screen.

10. On the Main tab, click SSL Orchestrator > Configuration and validate the SSL Orchestrator RPM Version number showing in the top right banner. You can also hover your mouse over the information icon for more detailed information.

11. On the Main tab, click iApps > Application Services > Applications LX and validate that the system is still deploying a configuration. If system is deploying a configuration then wait on this screen until all deployments are complete before moving to next step.

12. In case of an error, perform the following troubleshooting steps:

    1. Correct the configuration and deploy again.

    2. If the above step does not resolve the issue, delete all SSL Orchestrator configurations and restore the UCS backup by restarting restnoded.

    3. If the UCS backup does not resolve the issue, open a support case to debug.

13. Update the active device by installing the new ISO

14. On the Main tab, click System > Software Management > Image List and select the software image that needs to be installed by selecting its corresponding checkbox.

    1. Click Install and wait until the image installation completes. This page will continue to refresh to display the install status.

15. After the image install is complete, perform the following steps to boot into the new partition:

    1. On the Main tab, click System > Software Management > Boot Locations.

    2. Select the Boot Location the new software image is installed (this will appear on the General Properties screen).

    3. Click Activate and OK and wait until the device fully boots into the new partition.

16. After the active device successfully boots, verify the HF software successfully upgrades on the active device and review the logs for errors.

    Note: Do not click on any tab or link on the SSL Orchestrator menu, submenu, or screen.

17. After verifying a successful ISO upgrade, on the Main tab, click SSL Orchestrator > Configuration to auto upgrade SSL Orchestrator with the new RPM.

    Note: Once the SSL Orchestrator screen loads, do not click on any tab or link on the menu, submenu, or screen.

18. On the Main tab, click SSL Orchestrator > Configuration and validate the SSL Orchestrator RPM Version number showing in the top right banner. You can also hover your mouse over the information icon for more detailed information.

19. On the Main tab, click iApps > Application Services > Applications LX to validate the system is deploying a configuration. If system is deploying a configuration, wait on this screen until all deployments are complete.

20. In case of an error, perform the following troubleshooting steps:

    1. Correct the configuration and deploy again.

    2. If the above step does not resolve the issue, delete all SSL Orchestrator configurations and restore the UCS backup by restarting restnoded.

    3. If the UCS backup does not resolve the issue, open a support case to debug.

21. To add devices to a device group, perform the following steps:

    1. On the Main tab, click Device Management > Device Group and click on the Include Self device.

    2. Select all the device group members and move them to the Includes section and click Update.

    3. Verify that the Observer Box status at the top left of the screen changes from Active and Standby mode.

22. If any changes are pending after the upgrade, initiate a configuration sync.

23. After a successful upgrade, to install a different version of the SSL Orchestrator RPM, perform the following steps:

    1. On the Main tab of an active box, click SSL Orchestrator > Configuration and click on the Upgrade SSL Orchestrator link.

    2. Select a new RPM and click Upload and install.

    3. Wait until the installation completes and the screen is refreshed.

       Note: Before continuing with any further configurations, log in to the standby device and validate that the standby device also upgraded to the new RPM version.

Diagnosing and fixing a high availability deployment

Use the following methods to help diagnose, verify, and fix a failed high availability (HA) deployment:

• Verify your deployment and view the logs.
• Verify the RPM file version on both devices.
• Configure your deployment settings and redeploy.
• Review the error logs and perform any necessary recovery steps.

Verifying deployment and viewing logs

You can verify your deployment by verifying that the required virtuals, profiles, and BIG-IP LTM and network objects have been created, checking that the RPM files are in sync, and reviewing logs for failures, for example.

Note: Because the initial device in the HA device group repeats the configuration requests and propagates the configuration to other BIG-IP devices, make sure you verify the initial configured device first, followed by each device in the HA device group. If the initial device deployment configuration fails, all other device configuration deployments will not successfully be configured.

1. Verify that all expected and required virtuals, profiles, and BIG-IP LTM and network objects (route-domains, VLANs, self IPs) have been created on each device in the HA device group.

   These will be items beginning with the name given to the application (for example, if the application was named SSLO, verify that all of the items named | Summary SSL Orchestrator 15.1.0 | 9 SSLO_* are the same on all devices).

2. Ensure that all RPM file versions are identical.

3. Verify your deployment with, or without, services.

4. Review the following logs for failures:

   • /var/log/restnoded/restnoded.log
   • /var/log/restjavad.0.log

Verifying the RPM file version on both devices

After a successful F5® SSL Orchestrator™ HA deployment, verify that the latest version of the SSL Orchestrator zip file is installed on both devices.

Note: The following details are for SSL Orchestrator versions 14.1.x-5.x or higher.

1. On the Main tab, click SSL Orchestrator > Configuration.

   The SSL Orchestrator screen opens.

2. For both devices, validate the SSL Orchestrator RPM Version number showing in the top right banner. You can also hover your mouse over the information icon for more detailed version information.

If the versions are not identical, you must install an updated RPM file and verify that both devices are identically configured.

Configuring deployment settings and redeploying

If your configured deployment continues to fail, you can remove and reconfigure all deployment settings.

1. Remove all configurations present on all devices.

2. For all devices, individually configure each section in the F5® SSL Orchestrator™ deployment settings and select Finished. Verify that all new objects are properly synced and deployed.

   Note: If synchronization or deployment issues persist after deploying after each section, attempt to deploy after updating each item (instead of after each section) in the SSL Orchestrator deployment settings and verify that all new objects are properly synced and deployed.

Reviewing error logs and performing recovery steps

You can review log messages to help you debug system activity and perform recovery steps. Refer to the Setting up F5 Guided Configuration for SSL Orchestration logs settings section of this document for more information on generating logs and setting the level of logging you want the system to perform.

1. Verify that all BIG-IP® LTM®and network objects are present on each of the devices in the HA device group.

2. If the configuration deployment fails on each device, review the logs:

   • /var/log/restnoded/restnoded.log
   • /var/log/restjavad.0.log

3. Use the following REST GET command to determine the state of the deployed device block in the REST storage:

   • curl -s -k -u admin:admin https://localhost/mgmt/shared/iapp/blocks | json-format

4. Since failure scenarios can vary, after reviewing the logs, attempt the following recovery steps:

   1. Redeploy SSL Orchestrator.

      If this succeeds, you have recovered from the failure situation.

   2. Undeploy SSL Orchestrator.

      By undeploying, a cleanup of MCP objects on each of the devices occurs while also cleaning up required data properties within the block stored in REST storage. If this succeeds, attempt to redeploy again.

   3. If redeploy or undeploy fails, do the following:

      1. From command line (back door), run > touch /var/config/rest/iapps/enable.
      2. Refresh the SSL Orchestrator menu UI.
      3. Select the deployed application from the list and delete the application.
      4. Redeploy and undeploy again.
      5. Once done, remove the file rm -f /var/config/rest/iapps/enable.

   4. If these recovery steps do not work, you may need to clean up the REST storage.

Note: For more detailed information on setting up HA, see the BIG-IP Device Service Clustering: Administration document.