F5 Logo MyF5
Search
  1. MyF5 Home
  2. BIG-IQ: Sizing Guidelines
  3. DCD Sizing Guidelines
Manual Chapter : DCD Sizing Guidelines

Applies To:

Show Versions Show Versions

BIG-IQ Centralized Management

  • 8.3.0, 8.2.0, 8.1.0
Manual Chapter

DCD Sizing Guidelines

What should I know about estimating disk space requirements devices in a DCD cluster?

When estimating how much disk space you'll need for the devices in your data collection device cluster, consider these things:
  • Number of BIG-IP devices you're managing
  • Alert rate and size
  • Data retention policy
Although you can expand the alert partition for devices in a DCD cluster, be aware that if you exceed 50% of the available disk space (which is not the same as raw disk space), you will not be able to upgrade the devices in the DCD cluster. After deployment, BIG-IQ uses a certain amount of the disk space for the system software and is unavailable for storing data.
For more information about how DCDs scale data see
Configure Data Collection Devices
 in the
Setting up and Configuring a BIG-IQ Centralized Management Solution
 guide found on
support.f5.com
.

Determining available disk space for devices in a DCD cluster

If you are planning to upgrade devices in a deployed DCD cluster, you can determine the amount of available disk space.
  1. Use SSH to log in to your DCD as
    root
    .
  2. Check the disk space available by running the
    vgdisplay
     command:
    For example: 
     vgdisplay --- Volume group --- VG Name vg-db-sda System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 179 VG Access read/write VG Status resizable MAX LV 0 Cur LV 16 Open LV 7 Max PV 0 Cur PV 1 Act PV 1 VG Size 498.50 GiB PE Size 4.00 MiB Total PE 127617 Alloc PE / Size 27936 / 109.12 GiB Free PE / Size 99681 / 389.38 GiB
    In this example, the total disk space (VG Size) is 498.50 GB.
  3. Find out how the disk space is allocated across your partitions by running the
    lvs
     command.
    For example: 
     [root@vBIGIQ-LN1:Active:Standalone] config # lvs LV VG Attr LSize … dat.log.1 vg-db-vda -wi-ao--- 7.00g dat.maint.1 vg-db-vda -wi-a---- 300.00m dat.share.1 vg-db-vda -wi-ao--- 28.00g dat.swapvol.1 vg-db-vda -wi-ao--- 1.00g set.1._config vg-db-vda -wi-a---- 3.17g set.1._usr vg-db-vda -wi-a---- 3.29g set.1._var vg-db-vda -wi-a---- 10.00g set.1.root vg-db-vda -wi-a---- 440.00m set.2._config vg-db-vda -wi-ao--- 3.17g set.2._usr vg-db-vda -wi-ao--- 3.29g set.2._var vg-db-vda -wi-ao--- 10.00g set.2.root vg-db-vda -wi-ao--- 440.00m
    There are four entries to note in this response:
    • dat.log.1
       (highlighted in yellow) displays the size of the
      /var/log
       partition that is used to store the collected log data.
    • dat.share.1
       (highlighted in aqua) displays the size of the
      /shared
       partition that is used for ISOs and backups on the BIG-IQ system.
    • set.1._var
       (highlighted in green) displays the size of the
      /var
       partition for HD1.1 that is used for the initial software installation.
    • set.2._var
       (also highlighted in green) displays the size of the
      /var
      partition for HD1.2 that is used for the upgrade installation. Note that these two partitions on the active and upgrade volumes must be the same size (as in this example).
  4. If you determine that you need additional disk space, increase the disk space (for both volumes) to the size you need.
    For instructions detailing how to increase disk space, along with additional detail about disk space for BIG-IQ virtual machines, refer to
    F5 BIG-IQ Centralized Management Disk Space Management
     on
    support.f5.com
    .

What resources do I need if I’m just collecting statistics?

We’ve created a spreadsheet and a script to help you determine how many DCDs you need (and how much storage you need on each one) to handle the data generated by the BIG-IP devices you manage. You can download these tools from
downloads.f5.com
.
The spreadsheet uses metrics that you provide to calculate your resource requirements.

Using the sizing tools to estimate resources for statistics collection

The sizing tools uses metrics that help you to calculate your resource requirements. You can also use a script to evaluate your current system needs. To use the spreadsheet of the resource requirements, you must first download it.
  1. Log in to
    downloads.f5.com
    .
  2. Click
    Find a Download
    .
  3. Navigate to
    BIG-IQ
     F5 Product Family and select
    Centralized Management
    .
  4. Select
    8.0.0. - BIG-IQ
     for the product version.
  5. Select
    BIG-IQ_Sizing_Tools
    , and click
    I Accept
     to accept the EULA.
  6. Select
    F5_Networks_Disk_Size_Tools.tar
    (or
    F5_Networks_Disk_Size_Tools.tar.md5
    for the MD5 file).
    After a brief pause, the Download Locations screen opens.
  7. Click a download location to start the download.
  8. When the download completes, unzip the file.
    Two files will extract:
    • F5_Networks_BIG_IQ_DCD_Sizing_Tool.xlsx
    • F5_Networks_BIG_IQ_Apm_Dcd_Sizing_Metrics.sh
  9. Open the spreadsheet to the
    Statistics DCD Sizing Tool
     tab, and type values that describe the environment and data retention and redundancy policies for which you want to collect statistics.
    When estimating the sizing for your environment:
    If your system includes service scaling groups (SSGs) you can estimate the sizing according to the SSG environment.
    If your system does not include SSGs, enter "1" the field
    Total number of SSGs.
     Then, enter in the next field the estimated number of BIG-IP devices, and then the average number of applications per BIG-IP device.
  10. After you fill in all of the fields, the recommended number and configuration of DCDs is calculated and displayed under
    Sizing Recommendation for DCD Nodes
    . You can see the impact of your storage decisions by noting how the values under
    Time Layer Calculations
     change when you enter different values.
    The calculations performed for these elements are based on some assumptions. Bear these assumptions in mind as you complete the spreadsheet.
    • The amount of device data increases marginally as you increase the number of CPUs and interfaces on your BIG-IP devices.
    • The amount of pool and pool member data is based on an assumed average number of pool members per pool. For pools that have a larger than average number of pools, the amount of data increases as well.
    • The amount of BIG-IP VE devices and applications per service scaling group (SSG) data is based on an assumed average number of devices and applications per SSG. For SSGs that have a larger than average number of devices or applications, the amount of data increases as well.
    As you calculate the optimum number of DCDs for your solution, make sure to consider the upgrade process. If you plan to use what's referred to as a minimum downtime (or rolling) upgrade, you must have at least 3 DCDs in your Elasticsearch cluster.

What resources do I need to collect ASM data?

Use this table to estimate the number of DCDs necessary to collect and display ASM event data from BIG-IP devices.
  • If you are using an off-box logging solution to log ASM events, review a week’s worth of alert logs to identify the logging load you need to support. Take note of both the average and maximum alerts per second.
  • If you are logging ASM events locally on the BIG-IP device, you can also use those logs to identify your peak event logging rates. Compare the peak rates against the alerts per second (Alerts/Sec column) to estimate the number of DCDs you need.
  • If you have not yet deployed an ASM solution in your environment, but you have an estimate of your application’s peak HTTP/HTTPS traffic requirements, you can estimate your logging needs by projecting that 10% of your peak traffic could generate logging events. You can then use that value to estimate the number of DCDs you need.
Alerts/Sec
Data Redundancy
Recommended DCD count
10000
No
1
14000
Yes
2
17000
Yes
3
20000
Yes
4
23000
Yes
5
26000
Yes
6
  • The sizing recommendations in this table assume that you provision your DCDs with 8 cores/CPUs and 32 GB of memory.
  • When you choose an alert rate, consider both the average and peak loads that you anticipate. If your peak loads are frequent, consider provisioning additional data-collection nodes for the increased load. Also, when you determine your peak rates, plan for growth by considering the rate at which you expect your traffic load to increase.
As you calculate the optimum number of DCDs for your solution, make sure to consider the upgrade process. If you plan to use what's referred to as a minimum downtime (or rolling) upgrade, you must have at least 3 DCDs in your Elasticsearch cluster.

What resources do I need to collect APM data?

For APM, we provide a spreadsheet and a script to help you determine the number of DCDs required to handle the data generated by the BIG-IP devices you manage. You can download these tools from
downloads.f5.com
. The APM data script collects metrics for the data your APM devices generate and stores the data in a file on your BIG-IP device.
The script reads the last 10 days of logging data and provides the metrics you need for the spreadsheet. The spreadsheet uses those metrics to calculate your resource requirements. You enter information about the amount of data you expect to generate each day, and the spreadsheet calculates the number of DCDs required to handle that data.
You can use the script only if you have enabled local db-logging. For information about enabling local-db logging, refer to Overview: Configuring remote high-speed APM and SWG event logging on support.f5.com.
If you use another logging mechanism, you can run reports (such as log reports or sessions reports) to get this data.
Unless there has been a recent HA failure event (within the last 3 days), run this script on the active device. In this case, the active device may not have enough data and it is better to run the script on the standby device.
Even with the script, it might be tedious to collect data from each of your BIG-IP devices, so we recommend that you take the values from one of the BIG-IP devices with the most users and access profiles to account for the worst-case scenario.
As you calculate the optimum number of DCDs for your solution, make sure to consider the upgrade process. If you plan to use what's referred to as a minimum downtime (or rolling) upgrade, you must have at least 3 DCDs in your Elasticsearch cluster.

Using the script to estimate resources for APM data collection

The sizing tools uses metrics and a script that help you to calculate your resource requirements. To use the spreadsheet, you must first download it.
  1. Log in to
    downloads.f5.com
    .
  2. Navigate to current version of BIG-IQ Centralized Management, and click
    I Accept
     to accept the EULA.
  3. Click
    F5_Networks_BIG_IQ_DCD_Sizing_Tools.zip
    .
    After a brief pause, the Download Locations screen opens.
  4. Click a download location to start the download.
  5. When the download completes, unzip the file.
    Two files will extract:
    • F5_Networks_BIG_IQ_DCD_Sizing_Tool.xlsx
    • F5_Networks_BIG_IQ_Apm_Dcd_Sizing_Metrics.sh
  6. After the script has run, navigate to the
    /var/tmp
    folder on the BIG-IP device and open the output file to see the collected metrics.
     
    
                        NUMBER_OF_EVENTLOGS_PER_DAY
                        400,000
                        -----------------------------
                        NUMBER_OF_SESSIONS_PER_DAY
                        20,000
                        -----------------------------
                        LOGS_PER_SESSION
                        20
                        -----------------------------
                        NUMBER_OF_ACCESSS_PROFILES
                        10
                        -----------------------------
                        NUMBER_OF_ACLs
                        50
  7. Open the spreadsheet to the
    Access DCD Sizing Tool
     tab, and type the values from the script output file into the appropriate fields on the spreadsheet.
    For example, for the sample output above, you would type
    20,000 in the Number of sessions
     per day field.
  8. After you fill in all of the fields, the recommended number and configuration of DCDs is calculated and displayed in the
    Recommended number of DCD nodes
    field.

What resources do I need to collect FPS data?

Use this table to estimate the number of DCDs necessary to collect and display FPS alert data from BIG-IP devices.
  • If you are using an off-box logging solution to log FPS events, review a week’s worth of alert logs to identify the logging load you need to support. Take note of both the average and maximum alerts per second.
To determine the peak alerts/second rate to use in the table, first estimate the peak user load for the protected application over your highest period of production hours, and then plug that value into the following calculation:
<peak user load> users/(peak duration (h)*60m*60s)≈ number of logins/sec ≈ number of alerts/sec
For example, if you estimate that there are, at most, 1 million users logging in to access your FPS environment over a 4-hour timespan, the calculation results in a login rate of:
1,000,000 users/(4h*60m*60s) = ≈ 70 logins/sec ≈ 70 alerts/sec
Although each user login does not generate an alert, even single user logins or application transactions can generate multiple alerts depending on the protection policy’s configuration. For determining the number of DCDs needed to operate during peak conditions, 70 alerts per second is a reasonable assumption for this login rate.
Alerts/Sec
Data Redundancy
Recommended DCD count
100
No
1
200
Yes
2
300
Yes
3
400
Yes
4
500
Yes
5
600
Yes
6
  • The sizing recommendations in this table assume that you provision your DCDs with 8 cores/CPUs and 32 GB of memory.
  • When you choose an alert rate, consider both the average and peak loads that you anticipate. If your peak loads are frequent, consider provisioning additional data-collection nodes for the increased load. Also, when you determine your peak rates, plan for growth by considering the rate at which you expect your traffic load to increase.
  • The outgoing data rate from your BIG-IP devices also depends on the FPS policy configuration, the number of simultaneous application users, and their expected transaction rate.
  • The scale numbers provided assume the use of three forwarding rules (100% syslog forwarding, 10% SOC forwarding, 10% customer forwarding), and 17000 transform rules.
Increasing the number of forwarding targets or the distance between your data collection devices increases the performance requirements for each node in the DCD cluster.
As you calculate the optimum number of DCDs for your solution, make sure to consider the upgrade process. If you plan to use what's referred to as a minimum downtime (or rolling) upgrade, you must have at least 3 DCDs in your Elasticsearch cluster.

What resources do I need to collect data for multiple modules?

Use this process to estimate the number of DCDs that you need to collect and display more than one kind of data from your BIG-IP devices.
  1. Perform the estimate for each module independently.
  2. Record the maximum number of DCDs recommended by any one estimate.
  3. Sum the total storage needed by all estimates.
  4. Divide the result in step 3 by the result of step 2.
For example:
You’ve performed the estimate for an ASM module, and the result is 2 DCDs and 60GB of storage.
You’ve performed the estimate for Statistics, and the result is 3 DCDs and 120GB of storage.
In this example, the maximum number of DCDs is 3 and the sum of the total storage is 180GB. Consequently, the recommendation is 3 DCDs (the maximum, from the statistics estimate) and 60GB storage (The sum of both, divided by the recommended number of DCDs).
As you calculate the optimum number of DCDs for your solution, make sure to consider the upgrade process. If you plan to use what's referred to as a minimum downtime (or rolling) upgrade, you must have at least 3 DCDs in your Elasticsearch cluster.
Contact Support Contact Support

Have a Question?

Support and Sales >

About F5

Education

F5 Sites

Support Tasks




©2023 F5, Inc. All rights reserved.


Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information