Manual Chapter : Health Engine API and Metadata Reference

Updated Date: 10/27/2026

Health Engine API and Metadata Reference

This reference describes the heuristic script metadata fields, data sources, suite files, the system health recommendations API, the result file structure, the database storage model, and the severity levels used by the F5OS health engine.

Each heuristic script declares a metadata block that the engine reads at load time. The following table lists all supported metadata fields.

Field Type Description
id string Unique identifier for the heuristic (for example, H00001).
name string A short, human-readable title that describes what the heuristic does.
description string A detailed description of what the heuristic does. Can be an empty string.

Field Type Description
tags array of strings Labels used for filtering (for example, data-path, control-plane, drives, aom).
role string The platform role where the script runs. Values: appliance, partition, blade, chassis.
platforms array of strings Supported platform hardware types. Uses the F5OS HAL hostType value.
triggers array of strings System events that activate the heuristic (for example, firmware update complete, blade powered on).
mode string Execution mode. Examples: container:h-engine, remote:10.0.0.1.
os_version string A semver expression that specifies which F5OS versions the script supports.
impact string Qualitative impact level of the heuristic.
engine_version string Engine version constraint string.
is_containerized boolean Set to true when mode begins with container:.
is_remote boolean Set to true when mode begins with remote:.
parameters object Default parameter values declared in the heuristic metadata.

The engine supports heuristic scripts written in Starlark, a Python-like language. Starlark scripts use the .pyh file extension.

Heuristic scripts can access the following system interfaces to inspect live platform data:

Data source Description
F5OS confd API Query configuration and operational state through the confd interface.
Docker daemon Inspect container status and configuration.
File system Read files and directories on the platform.
Platform services Access data from platform-hal, platform-fwu, diag-agent, and other PSF services.
tmstat database Query internal platform statistics.
F5OS log files Read and analyze log data.

Heuristic scripts can produce the following types of output:

Output type Description
Log messages Internal diagnostic messages used by F5 for troubleshooting. Not directly visible to customers.
Result items Conditions detected by the script, each with a severity level.
Action items Recommended or automated actions based on detected conditions.
Files Files written to the logging directory by the script.

A suite file groups one or more heuristics into a single file. Use suite files to define a reusable set of heuristics that you want to run together.

When you provide a suite file to the engine, it loads and runs all the heuristics defined in that file.

The health engine exposes health recommendations through the system health API. Use this to view recommended actions for each node.

show system health recommendations nodes node recommendation

Example output:

NODE NAME     ID                 TIMESTAMP             COMPONENT   ACTION           DESCRIPTION                                                    SEVERITY
-------------------------------------------------------------------------------------------------------------------------------------------------------------
blade-2       A.69d5f9e2.1b      2026-04-08T07:01:20Z  blade       service-restart  [S00002]: The platform-fwu service needs to be restarted       error
              A.69d5f9e2.1c      2026-04-08T07:01:20Z  blade       service-restart  [S00002]: The platform-monitor service needs to be restarted   error
controller-1  C.1.1.69ddefe5.25  2026-04-14T07:43:05Z  controller  service-restart  [S00012]: The name-service-ldap service needs to be restarted  error
              C.1.1.69ddefe5.26  2026-04-14T07:43:05Z  controller  service-restart  [S00012]: The platform-monitor service needs to be restarted   error

The following actions can appear in health recommendations:

Action Description
Restart a node Restart a blade, controller, or appliance.
Shut down a node Shut down a blade, controller, or appliance.
Capture a qkview Generate a qkview for further analysis.
Restart a service Restart a specific platform service.
Generate or clear an alarm Create or remove an alarm on the system.
Generate an event Create a system event.
Adjust logging level Change the logging verbosity for a component.
Update diag-agent component health Update the health status of a component in the diag-agent model.

Some actions run automatically. Actions that could affect system performance or operation appear as recommendations only and are not performed automatically.

For the complete list of actions, data fields, and automation behavior, see Health engine actions reference.

The engine writes results to JSON files after each run.

File When created
${log-dir}/results.json Once per run of the h-engine tool.
${log-dir}/<heuristic-id>.<iteration>/result.json Once per heuristic script run, unless --skip-results is specified.

The top-level structure contains three objects:

json
{
  "runner":     { ... },
  "heuristics": [ ... ],
  "results":    { ... }
}
Object Description
runner Describes how the run was configured and how long it took.
heuristics Lists the heuristic scripts that were loaded and executed.
results Contains the aggregated results from all heuristic scripts in the run.

Field Type Description
startTime string (RFC 3339) When the run started.
duration number Total run duration in seconds.
options object Configuration options for the run (see below).

Field Type Description
logDirectory string Path to the log output directory.
randomizeList boolean Whether the heuristic execution order was randomized.
repeatCount integer Number of times to repeat the run.
repeatDuration integer Duration-based repeat interval.
stopOnError boolean Whether to stop the run when an error occurs.
stopOnFailure boolean Whether to stop the run when a failure occurs.
concurrency integer Number of heuristics to run concurrently.
verbose boolean Whether verbose logging is enabled.
debug boolean Whether debug logging is enabled.
perHeuristicResults boolean Whether per-heuristic result files are written.

Each entry in the heuristics array contains the following fields:

Field Type Description
id string Unique identifier for the heuristic.
name string Short title of the heuristic.
description string Detailed description of the heuristic.
path string File path to the heuristic script.
tags array of strings Filtering tags.
role string Platform role where the script runs.

Results are keyed by heuristic ID. Each value is an array of execution results. A heuristic appears more than once in the array when --repeat-count or --repeat-duration is used.

Field Type Present when Description
runResult string Always Overall outcome (see runResult values below).
startTime string (RFC 3339) Always When this heuristic execution started.
duration number Always Execution time in seconds.
iteration number Always Iteration index when repeating (1-based).
results array Always Diagnostic result items emitted by the script.
actions array Always Recommended action items emitted by the script.
inputValue object Non-null Arbitrary input values set by the script.
outputValue object Non-null Arbitrary output values produced by the script.
error string On script error Error message when the script raised an uncaught exception.
backtrace string On script error Starlark stack trace when the script raised an uncaught exception.
files array of strings Non-empty Paths to files written into the logging directory by the script.
environment object Non-empty Key/value environment snapshot for this execution (for example, HE_LOG_DIR).

Value Meaning
passed The heuristic ran to completion. No result items with severity error or higher were emitted.
failure One or more result items with severity error, critical, alert, or emergency were emitted.
error The script raised an uncaught exception.

Each entry in the results array contains the following fields:

Field Type Present when Description
timestamp string (RFC 3339) Always When the result was generated.
severity string Always Syslog-aligned severity level (see severity levels below).
message string Always Human-readable description of the condition.
value any Non-null Optional measured value associated with the result.

json
{
  "runner": {
    "startTime": "2026-04-15T20:31:00Z",
    "duration": 3.142,
    "options": {
      "logDirectory": "/var/log/h-engine/run_20260415.203100",
      "randomizeList": false,
      "repeatCount": 0,
      "repeatDuration": 0,
      "stopOnError": false,
      "stopOnFailure": false,
      "concurrency": 0,
      "verbose": false,
      "debug": false,
      "perHeuristicResults": true
    }
  },
  "heuristics": [
    {
      "id": "H00001",
      "name": "Platform check",
      "description": "Verifies the platform type is supported",
      "path": "inventory/H00001.pyh",
      "tags": ["platform"],
      "role": "appliance"
    }
  ],
  "results": {
    "H00001": [
      {
        "runResult": "passed",
        "startTime": "2026-04-15T20:31:01Z",
        "duration": 0.042,
        "iteration": 1,
        "results": [
          {
            "timestamp": "2026-04-15T20:31:01Z",
            "severity": "info",
            "message": "Platform is rSeries appliance",
            "value": "appliance-1"
          }
        ],
        "actions": [],
        "environment": {
          "HE_LOG_DIR": "/var/log/h-engine/run_20260415.203100/H00001"
        }
      }
    ]
  }
}

json
{
  "heuristic": {
    "id": "H00040",
    "name": "Disk utilization check",
    "description": "Alerts when disk utilization exceeds threshold",
    "path": "inventory/H00040.pyh",
    "tags": ["storage"],
    "role": "appliance"
  },
  "result": {
    "runResult": "failure",
    "startTime": "2026-04-15T20:31:02Z",
    "duration": 0.105,
    "iteration": 1,
    "results": [
      {
        "timestamp": "2026-04-15T20:31:02Z",
        "severity": "error",
        "message": "Disk utilization is 98%, threshold is 90%",
        "value": 98
      }
    ],
    "actions": [
      {
        "timestamp": "2026-04-15T20:31:02Z",
        "severity": "critical",
        "action": "generate-alarm",
        "recommend": true,
        "message": "Disk utilization exceeds threshold",
        "data": {
          "alert": "disk-high",
          "source": "H00040"
        },
        "value": 98
      }
    ],
    "environment": {
      "HE_LOG_DIR": "/var/log/h-engine/run_20260415.203100/H00040"
    }
  }
}

The engine writes a log file for each run to the following default directory:

/var/log/h-engine/run_<date/time>

The health engine can store run results to a SQLite database and retrieve run history. This provides a persistent record of past runs beyond the JSON result files.

  • The engine writes run results to a SQLite database file called agent.db.
  • You can retrieve past run results from the database to review diagnostic history.
  • When the diag-agent runs the health engine, results are stored in agent.db automatically. The content of this database is included in qkview captures.

Execution context Storage behavior
Diag-agent Results are stored in agent.db and included in qkview captures automatically.
Command line (h-engine) Results are written to JSON files in the log directory.

The health engine uses standard syslog severity levels. Severity levels from emergency through error cause the run result to report as failure. Levels from warning through debug do not.

Value Causes run failure Meaning
emergency Yes System is unusable.
alert Yes Immediate action required.
critical Yes Critical condition.
error Yes A problem was detected.
warning No Warning condition; the system is still healthy.
notice No Normal but significant event.
info No Informational.
debug No Debug information only.

Heuristic scripts can target the following component types:

  • Blade
  • Controller
  • Appliance
  • Tray
  • PSU (Power Supply Unit)
  • PSU controller
  • Drive