VCF / VCF9 · July 11, 2026

Cleaning Up Failed Components in VCF 9.1: A Deep Dive into

cleanup_component.py

A practical, nitty-gritty walkthrough of Broadcom KB 441333 — “Failed component cleanup during VCF 9.1 installation or upgrade” — plus the known issues you’re likely to run into around Identity Broker, Log Management, and VCF Automation.

Why This KB Exists

VCF 9.1 introduced VCF Management Services (VSP) — a shared, Kubernetes-based runtime that now hosts Fleet Lifecycle, Identity Broker (VIDB), Log Management, and VCF Automation as unified “components” instead of standalone appliances. That architectural shift is powerful, but it also means that when something goes wrong during import, upgrade, or deployment of one of these components, you can’t just delete a VM anymore — the component’s metadata lives inside the Fleet Lifecycle and VSP databases and has to be cleaned up in a structured way.

That’s exactly what Broadcom KB 441333 addresses. It ships an official Python script, cleanup_component.py, and defines the only three scenarios where Broadcom sanctions manual cleanup:

  1. Duplicate Component Imports — the same 9.0 component got imported into Fleet Lifecycle twice.
  2. Imported component is no longer upgradable — a 9.0 Log Management or Identity Broker import can’t be upgraded in place and needs a fresh 9.1 deployment instead.
  3. Incorrect FQDN Configuration — a component was deployed with the wrong FQDN and needs a full redeploy with the correct one.

Outside of these three scenarios, this script is not a general-purpose “undo” button — and Broadcom is explicit about where it does not apply, which is worth calling out before you touch anything.

Where This Script Should NOT Be Used

This is the part that trips people up, especially with VCF Automation (VCFA):

  • If VCFA is in a failed state and you have a valid backup, do not use this script. Use the proper Redeploy and Restore VCF Automation procedure instead.
  • VCF Automation upgrade failure cleanup is explicitly not supported via this script. If your 9.0 → 9.1 VCFA upgrade fails, the KB tells you to open a support case rather than attempt self-service cleanup.
  • The VCFA procedures in this KB are meant only for: (a) permanently removing VCFA from the fleet with no re-deploy planned, or (b) wiping VCFA for a fresh redeployment with no restore.

In short: this script deletes metadata and, in some cases, infrastructure. It is not a rollback tool, and running it against the wrong scenario (e.g., a recoverable VCFA failure) can put you in a worse spot than where you started — which is why Broadcom puts a backup warning right at the top of the resolution steps.

Preparatory Information (Don’t Skip This)

A few operational details from the KB that matter more than they look:

  • “VSP” = VCF Management Services. You’ll see vsp-component and vsp-cluster as object types throughout the script syntax — these map to individual components (Identity Broker, Log Management, VCF Automation) and to the underlying VCF Services Runtime cluster itself, respectively.
  • Where to run it: Any machine with Python3 and network reachability to Fleet Lifecycle and the VSP components will work, but Broadcom recommends running it from SDDC Manager — and for the VCF Automation / VCF Services Runtime deletion steps, running as root inside the SDDC Manager VM is mandatory, not optional.
  • Take a backup first. The KB explicitly warns: verify backups of logs/databases for the component exist before you run a delete operation. There’s no “undo.”
  • A stale failed-upgrade entry is expected and harmless. After cleanup, you may still see a failed upgrade task in the UI. The KB confirms this doesn’t block future upgrade attempts — don’t waste time trying to clear it.
  • Get your FQDNs right. Every command uses placeholders like <fleet-lcm-fqdn> and <vcf-services-runtime-fqdn>. Pull the real values from VCF Operations UI → Build → Lifecycle → VCF Management → Components rather than guessing — a wrong FQDN here is itself one of the three scenarios this script exists to fix, so don’t create a second problem while solving the first.
  • Password gotcha: For admin@vsp.local, use the password of the vmware-system-user account — both accounts share a password. This isn’t obvious and is a common source of authentication failures when running the script for the first time.

Running the Script: Component by Component

The script always follows a two-step pattern: list (to get the component-id), then delete (using that ID). Below is the full breakdown as it applies to each component type.

1. Identity Broker (VIDB)

a) Removing an installed VIDB 9.0/9.1 prior to a fresh install This isn’t done with cleanup_component.py at all — the KB redirects you to a separate, dedicated procedure: Deletion of Identity Broker from VCF Operations 9.1 (KB 444724). Worth bookmarking separately since it’s a distinct workflow.

b) Removing a previously imported VIDB 9.0 instance from the Fleet Lifecycle This is a metadata-only cleanup — VCF SSO keeps working and the 9.0 VIDB cluster is untouched. Use this when a 9.0 VIDB got imported (possibly twice, or with a broken FQDN) and you just need Fleet Lifecycle to forget about that bad record.

bash
# List the VIDB component to obtain its component-id
python cleanup_component.py list vsp-component \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

# Delete the identified VIDB component
python cleanup_component.py delete vsp-component \
  --component-id <vsp-component-id> \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

2. Log Management

a) Log Management 9.1 — full removal This is a destructive, complete removal of the Log Management solution from the fleet — not a metadata-only cleanup like the 9.0 import case below. Same list-then-delete pattern:

bash
python cleanup_component.py list vsp-component \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

python cleanup_component.py delete vsp-component \
  --component-id <vsp-component-id> \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

b) Operations for Logs 9.0 — removing a previously imported instance Like the VIDB 9.0 case, this only cleans Fleet Lifecycle records; the existing Operations for Logs 9.0 instance keeps running untouched. Same command syntax as above — the scenario determines whether this is safe (import cleanup) versus destructive (9.1 full removal), not the syntax, so make sure you know which component version/state you’re actually targeting before you hit delete.

3. VCF Automation

This is the most involved section, and the KB flags it with a hard caveat: VCFA upgrade-failure cleanup is not supported — engage support for that scenario. What follows applies only when an existing, healthy-on-paper VCFA 9.1 instance has critical issues and needs a from-scratch redeploy (no restore).

a) VCF Automation 9.1 — full removal This has two ordered sub-steps, and order matters: remove VCF Automation and the Migration Service Engine before touching the VCF Services Runtime cluster underneath it.

Step 1 — VCF Automation and Migration Service Engine:

bash
# List components on the VCF Services Runtime
python cleanup_component.py list vsp-component \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

# Delete the VCFA component from the VCF Services Runtime
python cleanup_component.py delete vsp-component \
  --component-id <vsp-component-id> \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local \
  --vcfa-vcf-services-runtime-fqdn <vcfa-vcf-services-runtime-fqdn> \
  --vcfa-vcf-services-runtime-username <vcfa-vcf-services-runtime-username>

Step 2 — VCF Automation VCF Services Runtime (the cluster itself): This must be run as root, inside SDDC Manager — no exceptions per the KB.

bash
# List the VCF Services Runtime cluster
python cleanup_component.py list vsp-cluster \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local

# Delete the runtime cluster
python cleanup_component.py delete vsp-cluster \
  --component-id <vsp-cluster-id> \
  --fleet-fqdn <fleet-lcm-fqdn> \
  --vcf-services-runtime-fqdn <vcf-services-runtime-fqdn> \
  --vcf-services-runtime-username admin@vsp.local \
  --vcenter-username <vcenter-username>

<vcenter-username> here is your SSO administrator account (e.g. Administrator@vsphere.local) — this command reaches into vCenter to tear down the underlying infrastructure, so make sure that account has the rights to do so.

b) VCF Automation 9.0 — removing a previously imported instance Again, metadata-only: the existing VCFA 9.0 cluster and its configuration are left intact. Same list/delete pattern as the other import-cleanup scenarios above.

After Cleanup: Re-Importing a 9.0 Component

If your goal was to fix a broken import rather than permanently remove the component, the KB points you to a companion article: KB 441858 — “Deploy VCF Management Components” task fails during “Import VCF Operations in Fleet Lifecycle” subtask. That article provides a separate remediate_deploy_mgmt_components.py script (also run as root on SDDC Manager) to clear invalid metadata and re-trigger the import cleanly.

Known Issues Related to These Components

Since this cleanup script sits right at the intersection of Identity Broker, Log Management, and VCF Automation, it’s worth knowing the broader set of known issues currently circulating for VCF 9.1 around these same components — several of them are the root cause of the scenarios this KB is designed to fix.

Identity Broker (VIDB)

  • VCF SSO login fails after upgrading to 9.1 (KB 442139): After upgrade, the VIDB container’s outbound LDAP/LDAPS traffic to Active Directory starts sourcing from the node IP of the host running the container instead of the previous source address format. If your firewall only allows the legacy source address through to AD ports (389/636), those new connections get dropped, breaking SSO login entirely.
  • VIDB incorrectly shows as “Embedded” after upgrade (KB 444003): After a 9.0.2 → 9.1 upgrade, an externally deployed VIDB gets misidentified as “Embedded” in the UI, blocking configuration of the external appliance. Root cause is stale metadata left behind in the VCF Operations database and vCenter inventory from the legacy 9.0.2 setup. Fix requires a dedicated Broadcom Engineering script (reset_idb_vcf91_sso_for_vcf_operations.ps1) to purge the stale registration — distinct from cleanup_component.py.
  • Cannot upgrade or uninstall IDB 9.0.2 from VCF Operations 9.1 (KB 440718): Happens when the required “Transition a VCF Identity Broker 9.0.x Instance to the VCF Management Network” step was skipped during the initial upgrade. Since VCF 9.1 architecture requires VIDB to live on the VCF Management Network, prechecks for the 9.0.2 → 9.1 upgrade fail outright until that transition is completed.
  • VIDB log bundle generation failure (KB 441590): When SSO login failures cause VIDB’s redirect URI to revert to an IP address instead of FQDN, automatic inventory sync for VIDB gets disabled in the Fleet Manager database — which then blocks Fleet Manager from generating VIDB log bundles for support cases.
  • SSO login to Operations for Logs 9.0 UI fails after VIDB is upgraded to 9.1 (documented in the VCF Operations 9.1 known issues): Once VIDB moves to 9.1 and transitions to a VCF Services Runtime instance, TLS handshakes rely on Server Name Indication (SNI) to select the right certificate. Operations for Logs 9.0.x running in FIPS mode fails to send the SNI extension, so the gateway falls back to a default, untrusted certificate — and SSO breaks.

Log Management / Components Missing from Inventory

  • Components missing from VCF Operations UI after 9.0 → 9.1 upgrade (KB 438758 via the VCF Operations 9.1 known issues page): If VCF Automation, VIDB, Operations for Networks, or Operations for Logs were deleted and re-imported in VCF Operations 9.0 before upgrading to 9.1, those components simply don’t show up in the 9.1 inventory after upgrade.
  • Unable to see components in VCF Operations 9.1 after Brownfield import (KB 445736): A related, broader version of the above — components imported into 9.0.x and then upgraded to 9.1 (including via the VCF Brownfield Import tool) may not reflect under Build → Lifecycle in the UI at all. Broadcom has confirmed this is a known defect slated for a fix in 9.1.1.
  • “Import VCF Operations in Fleet Lifecycle” subtask failures — this is effectively the upstream trigger for the “duplicate import” and “bad FQDN” scenarios that KB 441333 is designed to clean up after:
    • KB 442048 — deployment halts at this subtask due to connectivity issues (NoRouteToHostException) between SDDC Manager and the VSP cluster FQDN, or expired/locked management credentials.
    • KB 444535 — retrying a failed “Deploy VCF Management Components” step generates duplicate VCF Operations network adapters with warning status, requiring manual cleanup of duplicate adapters and stale trusted certificates in addition to the Fleet Lifecycle-side fix.
    • KB 441858 — the companion remediation script (remediate_deploy_mgmt_components.py) for clearing invalid metadata and re-importing 9.0 components, run as root on SDDC Manager.

VCF Automation

  • VCFA upgrade fails during prechecks with reference codes F4504971 / D54C671E (KB 443212): During a 9.0.2 → 9.1 VCFA upgrade, the “Deploy VCF services runtime cluster” step fails while bootstrapping a new VCF Services Runtime cluster, citing an internal system error with invalid input. Broadcom attributes this to an intermittent library validation issue. This is exactly the class of failure the KB 441333 VCFA cleanup steps exist for — though remember, VCFA upgrade-failure cleanup specifically routes to support, not self-service.
  • General VCF Automation known issues carried from earlier releases (from the VCF Automation 9.0.1 release notes) worth being aware of if you’re troubleshooting adjacent symptoms: backup failures caused by the controller selecting a stale backup instance, vco-app pods not starting after upgrading from Aria Automation Orchestrator 8.18.1 with vSphere authentication configured (breaks when special characters like spaces exist in domain/admin group names), and cluster upgrade failures from disk-dismount timeouts (since increased from 2 to 5 minutes).

Detailed Troubleshooting Checklist

If you’re heading into a cleanup operation, work through this sequence rather than jumping straight to delete:

  1. Confirm which of the three sanctioned scenarios you’re actually in. Duplicate import, non-upgradable import, or wrong FQDN. If your situation is “VCFA upgrade failed” and you’re not certain it fits cleanly into one of these, stop and open a support case instead — don’t force it through this script.
  2. Check whether you have a valid backup before running any delete command. No backup, no delete — this applies doubly to VCF Automation and Log Management, since those full removals are destructive to the running service, not just metadata.
  3. Pull real FQDNs from the UI, don’t reuse values from memory. Go to VCF Operations UI → Build → Lifecycle → VCF Management → Components and copy the exact FQDN for each component before building your command.
  4. Always run list before delete, and copy the component-id (or cluster-id) precisely. There is no confirmation prompt once you run delete with a valid ID — treat the list step as your last chance to double-check you’re targeting the right object.
  5. Watch for authentication failures against admin@vsp.local. If your list/delete commands are failing auth, verify you’re using the vmware-system-user password, not a separately configured VSP local password.
  6. For VCF Automation full removal, respect the order of operations: VCFA + Migration Service Engine cleanup first, VCF Services Runtime cluster cleanup second. Reversing this order can leave orphaned dependencies.
  7. Run root/SDDC Manager steps exactly where required. The VCF Automation VCF Services Runtime deletion step explicitly requires execution as root inside the SDDC Manager VM — running it from a workstation with network access alone isn’t sufficient for this particular step.
  8. If SSO/login issues surface around the same time as a cleanup, check whether you’re actually hitting one of the known VIDB issues above (source-IP/firewall mismatch, embedded misidentification, SNI/FIPS certificate issue) rather than a symptom of the cleanup itself — these are easy to conflate since they cluster around the same 9.0→9.1 transition window.
  9. If components silently vanish from the Build → Lifecycle UI after upgrade, check KB 438758 and KB 445736 first — this may be the known 9.1.1-pending defect rather than something the cleanup script should touch.
  10. Ignore stray “failed upgrade” entries left behind after a successful cleanup. The KB confirms these are cosmetic and won’t block subsequent upgrade attempts.
  11. Verify the script integrity before running it in production by checking the SHA256 checksum published in the KB: 27A82D393333D17261EF1E8629A69AD6C40659A8537DF961DE95E545FE656F5B.
  12. After cleanup, if the goal is re-import (not permanent removal), proceed directly to KB 441858’s remediation script rather than attempting a manual re-import through the UI — going straight through the UI without clearing invalid metadata first is a common way to reproduce the same failure.

Bottom Line

cleanup_component.py is a precise, scenario-bound tool — not a general troubleshooting hammer. It exists to resolve exactly three situations (duplicate imports, stuck non-upgradable imports, and bad FQDN deployments) for Identity Broker, Log Management, and VCF Automation within VCF 9.1’s new unified Management Services architecture. The bigger risk in practice isn’t the script itself — it’s misdiagnosing which known issue you’re actually looking at (an SSO/firewall problem, a stale metadata defect pending a 9.1.1 fix, an upstream import-stage failure, or a genuine case for this cleanup script) before you run delete. Confirm the scenario, confirm your backups, confirm your FQDNs and component IDs, and only then execute.

References

You may also like...