While rotating credentials in the same 9.1 environment I covered in the expired password sync post, an Update of the NSX Manager admin password (credential type API) from VCF Operations kept failing after about a minute with error code VCFOPSPWID03 and a spectacularly unhelpful message: “call from NSX received invalid response.” The actual cause turned out to be simple — NSX rejected the new password because it had already been used on that manager. This being a lab, I was deliberately recycling a previous password. That goes against best practices, and NSX enforces it. But the way the error surfaces (and the way the reuse policy behaves in this build) is worth writing down. ...

In VCF 9.1, Fleet Password Management accounts can get stuck in EXPIRED status, and the password workflows changed compared to 9.0. In 9.0, you may have seen UI actions such as Disconnected or Remediate. In the 9.1 environment I was working on, those actions were not available. The UI exposed Update, and the Fleet Password Management API provides the same kind of workflow: query password accounts, then update or rotate the password. ...

The 9.1.0.0300 patch train for VMware Cloud Foundation 9.1 is applied through a few different lifecycle views. In my lab, the update covered the Fleet Lifecycle component, VCF Operations, SDDC Lifecycle, and SDDC Manager. Before starting, review the official Broadcom release notes: SDDC Manager 9.1.0.0300 Release Notes VCF Operations 9.1.0.0300 Release Notes Broadcom describes these patch releases as time-sensitive fixes that can be applied between major, minor, and maintenance releases. That also means every component may not move at the same time, so check what is actually offered in your environment instead of assuming the whole VCF bill of materials changes together. ...

Breaking Enhanced Linked Mode is one of those changes where the technical command is not the hardest part. The hard part is making sure administrators, service accounts, and external products can still log in and do their jobs after the vCenters no longer share the same SSO domain. In this post I will walk through a repeatable way to export and restore vCenter 8.x RBAC data before splitting Enhanced Linked Mode. The focus is on three permission layers: ...

While reviewing an SOS support bundle from a VMware Cloud Foundation environment, I noticed every ESXi host in the management cluster was logging the same warning, over and over, right up to the moment the bundle was collected: [vob.visorfs.ramdisk.full] Cannot extend visorfs file /vsantraces/vsantracesLSOMVerbose--...zst because its ramdisk (vsantraceFailover) is full. These are -INFO level VOB events, not errors, and they do not touch your data or VMs. But when the same symptom appears on all hosts at once, fires continuously, and never clears on its own, it is worth understanding what is actually happening and fixing it cluster-wide rather than logging into each host by hand. This post walks through the diagnosis and a small Bash script that queries, fixes, and reclaims space on every host in parallel. ...

If you’re moving vCenter Server from 8.0 U3x to 9.1.0.0 — typically as part of a VCF 9.1 deployment or a VVF 9.1 upgrade — the run can fail partway through with: Error attempting Upgrade Database Schema Please check vcintegrity migration logs for details. This is one of those failures that looks scary in the UI but has a clean, well-understood root cause: a handful of orphaned rows in the vcIntegrity database that trip a new foreign-key constraint introduced in 9.1. Below is the whole story — what the error means, how to read the logs, why it happens, and how to clear it so the upgrade completes. It’s the same shape of problem I wrote about in fixing PostgreSQL on Fleet Management 9.0.1: a schema change colliding with pre-existing data. ...

If you have upgraded your VMware Cloud Foundation (VCF) environment to version 5.x from an earlier release, you may find yourself unable to update BOM (Bill of Materials) components from the SDDC Manager UI. The “Update Now” or “Configure Update” button is greyed out and you see a frustrating error: “Update cannot be started now as compatibility file is missing” This post walks through the symptoms, root cause, and the step-by-step fix based on Broadcom KB 396202. ...

If you’ve been working with VCF Operations 9.1, you may have noticed that on the Build > Lifecycle > VCF Management > Components page, manually-added components such as VCF Operations for Networks, Log Management, Real-Time Metrics or VCF Automation no longer have a “Delete” option available in the UI. In VCF Operations 9.0, this was possible on the Fleet Management > Lifecycle > Components page by clicking the three vertical dots next to a component and selecting Delete. That option is gone in 9.1. ...

In the previous post, I covered how to license a standalone ESX 9.0 host by importing a private license file directly on the host with esxcli. That workflow was straightforward: generate a keypair, grab a license file from the portal, scp both files to the host, run two esxcli commands, done. VCF 9.1 changes everything. Starting with 9.1, you cannot generate individual ESX host licenses from the VCF Business Services console and apply them directly. Instead, Broadcom introduced a License Server appliance — a small OVA that sits between the portal and your hosts. You import your private license file into the License Server, and the License Server issues host-scoped licenses to each ESX host via API. ...

Customizing Windows guest instances in VMware environments has traditionally been handled by the VMware Guest Customization Specification (Sysprep-based). While this approach works, it has limitations — especially when you need cloud-agnostic blueprints that work across vCenter, Azure, AWS, and GCP cloud accounts. Enter Cloudbase-Init — the Windows equivalent of Cloud-Init. It provides powerful guest customization capabilities including user creation, password injection, hostname configuration, SSH public keys, and user-data script execution. This guide walks through the full process of installing and configuring Cloudbase-Init on a Windows Server template for use with VCF Automation (formerly VMware Aria Automation / vRealize Automation). ...