<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>VCF 9 Fleet Management on Cosmin.us</title><link>https://cosmin.us/series/vcf-9-fleet-management/</link><description>Recent content in VCF 9 Fleet Management on Cosmin.us</description><generator>Hugo</generator><language>en-US</language><dc:creator>Cosmin Trif</dc:creator><lastBuildDate>Wed, 08 Jul 2026 12:50:00 +0000</lastBuildDate><atom:link href="https://cosmin.us/series/vcf-9-fleet-management/index.xml" rel="self" type="application/rss+xml"/><item><title>Fixing Expired Password Sync in VCF 9.1 Fleet Management</title><link>https://cosmin.us/fixing-expired-password-sync-in-vcf-9-1-fleet-management/</link><pubDate>Tue, 07 Jul 2026 17:30:00 +0000</pubDate><dc:creator>Cosmin Trif</dc:creator><guid>https://cosmin.us/fixing-expired-password-sync-in-vcf-9-1-fleet-management/</guid><description>&lt;p&gt;In VCF 9.1, Fleet Password Management accounts can get stuck in &lt;strong&gt;EXPIRED&lt;/strong&gt; status, and the password workflows changed compared to 9.0. In 9.0, you may have seen UI actions such as &lt;strong&gt;Disconnected&lt;/strong&gt; or &lt;strong&gt;Remediate&lt;/strong&gt;. In the &lt;a href="https://cosmin.us/upgrading-vcf-9-1-to-9-1-0-0300-step-by-step/"&gt;9.1 environment I was working on&lt;/a&gt;, those actions were not available. The UI exposed &lt;strong&gt;Update&lt;/strong&gt;, and the Fleet Password Management API provides the same kind of workflow: query password accounts, then update or rotate the password.&lt;/p&gt;</description><content:encoded>&lt;p&gt;In VCF 9.1, Fleet Password Management accounts can get stuck in &lt;strong&gt;EXPIRED&lt;/strong&gt; status, and the password workflows changed compared to 9.0. In 9.0, you may have seen UI actions such as &lt;strong&gt;Disconnected&lt;/strong&gt; or &lt;strong&gt;Remediate&lt;/strong&gt;. In the &lt;a href="https://cosmin.us/upgrading-vcf-9-1-to-9-1-0-0300-step-by-step/"&gt;9.1 environment I was working on&lt;/a&gt;, those actions were not available. The UI exposed &lt;strong&gt;Update&lt;/strong&gt;, and the Fleet Password Management API provides the same kind of workflow: query password accounts, then update or rotate the password.&lt;/p&gt;
&lt;h2 id="symptoms"&gt;Symptoms&lt;/h2&gt;
&lt;p&gt;The issue in this case was simple to describe but easy to chase in the wrong place:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Some password accounts showed as &lt;strong&gt;EXPIRED&lt;/strong&gt; in Fleet password management.&lt;/li&gt;
&lt;li&gt;The VCF Operations &lt;code&gt;root&lt;/code&gt; password had already been changed outside Fleet.&lt;/li&gt;
&lt;li&gt;The 9.1 UI did not expose the older remediation-style options, so the useful paths were the &lt;strong&gt;Update&lt;/strong&gt; action for single accounts and the API for repeatable remediation.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This post walks through the recovery path I used for VCF 9.1.&lt;/p&gt;
&lt;h2 id="what-changed-in-vcf-91-password-management"&gt;What Changed in VCF 9.1 Password Management&lt;/h2&gt;
&lt;p&gt;The important realization is that VCF 9.1 Fleet Password Management does not expose a documented &amp;ldquo;store this already changed password in the vault&amp;rdquo; workflow.&lt;/p&gt;
&lt;p&gt;The UI exposes &lt;strong&gt;Update&lt;/strong&gt; for a password account. For automation or for working through a larger list of expired entries, the official API exposes:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;POST /suite-api/api/fleet-management/password-management/accounts/query&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;PUT /suite-api/api/fleet-management/password-management/accounts/{passwordAccountKey}/password&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The update workflow requires both the current password and the new password. In the API these fields are &lt;code&gt;currentPassword&lt;/code&gt; and &lt;code&gt;newPassword&lt;/code&gt;; in the UI they are entered through the &lt;strong&gt;Update&lt;/strong&gt; action. Either way, this is a rotation workflow. If a password was already changed directly on the appliance, you need to provide that current real password and rotate once more through Fleet so the appliance password, the vault, and the expiration metadata are back in sync.&lt;/p&gt;
&lt;p&gt;References:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://developer.broadcom.com/xapis/vcf-operations-api/latest/suite-api/api/fleet-management/password-management/accounts/query/post/"&gt;Get VCF Password Accounts&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.broadcom.com/xapis/vcf-operations-api/latest/suite-api/api/fleet-management/password-management/accounts/passwordAccountKey/password/put/"&gt;Update Password&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.broadcom.com/xapis/vcf-operations-api/latest/"&gt;VCF Operations API Authentication&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="get-a-vcf-operations-api-token"&gt;Get a VCF Operations API Token&lt;/h2&gt;
&lt;p&gt;The VCF Operations API supports &lt;code&gt;OpsToken&lt;/code&gt; authorization. Set the VCF Operations API host first:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;OPS&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;https://&amp;lt;vcf-operations-fqdn&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; These examples use &lt;code&gt;curl -k&lt;/code&gt;, which skips TLS certificate verification — common in labs where the appliance still has its self-signed certificate. With &lt;code&gt;-k&lt;/code&gt; you are sending credentials to whatever answers on that FQDN. In production, drop &lt;code&gt;-k&lt;/code&gt; and trust the VCF Operations CA instead, for example &lt;code&gt;curl --cacert /path/to/vcf-ops-ca.pem ...&lt;/code&gt;, or add the CA to the system trust store.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Use an account that is authorized to call the Fleet password management APIs. Avoid echoing the password back to the terminal. The &lt;code&gt;read -rp&lt;/code&gt; / &lt;code&gt;read -rsp&lt;/code&gt; prompts in this post use bash syntax; if your interactive shell is zsh, either start &lt;code&gt;bash&lt;/code&gt; first or use the zsh form, for example &lt;code&gt;read -rs 'VCFOPS_PASSWORD?VCF Operations password: '&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Acquire a token:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;read&lt;/span&gt; -rp &lt;span class="s2"&gt;&amp;#34;VCF Operations username: &amp;#34;&lt;/span&gt; VCFOPS_USERNAME
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;read&lt;/span&gt; -rsp &lt;span class="s2"&gt;&amp;#34;VCF Operations password: &amp;#34;&lt;/span&gt; VCFOPS_PASSWORD&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;TOKEN&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;jq -nc &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; --arg u &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$VCFOPS_USERNAME&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; --arg p &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$VCFOPS_PASSWORD&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s1"&gt;&amp;#39;{username:$u,password:$p}&amp;#39;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; curl -sk -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; -d @- &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/auth/token/acquire&amp;#34;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; jq -r .token&lt;span class="k"&gt;)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;unset&lt;/span&gt; VCFOPS_PASSWORD
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="o"&gt;[&lt;/span&gt; -n &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; !&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;null&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;Token acquisition failed&amp;#34;&lt;/span&gt; &amp;gt;&lt;span class="p"&gt;&amp;amp;&lt;/span&gt;&lt;span class="m"&gt;2&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Test the token:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/versions/current&amp;#34;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt; jq .
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id="query-expired-password-accounts"&gt;Query Expired Password Accounts&lt;/h2&gt;
&lt;p&gt;Now query the expired password entries:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -d &lt;span class="s1"&gt;&amp;#39;{&amp;#34;status&amp;#34;:&amp;#34;EXPIRED&amp;#34;}&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/fleet-management/password-management/accounts/query?pageSize=200&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;|&lt;/span&gt; tee /tmp/expired-password-accounts.json &lt;span class="p"&gt;|&lt;/span&gt; jq .
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;For a cleaner table:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;jq -r &lt;span class="s1"&gt;&amp;#39;.vcfPasswordAccounts[] |
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s1"&gt; [.appliance, .applianceFqdn, .userName, .credentialType, .expiryDate, .passwordAccountKey] |
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="s1"&gt; @tsv&amp;#39;&lt;/span&gt; /tmp/expired-password-accounts.json
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;I usually sort the output into a tracking table like this:&lt;/p&gt;
&lt;table&gt;
	&lt;thead&gt;
			&lt;tr&gt;
					&lt;th&gt;Appliance&lt;/th&gt;
					&lt;th&gt;FQDN&lt;/th&gt;
					&lt;th&gt;User&lt;/th&gt;
					&lt;th&gt;Credential Type&lt;/th&gt;
					&lt;th&gt;Action&lt;/th&gt;
			&lt;/tr&gt;
	&lt;/thead&gt;
	&lt;tbody&gt;
			&lt;tr&gt;
					&lt;td&gt;VCF_OPERATIONS&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;&amp;lt;vcf-operations-fqdn&amp;gt;&lt;/code&gt;&lt;/td&gt;
					&lt;td&gt;root&lt;/td&gt;
					&lt;td&gt;SSH&lt;/td&gt;
					&lt;td&gt;Update in UI or rotate through API&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;SDDC_MANAGER&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;&amp;lt;sddc-manager-fqdn&amp;gt;&lt;/code&gt;&lt;/td&gt;
					&lt;td&gt;root&lt;/td&gt;
					&lt;td&gt;SSH&lt;/td&gt;
					&lt;td&gt;Confirm current password, then rotate&lt;/td&gt;
			&lt;/tr&gt;
			&lt;tr&gt;
					&lt;td&gt;VCF_SERVICES_RUNTIME&lt;/td&gt;
					&lt;td&gt;&lt;code&gt;&amp;lt;vcf-services-runtime-fqdn&amp;gt;&lt;/code&gt;&lt;/td&gt;
					&lt;td&gt;vmware-system-user&lt;/td&gt;
					&lt;td&gt;SSH&lt;/td&gt;
					&lt;td&gt;Confirm ownership before rotating&lt;/td&gt;
			&lt;/tr&gt;
	&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;Do not blindly rotate everything at once. Some accounts may be service users, some may belong to appliances that are temporarily unreachable, and some may be stale entries for components that no longer exist. Also note that domain-level credentials such as ESXi, vCenter, and NSX are owned by SDDC Manager rather than the Fleet vault — those rotate through the &lt;a href="https://cosmin.us/mastering-the-v1-credentials-api-in-sddc-manager-for-enhanced-security-management/"&gt;SDDC Manager /v1/credentials API&lt;/a&gt; instead.&lt;/p&gt;
&lt;h2 id="update-an-expired-password-account"&gt;Update an Expired Password Account&lt;/h2&gt;
&lt;p&gt;For a single account, you can do this from the VCF Operations UI by selecting the expired password account and using &lt;strong&gt;Update&lt;/strong&gt;. Enter the current working password and the new final password.&lt;/p&gt;
&lt;p&gt;For repeatable remediation, use the API equivalent below.&lt;/p&gt;
&lt;p&gt;One important CLI note: do not paste passwords with special characters directly into an inline curl or shell variable assignment. Passwords that contain an exclamation mark can be changed or rejected by the shell before they ever reach the API because &lt;code&gt;!&lt;/code&gt; may trigger history expansion in interactive shells.&lt;/p&gt;
&lt;p&gt;If you are using Bash, disable history expansion for the session:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;set&lt;/span&gt; +H
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If you are using Zsh, disable bang history expansion:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;unsetopt BANG_HIST
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The safer pattern is to prompt for the password and let &lt;code&gt;jq&lt;/code&gt; build the JSON payload.&lt;/p&gt;
&lt;p&gt;Pick one expired entry and export the key:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;KEY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&amp;lt;passwordAccountKey&amp;gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Run the update call. The &lt;code&gt;currentPassword&lt;/code&gt; must be the password that works on the target right now. The &lt;code&gt;newPassword&lt;/code&gt; is the new final password you want Fleet to set and store.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;read&lt;/span&gt; -rsp &lt;span class="s2"&gt;&amp;#34;Current password: &amp;#34;&lt;/span&gt; CUR&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;read&lt;/span&gt; -rsp &lt;span class="s2"&gt;&amp;#34;New password: &amp;#34;&lt;/span&gt; NEW&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;jq -nc --arg c &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$CUR&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; --arg n &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$NEW&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s1"&gt;&amp;#39;{currentPassword:$c,newPassword:$n}&amp;#39;&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk -X PUT &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -d @- &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/fleet-management/password-management/accounts/&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;KEY&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/password&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;|&lt;/span&gt; tee /tmp/pwupdate-task.json &lt;span class="p"&gt;|&lt;/span&gt; jq .
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;unset&lt;/span&gt; CUR NEW
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The response should create a password update workflow request. Track the task in VCF Operations, or use the returned request ID if your environment exposes the workflow request endpoint.&lt;/p&gt;
&lt;p&gt;After the task completes, query the account again. Filter by the FQDN of the appliance you just updated — it is in your tracking table or &lt;code&gt;/tmp/expired-password-accounts.json&lt;/code&gt; — so the result does not depend on how many password accounts the fleet has:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -d &lt;span class="s1"&gt;&amp;#39;{&amp;#34;applianceFqdn&amp;#34;:&amp;#34;&amp;lt;appliance-fqdn&amp;gt;&amp;#34;}&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/fleet-management/password-management/accounts/query?pageSize=200&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;|&lt;/span&gt; jq --arg KEY &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$KEY&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;.vcfPasswordAccounts[] | select(.passwordAccountKey==$KEY)&amp;#39;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;You want to see the account move out of &lt;code&gt;EXPIRED&lt;/code&gt; and show a new &lt;code&gt;expiryDate&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id="if-the-password-was-already-changed-outside-fleet"&gt;If the Password Was Already Changed Outside Fleet&lt;/h2&gt;
&lt;p&gt;This was the awkward case in my environment.&lt;/p&gt;
&lt;p&gt;If the password has already been changed manually on the appliance, the Fleet vault may still have the old value. Since 9.1 does not provide a documented vault-only update, use the actual current appliance password as the current password and rotate once more to a new final password through the UI &lt;strong&gt;Update&lt;/strong&gt; action or the API.&lt;/p&gt;
&lt;p&gt;That gives Fleet a successful workflow and lets it update:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The appliance password&lt;/li&gt;
&lt;li&gt;The stored credential&lt;/li&gt;
&lt;li&gt;The expiration metadata&lt;/li&gt;
&lt;li&gt;The password account status&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If you want to keep the exact same password that was already set manually, that is where the product gap shows up. The documented workflow wants a change from current to new.&lt;/p&gt;
&lt;h2 id="if-the-current-password-is-unknown"&gt;If the Current Password Is Unknown&lt;/h2&gt;
&lt;p&gt;If nobody knows the current password, Fleet cannot update it because the workflow requires the current password.&lt;/p&gt;
&lt;p&gt;Use the native recovery method for that appliance first. For example:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Open the VM console.&lt;/li&gt;
&lt;li&gt;Reset the account to a temporary known password.&lt;/li&gt;
&lt;li&gt;Confirm SSH or console login works.&lt;/li&gt;
&lt;li&gt;Immediately rotate from temporary to final through the UI &lt;strong&gt;Update&lt;/strong&gt; action or the Fleet API.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Do not leave the temporary password as the final state. The point is to re-enter the supported Fleet workflow as quickly as possible.&lt;/p&gt;
&lt;h2 id="if-an-entry-looks-stale"&gt;If an Entry Looks Stale&lt;/h2&gt;
&lt;p&gt;Before rotating, verify that the appliance still exists and belongs to the environment:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -d &lt;span class="s1"&gt;&amp;#39;{&amp;#34;applianceFqdn&amp;#34;:&amp;#34;&amp;lt;appliance-fqdn&amp;gt;&amp;#34;}&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/fleet-management/password-management/accounts/query?pageSize=50&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;|&lt;/span&gt; jq .
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Also check DNS and reachability from the shell where you are running the remediation:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;getent hosts &amp;lt;appliance-fqdn&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -k -I https://&amp;lt;appliance-fqdn&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;If the component has been removed but the password entry remains, do not delete database rows manually. Open a support case or use the &lt;a href="https://cosmin.us/scripted-components-cleanup-from-vcf-operations-9-1/"&gt;supported inventory cleanup path for that component&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id="verification-checklist"&gt;Verification Checklist&lt;/h2&gt;
&lt;p&gt;After the expired entries are addressed, verify the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The expired account query returns fewer entries, ideally zero for the scope you remediated.&lt;/li&gt;
&lt;li&gt;Updated accounts show a refreshed &lt;code&gt;expiryDate&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The password update workflow completed successfully.&lt;/li&gt;
&lt;li&gt;Direct login to the target appliance works with the new password.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Query all remaining expired entries:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;curl -sk &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s2"&gt;&amp;#34;Authorization: OpsToken &lt;/span&gt;&lt;span class="nv"&gt;$TOKEN&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -H &lt;span class="s1"&gt;&amp;#39;Content-Type: application/json&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; -d &lt;span class="s1"&gt;&amp;#39;{&amp;#34;status&amp;#34;:&amp;#34;EXPIRED&amp;#34;}&amp;#39;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$OPS&lt;/span&gt;&lt;span class="s2"&gt;/suite-api/api/fleet-management/password-management/accounts/query?pageSize=200&amp;#34;&lt;/span&gt; &lt;span class="se"&gt;\
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="p"&gt;|&lt;/span&gt; jq &lt;span class="s1"&gt;&amp;#39;.vcfPasswordAccounts&amp;#39;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Finally, clean up the working files, since they contain fleet inventory details such as appliance FQDNs, usernames, and password account keys:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;rm -f /tmp/expired-password-accounts.json /tmp/pwupdate-task.json
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id="final-notes"&gt;Final Notes&lt;/h2&gt;
&lt;p&gt;The main lesson is that VCF 9.1 password management needs to be treated as an update or rotation workflow. The password account state is exposed through VCF Operations Fleet Password Management, and expired entries need to be handled through the UI &lt;strong&gt;Update&lt;/strong&gt; action or the documented account query and password update API calls. This is the same rotate-only philosophy the platform has had since the &lt;a href="https://cosmin.us/changing-passwords-for-the-vrealize-suite-via-vrslcm-vrealize-suite-lifecycle-manager/"&gt;vRSLCM Locker password management&lt;/a&gt; days — the tooling changed, the workflow shape did not.&lt;/p&gt;
&lt;p&gt;For this kind of issue, I would avoid two shortcuts:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Do not use VCF 9.0 UI guidance for VCF 9.1.&lt;/li&gt;
&lt;li&gt;Do not update Fleet LCM database rows by hand.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The reliable path is to update expired accounts through the UI or API and verify the resulting password account state.&lt;/p&gt;</content:encoded></item><item><title>Scripted Components Cleanup from VCF Operations 9.1: A Step-by-Step Guide</title><link>https://cosmin.us/scripted-components-cleanup-from-vcf-operations-9-1/</link><pubDate>Wed, 03 Jun 2026 10:30:00 +0000</pubDate><dc:creator>Cosmin Trif</dc:creator><guid>https://cosmin.us/scripted-components-cleanup-from-vcf-operations-9-1/</guid><description>&lt;p&gt;If you&amp;rsquo;ve been working with VCF Operations 9.1, you may have noticed that on the &lt;strong&gt;Build&lt;/strong&gt; &amp;gt; &lt;strong&gt;Lifecycle&lt;/strong&gt; &amp;gt; &lt;strong&gt;VCF Management&lt;/strong&gt; &amp;gt; &lt;strong&gt;Components&lt;/strong&gt; page, manually-added components such as VCF Operations for Networks, Log Management, Real-Time Metrics or VCF Automation no longer have a &amp;ldquo;Delete&amp;rdquo; option available in the UI.&lt;/p&gt;
&lt;p&gt;In &lt;a href="https://cosmin.us/deploying-vcf-operations-9-from-ova/"&gt;VCF Operations 9.0&lt;/a&gt;, this was possible on the &lt;strong&gt;Fleet Management&lt;/strong&gt; &amp;gt; &lt;strong&gt;Lifecycle&lt;/strong&gt; &amp;gt; &lt;strong&gt;Components&lt;/strong&gt; page by clicking the three vertical dots next to a component and selecting &lt;strong&gt;Delete&lt;/strong&gt;. That option is gone in 9.1.&lt;/p&gt;</description><content:encoded>&lt;p&gt;If you&amp;rsquo;ve been working with VCF Operations 9.1, you may have noticed that on the &lt;strong&gt;Build&lt;/strong&gt; &amp;gt; &lt;strong&gt;Lifecycle&lt;/strong&gt; &amp;gt; &lt;strong&gt;VCF Management&lt;/strong&gt; &amp;gt; &lt;strong&gt;Components&lt;/strong&gt; page, manually-added components such as VCF Operations for Networks, Log Management, Real-Time Metrics or VCF Automation no longer have a &amp;ldquo;Delete&amp;rdquo; option available in the UI.&lt;/p&gt;
&lt;p&gt;In &lt;a href="https://cosmin.us/deploying-vcf-operations-9-from-ova/"&gt;VCF Operations 9.0&lt;/a&gt;, this was possible on the &lt;strong&gt;Fleet Management&lt;/strong&gt; &amp;gt; &lt;strong&gt;Lifecycle&lt;/strong&gt; &amp;gt; &lt;strong&gt;Components&lt;/strong&gt; page by clicking the three vertical dots next to a component and selecting &lt;strong&gt;Delete&lt;/strong&gt;. That option is gone in 9.1.&lt;/p&gt;
&lt;p&gt;Broadcom has published &lt;a href="https://knowledge.broadcom.com/external/article/441333"&gt;KB article 441333&lt;/a&gt; with an attached Python script (&lt;code&gt;cleanup_component.py&lt;/code&gt;) that fills this gap. In this post I walk through how to use it for each component type.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="prerequisites"&gt;Prerequisites&lt;/h2&gt;
&lt;p&gt;Before running the cleanup script, make sure the following are in place:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Python&lt;/strong&gt; installed on the system where you will run the script (can be any system with network access to the Fleet and VCF Management Services components).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Network access&lt;/strong&gt; to the &lt;a href="https://cosmin.us/deploying-vcf-fleet-management-9-from-ova-a-step-by-step-guide/"&gt;Fleet LCM appliance&lt;/a&gt; and VCF Services Runtime.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Credentials&lt;/strong&gt; — use username &lt;code&gt;admin&lt;/code&gt; as the value for the VCF Services Runtime username. The password is the same as the &lt;code&gt;vmware-system-user&lt;/code&gt; password.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;FQDN values&lt;/strong&gt; — replace all placeholder FQDNs such as &lt;code&gt;&amp;lt;fleet-lcm-fqdn&amp;gt;&lt;/code&gt; with the actual values. These can be obtained from &lt;strong&gt;VCF Operations UI&lt;/strong&gt; -&amp;gt; &lt;strong&gt;Build&lt;/strong&gt; -&amp;gt; &lt;strong&gt;Lifecycle&lt;/strong&gt; -&amp;gt; &lt;strong&gt;VCF Management&lt;/strong&gt; -&amp;gt; &lt;strong&gt;Components&lt;/strong&gt; (e.g., &lt;code&gt;vsp01.example.com&lt;/code&gt; for &lt;code&gt;&amp;lt;vcf-services-runtime-fqdn&amp;gt;&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Warning:&lt;/strong&gt; Prior to deletion, verify that you have a backup of any critical component data which needs to be retained, such as logs or database backups.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id="download-and-verify-the-script"&gt;Download and Verify the Script&lt;/h2&gt;
&lt;p&gt;Download &lt;code&gt;cleanup_component.py&lt;/code&gt; from &lt;a href="https://knowledge.broadcom.com/external/article/441333"&gt;KB 441333&lt;/a&gt; (attached at the bottom of the article).&lt;/p&gt;
&lt;p&gt;Verify the SHA256 checksum before running:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sha256sum cleanup_component.py
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Expected checksum:&lt;/p&gt;
&lt;pre tabindex="0"&gt;&lt;code&gt;27A82D393333D17261EF1E8629A69AD6C40659A8537DF961DE95E545FE656F5B
&lt;/code&gt;&lt;/pre&gt;&lt;hr&gt;
&lt;h2 id="what-the-script-does"&gt;What the Script Does&lt;/h2&gt;
&lt;p&gt;The LCM component cleanup script enables administrators to manage Day-N deployed components visible in the VCF Operations UI:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;List components&lt;/strong&gt; — View all deployed components currently eligible for removal.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Delete components&lt;/strong&gt; — Safely remove inoperable or misconfigured components from the system so they can be redeployed later through VCF Operations.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Important notes:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;If a component is running a version older than 9.1, it is removed solely from the Fleet lifecycle. The script does not delete the corresponding infrastructure deployments such as vCenter or VCF Services Runtime.&lt;/li&gt;
&lt;li&gt;VSP refers to VCF Management Services.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id="1-ova-components"&gt;1. OVA Components&lt;/h2&gt;
&lt;p&gt;Deleting an OVA-based deployment removes it from Fleet lifecycle and also deletes the associated virtual machines from vCenter.&lt;/p&gt;
&lt;h3 id="list-components-deployed-on-vcenter"&gt;List Components Deployed on vCenter&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py list ova-component --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="delete-component-deployed-on-vcenter"&gt;Delete Component Deployed on vCenter&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py delete ova-component --component-id &amp;lt;ova-component-id&amp;gt; --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt; --vcenter-username &amp;lt;vcenter-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;The script will prompt for the required passwords. Once confirmed, the OVA component will be removed from Fleet lifecycle and its associated VMs deleted from vCenter.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="2-vcf-management-services-vsp-components"&gt;2. VCF Management Services (VSP) Components&lt;/h2&gt;
&lt;p&gt;Deleting a VSP-based deployment removes it from Fleet lifecycle and also deletes the associated deployment from VCF Services Runtime.&lt;/p&gt;
&lt;h3 id="list-components-deployed-on-vcf-services-runtime"&gt;List Components Deployed on VCF Services Runtime&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py list vsp-component --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h3 id="delete-component-deployed-on-vcf-services-runtime"&gt;Delete Component Deployed on VCF Services Runtime&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py delete vsp-component --component-id &amp;lt;vsp-component-id&amp;gt; --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h2 id="3-vcf-automation"&gt;3. VCF Automation&lt;/h2&gt;
&lt;p&gt;VCF Automation is a multi-component deployment consisting of:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;VCF Automation&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Migration Service Engine&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;VCF Automation VCF Services Runtime&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Important:&lt;/strong&gt; You must remove VCF Automation and the Migration Service Engine &lt;strong&gt;before&lt;/strong&gt; you remove the VCF Automation VCF Services Runtime. Follow the order below.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h3 id="step-1-remove-vcf-automation-and-migration-service-engine"&gt;Step 1: Remove VCF Automation and Migration Service Engine&lt;/h3&gt;
&lt;h4 id="list-components-deployed-on-vcf-services-runtime-1"&gt;List Components Deployed on VCF Services Runtime&lt;/h4&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py list vsp-component --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id="delete-component-deployed-on-the-vcf-automation-vcf-services-runtime"&gt;Delete Component Deployed on the VCF Automation VCF Services Runtime&lt;/h4&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py delete vsp-component --component-id &amp;lt;vsp-component-id&amp;gt; --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt; --vcfa-vcf-services-runtime-fqdn &amp;lt;vcfa-vcf-services-runtime-fqdn&amp;gt; --vcfa-vcf-services-runtime-username &amp;lt;vcfa-vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Run the delete command for both the VCF Automation component and the Migration Service Engine component before proceeding to Step 2.&lt;/p&gt;
&lt;h3 id="step-2-remove-the-vcf-automation-vcf-services-runtime"&gt;Step 2: Remove the VCF Automation VCF Services Runtime&lt;/h3&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Warning:&lt;/strong&gt; Deleting a VCF Services Runtime requires this script to be executed as the &lt;strong&gt;root&lt;/strong&gt; user inside the &lt;strong&gt;SDDC Manager&lt;/strong&gt; virtual machine.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h4 id="list-vcf-services-runtime"&gt;List VCF Services Runtime&lt;/h4&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py list vsp-cluster --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h4 id="delete-vcf-services-runtime"&gt;Delete VCF Services Runtime&lt;/h4&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;python cleanup_component.py delete vsp-cluster --component-id &amp;lt;vsp-cluster-id&amp;gt; --fleet-fqdn &amp;lt;fleet-lcm-fqdn&amp;gt; --vcf-services-runtime-fqdn &amp;lt;vcf-services-runtime-fqdn&amp;gt; --vcf-services-runtime-username &amp;lt;vcf-services-runtime-username&amp;gt; --vcenter-username &amp;lt;vcenter-username&amp;gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;hr&gt;
&lt;h2 id="summary"&gt;Summary&lt;/h2&gt;
&lt;p&gt;The &lt;code&gt;cleanup_component.py&lt;/code&gt; script is a handy workaround for the missing &amp;ldquo;Delete&amp;rdquo; option in VCF Operations 9.1. The key things to remember:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Back up&lt;/strong&gt; any critical component data before running delete operations.&lt;/li&gt;
&lt;li&gt;For &lt;strong&gt;OVA components&lt;/strong&gt;, deletion also removes VMs from vCenter.&lt;/li&gt;
&lt;li&gt;For &lt;strong&gt;VSP components&lt;/strong&gt;, deletion also removes the deployment from VCF Services Runtime.&lt;/li&gt;
&lt;li&gt;For &lt;strong&gt;VCF Automation&lt;/strong&gt;, always delete VCF Automation and Migration Service Engine &lt;strong&gt;first&lt;/strong&gt;, then delete the VCF Services Runtime — and run that last step as &lt;strong&gt;root&lt;/strong&gt; on the SDDC Manager VM.&lt;/li&gt;
&lt;li&gt;Always verify the script checksum before running it.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;After removing components, also check Fleet password management for leftover or expired password accounts — I covered that remediation in &lt;a href="https://cosmin.us/fixing-expired-password-sync-in-vcf-9-1-fleet-management/"&gt;Fixing Expired Password Sync in VCF 9.1 Fleet Management&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;For the official script download and additional details, refer to &lt;a href="https://knowledge.broadcom.com/external/article/441333"&gt;Broadcom KB article 441333&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;I hope this helps!&lt;/p&gt;</content:encoded></item><item><title>Deploying VCF Fleet Management 9 from OVA: A Step-by-Step Guide</title><link>https://cosmin.us/deploying-vcf-fleet-management-9-from-ova-a-step-by-step-guide/</link><pubDate>Thu, 09 Oct 2025 13:04:27 +0000</pubDate><dc:creator>Cosmin Trif</dc:creator><guid>https://cosmin.us/deploying-vcf-fleet-management-9-from-ova-a-step-by-step-guide/</guid><description>&lt;p&gt;Deploying the VCF Fleet Management (Operations / Fleet Manager) appliance using an OVA is a common approach when standing up a &lt;a href="https://cosmin.us/deploy-a-vcf-9-instance-in-an-existing-vcf-fleet/"&gt;new VMware Cloud Foundation (VCF) 9 environment&lt;/a&gt; or re-deploying the component. Below is a refined guide covering the deployment, configuration, and integration steps.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="1-prerequisites--downloading-binaries"&gt;1. Prerequisites &amp;amp; Downloading Binaries&lt;/h3&gt;
&lt;p&gt;Before you deploy the OVA:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Log into the Broadcom Support Portal and obtain the required binaries and OVA files for the Fleet Manager. Direct link can be found &lt;a href="https://support.broadcom.com/group/ecx/productfiles?displayGroup=VMware%20Cloud%20Foundation%209&amp;amp;release=9.0.0.0&amp;amp;os=&amp;amp;servicePk=529536&amp;amp;language=EN&amp;amp;groupId=528990&amp;amp;viewGroup=true"&gt;here&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-21-1024x368.webp" alt="Portal screenshot shows VMware Cloud Foundation 9.0.0.0 fleet management OVA details." width="1024" height="368" loading="lazy" decoding="async"&gt;
&lt;/p&gt;</description><content:encoded>&lt;p&gt;Deploying the VCF Fleet Management (Operations / Fleet Manager) appliance using an OVA is a common approach when standing up a &lt;a href="https://cosmin.us/deploy-a-vcf-9-instance-in-an-existing-vcf-fleet/"&gt;new VMware Cloud Foundation (VCF) 9 environment&lt;/a&gt; or re-deploying the component. Below is a refined guide covering the deployment, configuration, and integration steps.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="1-prerequisites--downloading-binaries"&gt;1. Prerequisites &amp;amp; Downloading Binaries&lt;/h3&gt;
&lt;p&gt;Before you deploy the OVA:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Log into the Broadcom Support Portal and obtain the required binaries and OVA files for the Fleet Manager. Direct link can be found &lt;a href="https://support.broadcom.com/group/ecx/productfiles?displayGroup=VMware%20Cloud%20Foundation%209&amp;amp;release=9.0.0.0&amp;amp;os=&amp;amp;servicePk=529536&amp;amp;language=EN&amp;amp;groupId=528990&amp;amp;viewGroup=true"&gt;here&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-21-1024x368.webp" alt="Portal screenshot shows VMware Cloud Foundation 9.0.0.0 fleet management OVA details." width="1024" height="368" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Ensure DNS, IP addresses, and network configurations for the appliance are defined (fully qualified domain name, subnet, gateway, DNS servers, etc.).&lt;/li&gt;
&lt;li&gt;Ensure your vCenter / ESXi environment meets the compatibility and resource requirements (CPU, RAM, storage).&lt;/li&gt;
&lt;li&gt;Map the hostname, IP, and other parameters you’ll use for the Fleet Manager appliance.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id="2-deploy-the-ova-via-vcenter"&gt;2. Deploy the OVA via vCenter&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;In vCenter, choose Deploy OVA (or OVF template).&lt;/li&gt;
&lt;li&gt;Select the Fleet Manager OVA file (downloaded from the portal).&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-22-1024x770.webp" alt="Deploy OVF template wizard step showing file upload for VCF Fleet Manager 9." width="1024" height="770" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Provide a VM name and select the inventory location (folder, cluster, or host).&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-23-1024x768.webp" alt="Deploying VCF Fleet Management 9: Select VM name and folder location." width="1024" height="768" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Choose the compute resource (ESXi host or cluster) for placement.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-24-1024x770.webp" alt="Select a compute resource in the Deploy OVF Template wizard." width="1024" height="770" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Review the details and click Next.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-25-1024x766.webp" alt="Deploy OVF Template wizard, review details step, showing product, version, size, and provisioning options." width="1024" height="766" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Select the target storage (datastore) for the VM’s disks.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-26-1024x766.webp" alt="Deploy OVF Template wizard, Step 5: Select storage, showing storage options and compatibility check." width="1024" height="766" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Choose the network(s) to attach the appliance.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-27-1024x762.webp" alt="Selecting network destination for appliance deployment in VCF Fleet Management 9." width="1024" height="762" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;When prompted, fill in the customization metadata:
&lt;ul&gt;
&lt;li&gt;Root / admin passwords&lt;/li&gt;
&lt;li&gt;FQDN / hostname&lt;/li&gt;
&lt;li&gt;IP address, subnet mask, gateway&lt;/li&gt;
&lt;li&gt;DNS server(s), domain name&lt;/li&gt;
&lt;li&gt;NTP server(s)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-28-1024x770.webp" alt="Customize template step shows password fields, hostname, and SSH service options for deploying VCF Fleet Management 9." width="1024" height="770" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Review the settings and click Finish to deploy the OVA.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-29-1024x765.webp" alt="Deploy OVF Template wizard step: Ready to complete." width="1024" height="765" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Power on the appliance once deployment completes.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h3 id="3-initial-configuration--connection"&gt;3. Initial Configuration &amp;amp; Connection&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;Access the Operations appliance UI admin interface via its FQDN (e.g. https://&amp;lt;fleet-mgr-fqdn/admin&amp;gt;) in a browser.&lt;/li&gt;
&lt;li&gt;In the UI, go to the Fleet Management or Connect tile.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-30-1024x293.webp" alt="Cluster Status as Online and Fleet Management as Not Connected." width="1024" height="293" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Enter the IP / FQDN and admin credentials for the new appliance, then test the connection.&lt;/li&gt;
&lt;li&gt;Accept any SSL / certificate warnings as needed.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-31-1024x674.webp" alt="Fleet Management Node Details step shows fields for Node Address, Admin Password, and Certificate Information." width="1024" height="674" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Supply credentials for your primary &lt;a href="https://cosmin.us/deploying-vcf-operations-9-from-ova/"&gt;VCF Operations node&lt;/a&gt; (or cluster).&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-32-1024x671.webp" alt="Adding Fleet Management Node: Enter primary node password." width="1024" height="671" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Wait for a couple of minutes, the Fleet Management box should reflect as Connected&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-33-1024x286.webp" alt="Fleet Management box shows “Connected” status." width="1024" height="286" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="4-import--add-existing-components"&gt;4. Import / Add Existing Components&lt;/h3&gt;
&lt;p&gt;Once Fleet Manager is reachable, the next steps are to integrate it with any existing Operations / Logs instances:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Go to Operations user interface -&amp;gt; Fleet Management -&amp;gt; Lifecycle -&amp;gt; VCF Management -&amp;gt; Under operations click Import. this should start a task&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-34-1024x376.webp" alt="VCF Management deployment step in Fleet Management, with options to select Installation Type: New Install, Import, or Import from legacy Fleet Management." width="1024" height="376" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Provide the credentials, FQDN, and vCenter info. Submit the job.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-35-1024x525.webp" alt="VCF Management operations step showing credentials and vCenter info for deployment." width="1024" height="525" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Review the details and click submit. This will start a task to import VCF Operations under management.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-36-1024x373.webp" alt="VCF Management task progress with stages labeled and durations." width="1024" height="373" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id="5-configure-the-depot--deploy-vcf-automation-optional-next-step"&gt;5. Configure the Depot &amp;amp; Deploy VCF Automation (Optional Next Step)&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;In the UI, configure the Depot / Binary Management settings (online or offline depot).&lt;/li&gt;
&lt;li&gt;After the depot is set up and binaries are synchronized, you can use the UI to deploy VCF Automation, the next piece of your stack.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id="6-verification--post-deployment-checks"&gt;6. Verification &amp;amp; Post-Deployment Checks&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;In Lifecycle → Overview / Components, confirm that Operations and Fleet Manager show successful integration.&lt;/li&gt;
&lt;li&gt;Check connectivity, dashboard health, metrics, and logs to ensure no errors or warnings.&lt;/li&gt;
&lt;li&gt;Verify that future Lifecycle / patching tasks are visible and manageable under this new Fleet Manager hierarchy.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;</content:encoded></item><item><title>Upgrading VCF 9.0.0 Fleet Management to 9.0.1: A Step-by-Step Guide</title><link>https://cosmin.us/upgrading-vcf-9-0-0-fleet-management-to-9-0-1-a-step-by-step-guide/</link><pubDate>Thu, 09 Oct 2025 12:49:44 +0000</pubDate><dc:creator>Cosmin Trif</dc:creator><guid>https://cosmin.us/upgrading-vcf-9-0-0-fleet-management-to-9-0-1-a-step-by-step-guide/</guid><description>&lt;p&gt;Upgrading your VCF Fleet Management from version 9.0.0 to 9.0.1 is a relatively modest patch update, but careful execution ensures minimal disruption. Below is a step-by-step guide to walk you through the process safely.&lt;/p&gt;
&lt;h3 id="download-the-patch"&gt;Download the Patch&lt;/h3&gt;
&lt;p&gt;Download the patch by going to Fleet Management -&amp;gt; Binary Management -&amp;gt; Patch Binaries -&amp;gt; Select fleet management -&amp;gt; Download.&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-49-1024x279.webp" alt="VMware Cloud Foundation dashboard with options for upgrading, patching, and installing binaries for VCF 9.0.1." width="1024" height="279" loading="lazy" decoding="async"&gt;
&lt;/p&gt;</description><content:encoded>&lt;p&gt;Upgrading your VCF Fleet Management from version 9.0.0 to 9.0.1 is a relatively modest patch update, but careful execution ensures minimal disruption. Below is a step-by-step guide to walk you through the process safely.&lt;/p&gt;
&lt;h3 id="download-the-patch"&gt;Download the Patch&lt;/h3&gt;
&lt;p&gt;Download the patch by going to Fleet Management -&amp;gt; Binary Management -&amp;gt; Patch Binaries -&amp;gt; Select fleet management -&amp;gt; Download.&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-49-1024x279.webp" alt="VMware Cloud Foundation dashboard with options for upgrading, patching, and installing binaries for VCF 9.0.1." width="1024" height="279" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;h2 id="upload--stage-the-patch"&gt;Upload &amp;amp; Stage the Patch&lt;/h2&gt;
&lt;p&gt;Once the download is complete go to Settings -&amp;gt; System Patches -&amp;gt; New Patch&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-50-1024x372.webp" alt="VMware Cloud Foundation Operations interface with a focus on upgrades, recommending a snapshot before initiating an upgrade." width="1024" height="372" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;p&gt;Select the Patch that was just downloaded and click Next to proceed&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-51-1024x769.webp" alt="Select patch version 9.0.1.0, click Next to proceed." width="1024" height="769" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;p&gt;Review the &lt;a href="https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-0/release-notes/vmware-cloud-foundation-9-0-1-release-notes/vcf-operations-9-0-1-0000.html"&gt;release notes&lt;/a&gt; and click Install&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-52-1024x765.webp" alt="Review and Install step for VMware Cloud Foundation Fleet Management 9.0.1.0 Maintenance Patch 1." width="1024" height="765" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;p&gt;This will create a Task that patches the appliance. Click Return to Tasks&lt;/p&gt;
&lt;p&gt;&lt;img src="https://cosmin.us/images/2025/10/image-53-1024x384.webp" alt="Upgrading VCF 9.0.0 Fleet Management to 9.0.1: Task in Stage 1 of selfpatchinstall." width="1024" height="384" loading="lazy" decoding="async"&gt;
&lt;/p&gt;
&lt;h2 id="wait--confirm"&gt;Wait &amp;amp; Confirm&lt;/h2&gt;
&lt;p&gt;Allow several minutes for the task to complete. Do not interrupt the process. Once the patch job is finished, go to the Settings tab and verify that the Fleet Management component now shows 9.0.1.&lt;/p&gt;
&lt;p&gt;The SDDC Manager side of the 9.0.1 patch is covered in &lt;a href="https://cosmin.us/upgrading-vcf-9-0-sddc-manager-to-9-0-1-a-step-by-step-guide/"&gt;this guide&lt;/a&gt;, and if PostgreSQL won&amp;rsquo;t start after a reboot on 9.0.1, see &lt;a href="https://cosmin.us/fixing-postgresql-wont-start-after-reboot-on-vcf-operations-fleet-management-9-0-1-and-automating-it/"&gt;this fix&lt;/a&gt;.&lt;/p&gt;</content:encoded></item><item><title>Fixing “PostgreSQL won’t start after reboot” on VCF Operations Fleet Management 9.0.1 — and automating it</title><link>https://cosmin.us/fixing-postgresql-wont-start-after-reboot-on-vcf-operations-fleet-management-9-0-1-and-automating-it/</link><pubDate>Tue, 30 Sep 2025 21:23:32 +0000</pubDate><dc:creator>Cosmin Trif</dc:creator><guid>https://cosmin.us/fixing-postgresql-wont-start-after-reboot-on-vcf-operations-fleet-management-9-0-1-and-automating-it/</guid><description>&lt;p&gt;When &lt;a href="https://cosmin.us/upgrading-vcf-9-0-0-fleet-management-to-9-0-1-a-step-by-step-guide/"&gt;upgrading the VCF Operations Fleet Management appliance to 9.0.1&lt;/a&gt;, you might hit a reboot surprise: vPostgres fails to start, and the fleet-management certificate is regenerated. You’ll see journal errors like:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;pg_ctl: could not open PID file &amp;#34;/var/vmware/vpostgres/current/pgdata/postmaster.pid&amp;#34;: Permission denied
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;systemd[1]: vpostgres.service: Control process exited, code=exited, status=1/FAILURE
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Broadcom’s KB confirms the symptoms and provides the manual steps to fix permissions on the Postgres data dir and to normalize the regenerated cert/key filenames, followed by service restarts. If you are also moving the analytics side to the same release, I covered &lt;a href="https://cosmin.us/upgrading-vcf-operations-to-9-0-1-using-a-pak-file/"&gt;upgrading VCF Operations to 9.0.1 using a .PAK file&lt;/a&gt; separately.&lt;/p&gt;</description><content:encoded>&lt;p&gt;When &lt;a href="https://cosmin.us/upgrading-vcf-9-0-0-fleet-management-to-9-0-1-a-step-by-step-guide/"&gt;upgrading the VCF Operations Fleet Management appliance to 9.0.1&lt;/a&gt;, you might hit a reboot surprise: vPostgres fails to start, and the fleet-management certificate is regenerated. You’ll see journal errors like:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-text" data-lang="text"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;pg_ctl: could not open PID file &amp;#34;/var/vmware/vpostgres/current/pgdata/postmaster.pid&amp;#34;: Permission denied
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;systemd[1]: vpostgres.service: Control process exited, code=exited, status=1/FAILURE
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Broadcom’s KB confirms the symptoms and provides the manual steps to fix permissions on the Postgres data dir and to normalize the regenerated cert/key filenames, followed by service restarts. If you are also moving the analytics side to the same release, I covered &lt;a href="https://cosmin.us/upgrading-vcf-operations-to-9-0-1-using-a-pak-file/"&gt;upgrading VCF Operations to 9.0.1 using a .PAK file&lt;/a&gt; separately.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="manual-steps"&gt;(manual steps)&lt;/h2&gt;
&lt;ol&gt;
&lt;li&gt;Fix pgdata permissions&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;chmod &lt;span class="m"&gt;700&lt;/span&gt; /var/vmware/vpostgres/current/pgdata/
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;ol start="2"&gt;
&lt;li&gt;Normalize the regenerated certificate &amp;amp; key in /opt/vmware/vlcm/cert by renaming the timestamped server.crt.* / server.key.* back to server.crt / server.key, then restart services (nginx, vrlcm-server.service) and verify status/logs.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h2 id="one-shot-automated-remediation-idempotent"&gt;One-shot automated remediation (idempotent)&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;Run this as root on the Fleet Management appliance (e.g., via SSH). It:&lt;/p&gt;
&lt;/blockquote&gt;
&lt;ul&gt;
&lt;li&gt;Fixes the Postgres data directory permissions&lt;/li&gt;
&lt;li&gt;Backs up any existing certs&lt;/li&gt;
&lt;li&gt;Picks the newest timestamped server.crt.* and server.key.* and makes them active&lt;/li&gt;
&lt;li&gt;Ensures sane file permissions&lt;/li&gt;
&lt;li&gt;Restarts nginx and vrlcm-server.service&lt;/li&gt;
&lt;li&gt;Shows health checks + log tail hints&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="cp"&gt;#!/usr/bin/env bash
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# vcf-ops-fleet-901-postgres-cert-fix.sh&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Automates KB 412351 remediation on VCF Operations Fleet Management 9.0.1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Source: https://knowledge.broadcom.com/external/article/412351&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;set&lt;/span&gt; -euo pipefail
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;LOG&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;/var/log/vrlcm/vmware_vrlcm.log&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;PGDATA&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;/var/vmware/vpostgres/current/pgdata&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;CERT_DIR&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;/opt/vmware/vlcm/cert&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;BACKUP_DIR&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CERT_DIR&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/backup-&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;date +%Y%m%d-%H%M%S&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[*] Ensuring Postgres data dir permissions (700) …&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; -d &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$PGDATA&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; chmod &lt;span class="m"&gt;700&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$PGDATA&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[+] chmod 700 &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PGDATA&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; done&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;else&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[!] &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;PGDATA&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; not found; continuing…&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;fi&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[*] Working in &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CERT_DIR&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; …&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;cd&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$CERT_DIR&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;{&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[!] &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CERT_DIR&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; not found&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nb"&gt;exit&lt;/span&gt; 1&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="o"&gt;}&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Back up current active certs (if any)&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;mkdir -p &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$BACKUP_DIR&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;for&lt;/span&gt; f in server.crt server.key&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; -f &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$f&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; cp -a &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$f&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;BACKUP_DIR&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[+] Backed up &lt;/span&gt;&lt;span class="nv"&gt;$f&lt;/span&gt;&lt;span class="s2"&gt; to &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;BACKUP_DIR&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="k"&gt;fi&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;done&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Find newest timestamped variants&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;NEWEST_CRT&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;ls -1t server.crt.* 2&amp;gt;/dev/null &lt;span class="p"&gt;|&lt;/span&gt; head -n1 &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true&lt;/span&gt;&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nv"&gt;NEWEST_KEY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="k"&gt;$(&lt;/span&gt;ls -1t server.key.* 2&amp;gt;/dev/null &lt;span class="p"&gt;|&lt;/span&gt; head -n1 &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true&lt;/span&gt;&lt;span class="k"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; -z &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$NEWEST_CRT&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; -z &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$NEWEST_KEY&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[!] Could not find timestamped server.crt.* and/or server.key.* files.&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34; If the KB&amp;#39;s certificate regeneration occurred, these should exist. Aborting.&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt; &lt;span class="nb"&gt;exit&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="k"&gt;fi&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[*] Activating newest certificate and key:&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34; CRT: &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;NEWEST_CRT&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34; KEY: &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;NEWEST_KEY&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Move into place atomically&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cp -a &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$NEWEST_CRT&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; server.crt
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;cp -a &lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;&lt;span class="nv"&gt;$NEWEST_KEY&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt; server.key
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Tighten permissions&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;chmod &lt;span class="m"&gt;600&lt;/span&gt; server.key &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;chmod &lt;span class="m"&gt;644&lt;/span&gt; server.crt &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# Optionally set ownership if required by your distro:&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="c1"&gt;# chown root:root server.crt server.key&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[*] Restarting services (nginx, vrlcm-server.service) …&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;systemctl restart nginx
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;systemctl restart vrlcm-server.service
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[*] Waiting 10s and checking status …&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sleep &lt;span class="m"&gt;10&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;systemctl --no-pager --full status vrlcm-server.service &lt;span class="p"&gt;|&lt;/span&gt; sed -n &lt;span class="s1"&gt;&amp;#39;1,50p&amp;#39;&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;true&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[i] Tail the Lifecycle Manager log for readiness:&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34; tail -f &lt;/span&gt;&lt;span class="si"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;LOG&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34;[✓] Remediation attempted. If issues persist, also verify vpostgres status:&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;&amp;#34; systemctl status vpostgres.service&amp;#34;&lt;/span&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Save this as vcf-ops-fleet-901-postgres-cert-fix.sh, then:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;sudo bash vcf-ops-fleet-901-postgres-cert-fix.sh
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;blockquote&gt;
&lt;p&gt;Why not rename with mv? Using cp -a keeps the timestamped copies intact (for quick rollback) while still placing active server.crt / server.key in the expected paths.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id="verification-checklist"&gt;Verification checklist&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;systemctl status vrlcm-server.service shows active (running).&lt;/li&gt;
&lt;li&gt;UI/API flows that depend on Lifecycle Manager respond normally.&lt;/li&gt;
&lt;li&gt;Optional: confirm vpostgres.service is active (running) and that the PID file error is gone.&lt;/li&gt;
&lt;li&gt;tail -f /var/log/vrlcm/vmware_vrlcm.log shows normal startup without repeated cert/permission complaints.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id="faq"&gt;FAQ&lt;/h2&gt;
&lt;p&gt;Q: Does this script change ownership of certs?&lt;/p&gt;
&lt;p&gt;A: The KB doesn’t require ownership changes—only renaming the timestamped files and restarting services. The script tightens file perms (600 for key, 644 for cert) which is generally safe. Adjust ownership in environments with stricter policies.&lt;/p&gt;
&lt;p&gt;Q: Can I safely re-run the script?&lt;/p&gt;
&lt;p&gt;A: Yes. It’s idempotent: it always selects the newest timestamped cert/key, backs up existing actives, and restarts services.&lt;/p&gt;
&lt;p&gt;Q: Why fix Postgres permissions?&lt;/p&gt;
&lt;p&gt;A: The KB explicitly calls for chmod 700 on the Postgres pgdata directory to resolve the PID file “Permission denied” during startup.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Broadcom KB 412351 — “PostgreSQL Service fails to start after rebooting Fleet Management appliance post VCF Operations 9.0.1 upgrade” (symptoms, manual steps, service restarts).&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;</content:encoded></item></channel></rss>