No REST For The RESTful: Lifecycle Management APIs For Horizon

Omnissa's 2406 release of Horizon introduces Lifecycle Management (LCM) APIs to automate installs and upgrades of Connection Servers. It's an expansion of the Horizon Server API to include new RESTful API endpoints for managing updates.  Connection Servers where these endpoints are called upon leverage Microsoft’s WinRM service to communicate with a target machine, enabling the remote execution of a server installer package pulled from a web server. Practically speaking these LCM APIs allow Horizon admins to automate lifecycle management of Connection and Enrollment Servers using REST API calls, saving time and avoiding manual errors. 

Below is a video recording of an upgrade from Horizon 2312 to 2406 using the LCM APIs.  Postman is used to execute REST API calls against a Connection Server which in turn executes the upgrade on the target machine leveraging the WinRM service.  As the upgrade initiates on the target machine you'll see the download of the installer from a local web server followed by the silent execution of the Connection Server installer.  While the upgrade runs a status of the overall process is obtained through the retrieve-installer-status endpoint.

It's easy to see how a process like this could be quite helpful during a downtime. Say you have 7 Connection Servers in a POD to upgrade. You could easily knock out two upgrades at a time or even all 7 of them at once if you wanted to. This saves time during a critical update window while also eliminating the risk of manual error. Further, there's options to handle fresh Horizon installs as well, a capability that pairs wonderfully with LCM APIs support of Omnissa's Teraform provider for Horizon 8.  Finally, there's future plans to extend it's functionality to other Horizon infrastructure, so Horizon admins have plenty of motivation to explore these LCM APIs today.  

Guidance On LCM API Adoption

This post provides detailed guidance on the adoption of these new LCM APIs for Horizon. It builds upon previous articles within this No REST For The RESTful series, assuming familiarity with the Horizon Server API and RESTful principles that guide it's use.  If you're not already familiar with the Horizon Server API I highly recommend reading, No REST For The RESTful: Omnissa's Horizon Server API.  It will provide the foundational understanding necessary for this articles extensive review of the LCM API endpoints.  After this review we'll cover prerequisites for the LCM APIs, including technical deep dives on the current web server and WinRM requirements.  

Exploring The Lifecycle Management Endpoints Through Postman

While the official documentation explicitly details the different LCM API calls, to simplify their exploration you can import my Postman collection, Horizon Server API Shenanigans. The collection demonstrates various calls to the Horizon Server API, including the newer LCM APIs. You can easily import this collection into your own Postman environment by clicking on this Run in Postman button:

Once the import is complete you’ll have access to the entire Horizon Server API Shenanigans collection, including the, “Horizon LCM,” folder.

For more guidance on this Horizon Server API Shenanigans Postman collection, again, check out, No REST For The RESTful: Omnissa’s Horizon Server API.  We'll be using preconfigured calls within this collection's Horizon LCM folder as we explore the LCM APIs. 

Configuring The LCM Management Role

Once you’ve copied this collection into your own instance of Postman you can begin to tailor the calls for your Horizon environment. At minimum you’ll need to update key collection variables, including the baseUrl for your Connection server, along with the username, password and domain for the Horizon administrator you intend to use for the LCM APIs.

You'll use these credentials when retrieving an access token using the Login endpoint. 

Next, you’ll need to provide this Horizon admin account with the LCM management role. You can create this role on your server by leveraging the /config/v1/roles endpoint. Navigate to the LCM Configuration folder within Horizon Server API Shenanigans. There you’ll find a request named Create LCM Management Role.

After executing this call successfully you’ll see the new role on your Connection server. You can assign this role to your Horizon admin account either by leveraging the /config/v1/permissions endpoint or navigating through the Horizon admin interface and assigning it manually.

Registering The Server Installer Package

Next comes the critical task of registering the server installer package. This will instruct target machines where to download both the Connection Server installer and the LCM.zip bundle.

For the fileUrl parameter we’ll enter in the URL to the installer on our web server. Then, in the body of the request you populate metadata about the installer, including it’s build number, checksum and file size. A sample called Register Server Installer Package is available with the LCM configuration folder. For my own lab I used Horizon Connection Server 2406, build number 10070698069, so the request body is configured as follows:

To use a different version of Horizon Connection Server you’ll have to update these values accordingly. Upon successful execution you’ll get a 200 response, with a display of the new installer package ID.

You’ll leverage this package ID when you go on to execute an install or upgrade.

Validating Target Machine Requirements

There’s 3 different pre-check endpoints you can use to validate perquisites on the target machine regarding Active Directory, vCenter and general system requirements. These calls are relatively straightforward to configure and can be found within the LCM Target Machine Validation folder of Horizon Server API Shenanigans.  For example, to validate general system requirements we need only provide the FQDN of the target machine and Connection Server version we’re trying to upgrade to:

The Active Directory call requires nearly the same configuration, with the added requirement of a FQDN for AD:

Similarly, the vCenter pre-requisite check for vCenter requires 2 additional data points specific to the vCenter environment, the vCenter URL and vCenter version:

After receiving positive confirmation from these system check APIs we can confidently proceed with pushing out an install or upgrade. 

Executing An Upgrade

With our installer package created and prerequisite checks passed performing an actual upgrade is a relatively straight forward process. We need to provide the FQDN of the target machine along with the server installer package ID and admin credentials.

The admin credentials are going to map to the Horizon admin account we’ve entitled to the LCM role, including their username, domain and password.  This account will also need local admin access to the target machine. The installer package ID was initially provided in the response after a successful registration. If you didn’t copy it then, you can list all the registered builds and their associated attributes with the /config/v1/server-installer-packages endpoint.

After providing this ID and admin credentials to the upgrade-connection-server API you’re off to the races, with a fully automated Horizon Connection Server upgrade. Successful execution against the upgrade-connection-server endpoint will yield a 204 response code and within a matter of seconds you’ll see the upgrade begin to execute on the target machine. Task manager on the target machine will display a bunch of install activity, starting with a spike of PowerShell.

Executing A Fresh Install

 
While executing a fresh Connection Server installation using the LCM APIs isn’t rocket science there is a bit more data entry required in terms of properly populating the request body with required values. This reflects the more complex nature of a fresh install that requires additional input such as specifying the deployment type and recovery passwords.  If you navigate to the root of the Horizon LCM folder you'll find an example call to the install-connection-server endpoint.  

I’d say the most daunting part of this execution is providing the SID of the admin account. To simplify this process I’ve added a special call entitled, “Fetch Admin SID,” that displays attributes of the Horizon admin account currently used to interact with the Horizon Server API. It also assigns this account's SID to a variable called lcm_admin_cid.  This variable in turn is used for a sample call included in the LCM Sequence --> Install Sequence folder.  

More guidance on the required values for the install-connect-server endpoint is available from the Swagger instance on your local Connection Server:

This Swagger info, combined with an understanding of general install requirements, should get you everything you need for a successful install.

The LCM Sequence Folder 

The LCM Sequence folder within the Horizon LCM folder is my attempt to simplify as much as possible the execution of LCM API endpoints for a Horizon admin's specific environment.  Most of the required attributes to adapt these calls are configured through the collection scope variables for Horizon Server API Shenanigans.  These includes the baseUrl and horizon admin credentials.  

There's also some LCM specific variables like lcm_target_machine or lcm_target_cs_version.

The overall idea of these folders is to illustrate the natural sequence of calls one might execute for a typical upgrade or fresh install.  More details are available on the folders overview tab within Postman. 

The Prerequisites At A High Level

To successfully execute these LCM APIs you need a Connection Server running at least Horizon 2406, the first version to offer these APIs. You’ll also need a Horizon admin account with special LCM management privileges as well as local administrative access to target machines. For the actual execution of the installer you’ll need the Windows Remote Management (WinRM) service running on both the target machine getting updated and the Horizon Connection server where the APIs are invoked. Most notably, you’ll need a web server to host the Connection Server installer and LCM bundle originally downloaded from Customer Connect. These requirements are called out in the Omnissa Horizon 8 Installation And Upgrade guide.

The current requirement for a web server with a legitimate SSL cert is tough, as it falls slightly outside the required skill set of your typical Horizon admin.  Though future releases of the LCM API will support file shares for hosting the installer, in the current release a web server is an absolutely necessary. Fortunately, Horizon admins do typically have access to Windows servers and the following guidance will walk you through accommodating the web server requirement using Microsoft IIS.  Along with digging deeper into this web server pre-requisite we'll cover some WinRM deployment considerations and testing options.

Accommodating The Web Host Requirement With Microsoft's Internet Information Server (IIS) 

If you already have access to a web host, awesome possum, use it!  For those who don't I've put together this simple to follow recipe for meeting the requirement with Microsoft IIS.  The setup isn't rocket science, but there are some unforgiving aspects of the process we need to get right in order to test the LCM API successfully, particularly in regard to the SSL cert.  Three major steps to setting up a fresh IIS instance are: 

          Setup IIS through Server Administrator 

          Add a virtual directory to the default web site 

          Add a valid SSL certificate

I've gone ahead and broken down each of these tasks in the following sections.  (Note: a default install of IIS on your Connection Server will break Horizon, so be sure to setup IIS on a separate server.)

Setup IIS Through Server Administrator 

You can easily install IIS on Windows Server through Server Manager's Add Roles and Features Wizard.  Add the Web Server (IIS) role along with the required IIS Management Console feature.  Then next your way through this wizard, accepting all the defaults and never looking back.

 

Assuming everything goes as expected you'll soon have access to a default web site accessible over http from the localhost. After the installation is complete navigating to http://localhost on the local machine will yield the default IIS page.

Next, you need to add a virtual directory to this default web site. 

Add A Virtual Directory To The Default Website 

After adding the IIS role to your server open IIS manager and navigate to the Default Web Site. Click on the explore option:

This will take you to the physical file system representing the root of your website. Create a new folder at the root and name it whatever you'd like.  Copy the LCM.zip bundle and Connection server installer there.

At this point add a virtual directory to the website. Navigate back to IIS Manager, right click on the Default Web Site and select, “Add Virtual Directory."  Define an alias for this virtual directory and define the physical path for this virtual directory by navigating to the folder you've just created.  

Next, navigate to this new virtual folder under the Default Web Site.  Select Directory Browsing. 

Select the option to enable directory browsing available to the right.

 

Now, along with the default IIS page, you should be able to navigate to the virtual directory from the local browser.  For example, if you created a virtual directory called installer you should be able to browser it by pointing your browser to http://localhost/installer.

 

At last, it’s time for everyone’s least favorite thing to do: setup a proper SSL cert.

Add A Valid SSL Cert

If you go with a publicly trusted CA authority meeting the SSL requirement is a fairly straightforward process. For example, in my own lab I generated a CSR from IIS, submitted it to Namecheap.com and after following the normal process received my cert and CA bundle.  Then I followed a standard process for converting the cert and CA bundle to .p7b format, adding it to the IIS server, and then binding it to port 443 for the default web site.   

At this point, I had secure URL for my installer to work with.  

If by chance you're looking to use an internal Microsoft CA, i.e., one that's not publicly trusted, there's an additional step required to get things working properly.  You must import your root CA certificate for your internal CA into the JRE keystore located at JAVA_HOME\jre\lib\security\cacerts on your Connection Server.   Otherwise, when you go to register your server installer build the LCM API won't recognize the file path to your Connection Server installer, complaining you're using an "Invalid file URL," despite having a valid URL when testing with a browser. 

To avoid this challenge we need to add trust for this internal certificate authority to the JRE leveraged by Horizon, otherwise, the trust isn't there. Yes, this is the case even if the cert is trusted by your Windows host.  There's a distinction between certs trusted by the Windows OS and certificates trusted by Java, a nuance distinction, but utterly critical to the success of this endeavor.   

The first step is  to get a copy of the root CA certificate for your internal CA.  For my lab, since every desktop in the domain trusted the ca, I navigated to cert administrator on my Horizon Connection server.  Having opened cert management against the local computer I navigated to Trusted Root Certificate Authorities --> Certificates, and located my root CA.  Then I right clicked on it selecting All Tasks, then Export. 

From here, I selected to export the CA in the DER encoded format, leading to a exported copy of my cert named evengooder_internal_ca.cer. Then I navigated to the directory:

C:\Program Files\VMware\VMware View\Server\jre\lib\security>

From there I ran the command:

..\\..\bin\keytool.exe -import -trustcacerts -alias <ALIAS_NAME> -file <PATH_INTERNAL_CA_ROOT_CERT> -keystore cacerts

More specifically, I went with: 

..\\..\bin\keytool.exe -import -trustcacerts -alias evengooderCA -file c:\software\evengooder_internal_ca.cer -keystore cacerts

When executing the command you'll be prompted for the keystore password.  The default password for JRE keystore is, "changeit". 

I know it's messy, but it's necessary.  After executing this command you'll need to restart the horizon service, but at that point, you're JRE instance should trust certs issued from you're internal ca.  You can confirm the trust is there by running keytool with the -list option.   For example, I did this to get a list of trusted ca's redirected to a text file for searching.  

C:\Program Files\VMware\VMware View\Server\jre\bin>keytool.exe -list -cacerts >> c:\software\trusted_ca.txt

When navigating through the output I was able to locate my internal certificate authority.  The ultimate confirmation that things had gone right was when my server installer package started registering properly, without the,  "Invalid file url," error I was getting prior. 

The WinRM Service

In my lab there were no steps necessary to meet requirements for the Windows Remote Management (WinRM) service. It was automatically started by default on both my Connection Server and target machine. Further, since both my Connection Server and target machine were Windows 2022 servers joined to the same domain, no additional configuration was necessary.

For testing purposes, I made sure I could access the target machine from the Connection Server using Enter-PSSesssion, a PowerShell command that enables an interactive session with a remote computer using WinRM.

Enter-PSSession -ComputerName <RemoteComputerName> -Credential <Domain\Username>

At this point, I was able to run commands like dir, or even PowerShell commands like Get-Process against the remote target machine.  I was even able to launch notepad.exe on the remote target machine using Start-Process.  

I had originally anticipated more WinRM configuration, but it just wasn’t necessary in my lab. To my understanding, because both machines were joined to the same domain it wasn’t necessary to run, “winrm quickconfig,” or add trusted hosts.   Folks with unusual situations where the Connection Server is on a different domain than the target machine might require additional WinRM configs to get things working.  Otherwise, I think this WinRM requirement should be easy for typical Horizon environments to handle.  

 

Conclusion

Though adopting the LCM APIs is not for the faint of heart it's well worth the trouble.  In addition to it's already impressive functionality there's plans to extend it's scope to other Horizon infrastructure. Most Horizon admins stand to benefit from it's current capabilities, let alone it's future expanded ones. If you end up taking this technology for a test spin please let me know how it goes in the comments below.