The following blog post runs through a vRA Cloud Template that deploys a Windows server using Cloudbase-init Multi-part user data, including cloud-config and PowerShell code to configure various things.
This post is broken into 4 main sections…
- Cloud Template Inputs.
- Cloud Template Resources.
- Cloud-config for the Cloud.Machine resource.
- A breakdown of the Cloud-config components.
#Cloud Template Inputs
name: Windows
version: 0.0.66
formatVersion: 1
inputs:
image:
type: string
default: windows-server-2019-standard
title: Operating System and Version
oneOf:
- title: Windows Server 2019 Standard
const: windows-server-2019-standard
flavor:
type: string
title: Server Size
oneOf:
- title: Standard Small (2CPU, 8GB RAM)
const: s1.small
- title: Standard Medium (2CPU, 16GB RAM)
const: s1.medium
- title: Standard Large (4CPU, 16GB RAM)
const: s1.large
- title: Standard Extra-Large (4CPU, 32GB RAM)
const: s1.xlarge
serverUser:
type: string
serverPassword:
type: string
encrypted: true
title: Server Password
serverCount:
type: integer
maximum: 5
minimum: 1
default: 1
title: Number of Servers
dataDisk1Size:
type: integer
title: Data Disk Size
minimum: 0
maximum: 1000
default: 0
domainPassword:
type: string
readOnly: false
encrypted: true#Cloud Template Resources
We have configured 3x resources on our Cloud Template, 1x Cloud.Volume (labelled DataDisk), 1x Cloud.Network (labelled Network) and 1x Cloud.Machine (labelled Server).
There are some specific properties worth calling out…
- On the
ServerresourcecustomizeGuestOs: false- this stops vRA customising the VM, this would usually configure the VM Name and IP by creating a temporary customisation spec at deployment time. - On the
Serverresourceassignment: static- this pulls an IP from vRA’s IPAM.
resources:
DataDisk:
type: Cloud.Volume
allocatePerInstance: true
properties:
capacityGb: ${input.dataDisk1Size}
count: '${input.dataDisk1Size == 0 ? 0 : input.serverCount}'
Network:
type: Cloud.Network
properties:
networkType: outbound
constraints:
- tag: network.isolated:true
Server:
type: Cloud.Machine
allocatePerInstance: true
properties:
customizeGuestOs: false
image: ${input.image}
flavor: ${input.flavor}
count: ${input.serverCount}
attachedDisks: ${map_to_object(slice(resource.DataDisk[*].id, count.index, count.index + 1), "source")}
constraints:
- tag: cloud.zone.region:dc3
networks:
- network: ${resource.Network.id}
assignment: static
cloudConfig: #see below section
#Cloudbase-Init (Cloud Config)
This cloudConfig utilises the multi-part user data (Multi-part content), this allows us to run a variety of content types, including cloud-config and PowerShell scripts.
The example below uses a combination of Inputs and Property Groups to get its data. The cloudConfig does the following:
- Sets virtual machine hostname using cloud-config.
- Creates a local user based on two inputs, one for the username, one for the password and then adds that user to the Administrators group.
- Enables the “Remote Desktop” and “ICMPv4-In” firewall rules.
- Sets some Registry Key values.
- Formats the additional disk that was added in the Cloud Template.
- Configures various network adapter settings, including the IP and DNS.
- Sleeps between setting the network and Domain Join.
- Joins the server to the Domain.
- Removes any reference to entered passwords from the Cloudbase-init log files.
- Ejects any CD-ROMs.
- Reboots the server.
The virtual machine template is running Windows Server 2019, it has Cloudbase-init installed with the Cloudbase-init service set to “Automatic (Delayed)”. The Sysprep options were also unchecked on the Cloudbase-init installer. Refer to Michael Poore’s blog post on this - Installing Cloudbase-Init on Windows for vRA Customisation.
Below is the Cloud-config in full, we break down each section at the bottom.
#Cloud-config in full
cloudConfig: |
Content-Type: multipart/mixed; boundary="==NewPart=="
MIME-Version: 1.0
--==NewPart==
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config"
set_hostname: ${self.resourceName}
--==NewPart==
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="INIT.ps1"
#ps1_sysnative
$localUserPassword = ConvertTo-SecureString ${input.serverPassword} -AsPlainText -Force
New-LocalUser -Name "${input.serverUser}" -Password $localUserPassword
Add-LocalGroupMember -Group "Administrators" -Member "${input.serverUser}"
Set-NetFirewallRule -DisplayName "File and Printer Sharing (Echo Request - ICMPv4-In)" -enabled True
Enable-NetFirewallRule -DisplayGroup "Remote Desktop"
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters' -Name 'NV Domain' -Value "${propgroup.defaultServerValues.fqdnDomain}"
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters' -Name SyncDomainWithMembership -Value "0"
Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -name "fDenyTSConnections" -value 0
Get-Disk | Where-Object PartitionStyle -Eq "RAW" | Initialize-Disk -PassThru | New-Partition -AssignDriveLetter -UseMaximumSize | Format-Volume -NewFileSystemLabel "DATA"
Get-NetAdapter | Set-DnsClient -RegisterThisConnectionsAddress $False
Get-NetAdapter | Set-DnsClient -UseSuffixWhenRegistering $False
Get-NetAdapter | Set-DnsClient -ConnectionSpecificSuffix ${propgroup.defaultServerValues.fqdnDomain}
Set-DnsClientGlobalSetting -SuffixSearchList @("domain.com")
$adapter = Get-NetAdapter | ? { $_.Name -eq "Ethernet0" }
$adapter | Remove-NetIpAddress -Confirm:$false
$adapter | Remove-NetRoute -Confirm:$false
$adapter | New-NetIpAddress -IPAddress ${self.networks[0].address} -PrefixLength ${resource.Network.prefixLength} -DefaultGateway ${resource.Network.gateway}
$adapter | Set-DnsClientServerAddress -ServerAddresses("${join(resource.Network.dns,', ')}")
$adapter | Disable-NetAdapterBinding -ComponentID ms_tcpip6
Start-Sleep -Seconds 10
$domainJoinCreds = New-Object System.Management.Automation.PSCredential "${propgroup.defaultServerValues.windowsDomainJoinUsername}@${propgroup.defaultServerValues.windowsDomain}" ,(ConvertTo-SecureString -String "${input.domainPassword}" -AsPlainText -Force)
Add-Computer -DomainName "${propgroup.defaultServerValues.windowsDomain}"-Credential $domainJoinCreds -Server "${propgroup.defaultServerValues.windowsDomainServerDc3}" -Restart:$false -Force
(Get-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log") -replace "${input.domainPassword}","<DOMAIN-JOIN-PASSWORD>" | Set-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log" -Verbose
(Get-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log") -replace "${input.serverPassword}","<LOCAL-USER-PASSWORD>" | Set-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log" -Verbose
$drives = Get-WmiObject Win32_Volume -Filter "DriveType=5"
$drives | ForEach-Object { (New-Object -ComObject Shell.Application).Namespace(17).ParseName($_.Name).InvokeVerb("Eject") } -ErrorAction SilentlyContinue
shutdown /r /t 10#Identify Multi-part boundary
We establish our multi-part boundary, which tells cloudConfig where a new type starts. Our boundary parameter is ==NewPart==.
cloudConfig: |
Content-Type: multipart/mixed; boundary="==NewPart=="
MIME-Version: 1.0#Cloud-config Section
Our new section requires the use of our boundary parameter, this boundary also requires a prefix of --. In this section we set the virtual machine hostname using a reference the the resource itself - in otherwords, whatever the VM is called (based on Custom Names in vRA) we push it into the cloud-config section. This seems to automatically reboot the VM after the hostname is set.
In this section we specify the filename as cloud-config - Content-Disposition: attachment; filename="cloud-config". We can any cloud-config related commands in here. Refer to the Cloudbase-init docs for information - #cloud-config.
--==NewPart==
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config"
set_hostname: ${self.resourceName}#Start of the PowerShell Section
This section is the start of our PowerShell code, again we use our boundary and its double-dash prefix --==NewPart==. Again we set a filename but this time with the file extension .ps1 - Content-Disposition: attachment; filename="INIT.ps1". The addition of #ps1_sysnative tells cloudConfig what executable to use. Refer to the Cloudbase-init docs for information - #powershell.
--==NewPart==
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="INIT.ps1"
#ps1_sysnative#Local User Creation
We utilise the Inputs for serverPassword and serverUser to add and configure our local user.
$localUserPassword = ConvertTo-SecureString ${input.serverPassword} -AsPlainText -Force
New-LocalUser -Name "${input.serverUser}" -Password $localUserPassword
Add-LocalGroupMember -Group "Administrators" -Member "${input.serverUser}"#Set Firewall Rules
Set-NetFirewallRule -DisplayName "File and Printer Sharing (Echo Request - ICMPv4-In)" -enabled True
Enable-NetFirewallRule -DisplayGroup "Remote Desktop"#Set Registry Values
We utilise values from a Property Group called defaultServerValues and specifically the value of property fqdnDomain. An example value could be domain.com.
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters' -Name 'NV Domain' -Value "${propgroup.defaultServerValues.fqdnDomain}"
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters' -Name SyncDomainWithMembership -Value "0"
Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -name "fDenyTSConnections" -value 0#Format Additional Disk
This code looks for all disks that have a matching partition style and then initialises them before applying a driver letter and label. We only add a single disk in this Cloud Template, hence a single label - if you add more than one disk you will need to adapt the code to suit.
Get-Disk | Where-Object PartitionStyle -Eq "RAW" | Initialize-Disk -PassThru | New-Partition -AssignDriveLetter -UseMaximumSize | Format-Volume -NewFileSystemLabel "DATA"#Configure Network DNS Client Settings
Again we utilise values from a Property Group called defaultServerValues and specifically the value of property fqdnDomain. An example value could be domain.com.
Get-NetAdapter | Set-DnsClient -RegisterThisConnectionsAddress $False
Get-NetAdapter | Set-DnsClient -UseSuffixWhenRegistering $False
Get-NetAdapter | Set-DnsClient -ConnectionSpecificSuffix ${propgroup.defaultServerValues.fqdnDomain}
Set-DnsClientGlobalSetting -SuffixSearchList @("domain.com")#Configure Network DNS Client Settings
Our virtual machine template includes a single network adapter with the label Ethernet0, we first identify this network adapter object and we then configure the required values, including IP and DNS.
Within this Cloud Template we have configure the Machine resource to have assignment: static for it’s networking, so we are allocating an IP address from vRA’s IPAM.
On line 4 below we retrieve the IP address assigned to the virtual machine - ${self.networks[0].address}. For prefix and default gateway values we reference the attached Network resource - ${resource.Network.prefixLength} and ${resource.Network.gateway}.
Finally for line 5 we pull out the attached Networks DNS servers ${join(resource.Network.dns,', ')}. In this particular variable we also utilise one of vRA’s supported expression syntaxes join. Cloud Assembly expression syntax.
$adapter = Get-NetAdapter | ? { $_.Name -eq "Ethernet0" }
$adapter | Remove-NetIpAddress -Confirm:$false
$adapter | Remove-NetRoute -Confirm:$false
$adapter | New-NetIpAddress -IPAddress ${self.networks[0].address} -PrefixLength ${resource.Network.prefixLength} -DefaultGateway ${resource.Network.gateway}
$adapter | Set-DnsClientServerAddress -ServerAddresses("${join(resource.Network.dns,', ')}")
$adapter | Disable-NetAdapterBinding -ComponentID ms_tcpip6#Sleep
There is a slight delay between configuring the network and it being available, you could probably change this to a loop that better monitors the network status but during testing we found good success at the 10 second mark.
Start-Sleep -Seconds 10#Join Active Directory Domain
Within vRA we have configured Active Directory integration and we have made it available to all of the Projects that require it. This creates the Computer Object in our specified OU, which leaves us to only do the domain joining from within the deployed guest OS.
We utilise more Property Group values in this code as well as an additional Input for the domainPassword, our specific approach for this Input was to provide a Default Value for it on the Custom Form and set the Visibility to No, this way the user does not see it. The reason for this approach is because Secrets cannot be shared across the Organization, they are Project specific, which would have caused us to have multiple copies of this specific value (1x per Project).
In the first line we create a PowerShell Credentials object by using two Property Group values ${propgroup.defaultServerValues.windowsDomainJoinUsername} and ${propgroup.defaultServerValues.windowsDomain}, in combination with the @ symbol we construct a UPN that will be used for the domain join, such as [email protected], the second part of the Credentials object is the domainPassword value.
This Credentials object is then used on the second line along with some additional Property Group values, ${propgroup.defaultServerValues.windowsDomain}, for example ad.domain.com and ${propgroup.defaultServerValues.windowsDomainServerDc3} which is a specific server that we want to talk to, such as server1.ad.domain.com - this input is not required normally, but due to some network constraints we need to speak to a specific server.
$domainJoinCreds = New-Object System.Management.Automation.PSCredential "${propgroup.defaultServerValues.windowsDomainJoinUsername}@${propgroup.defaultServerValues.windowsDomain}" ,(ConvertTo-SecureString -String "${input.domainPassword}" -AsPlainText -Force)
Add-Computer -DomainName "${propgroup.defaultServerValues.windowsDomain}"-Credential $domainJoinCreds -Server "${propgroup.defaultServerValues.windowsDomainServerDc3}" -Restart:$false -Force#Clean the Cloudbase-init Log Files
Due to the configured logging level of Cloudbase-init we can see the inputted passwords in clear text. We run the below PowerShell code to look for the Input values for domainPassword and serverPassword and replace them with some placeholder values - <DOMAIN-JOIN-PASSWORD> and <LOCAL-USER-PASSWORD>. An alternative would be to change the Cloudbase-init logging level.
(Get-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log") -replace "${input.domainPassword}","<DOMAIN-JOIN-PASSWORD>" | Set-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log" -Verbose
(Get-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log") -replace "${input.serverPassword}","<LOCAL-USER-PASSWORD>" | Set-Content "C:\Program Files\Cloudbase Solutions\Cloudbase-Init\log\cloudbase-init.log" -Verbose#Eject CD-ROM
We remove the Cloud-config attached CD-ROM, im not sure if this is absolutely necessary but during testing we observed CD’s still being connected to the deployed VM.
$drives = Get-WmiObject Win32_Volume -Filter "DriveType=5"
$drives | ForEach-Object { (New-Object -ComObject Shell.Application).Namespace(17).ParseName($_.Name).InvokeVerb("Eject") } -ErrorAction SilentlyContinue#Shutdown VM
Finally, we reboot the VM, this is so the VM recognise it is domain joined.
shutdown /r /t 10
