Introduction

I’m working on a side project at the moment that involves storage servers, specifically CIFS/SMB shares. As it’s a side project, I’m working on it at home and I don’t have access to the resources at my day job, all I have to work with is my home workstation and network. Considering those constraints, what I really need is:

  • A virtual machine that I can run on my workstation and network because I have to use the hardware I already have. The wife is not going to let me buy a server rack for our apartment…
  • It can be alive only for as long as I’m testing this side project. I can’t permanently consume resources on my workstation because that will affect what I have available to do other work. So it has to be inexpensive for me to create and dispose of this server instance on-demand, for as long as I want to use it.
  • Ideally, there should be some scope for reuse in my solution. If I need a different type of server in the future for some other project, I should have to do less upfront work to meet that need.

I’ve chosen a great stack of automation tools that, together, will give us exactly what I’ve listed above. This blog post is the first in a two-part series that will take us from nothing to the on-demand provisioning of a file server, with the bonus of having a general purpose base image for reuse in other projects.

The technologies we will use are:

  • Vagrant: a tool designed to make it easy to create and destroy transient DevTest infrastructure. It integrates with many virtualization platforms and configuration management tools.
  • Windows Server 2016: CIFS/SMB is mostly associated with Windows Servers, so this is the best choice of OS for being our base image.
  • Hyper-V: our hypervisor (virtual machine manager) of choice. Vagrant is largely built around VirtualBox but has an integration for Hyper-V. Since Hyper-V is freely available to me on Windows 10 and will be better at running a Windows Server than the alternatives, it’s the obvious choice.
  • Chef: a configuration management tool that Vagrant also has a plugin for. It has a powerful domain-specific language for writing system configuration recipes. Chef is going to turn our base Windows Server image into a useful storage server.

Part 1 (this post) will cover creating the base image and integrating it with Vagrant so that we can spin up and tear down a clean Windows Server on Hyper-V at will. Part 2 covers the Chef integration and how to turn that general purpose base image into the storage server I need.

Because we’re using Hyper-V this post assumes that your host machine is running Windows. The walkthrough is going to be PowerShell heavy, because what kind of automation blog guides you via wizards?

Requirements

Tool Version used in this post
Vagrant 1.9.2
Hyper-V Manager* 10.0.14393.0
Windows Server (.iso)** 2016
Chef Client 12.19.36
7-Zip 16.04

*Hyper-V can be installed as a Windows Feature for free on Windows 10/8.1/8.

**Windows Server 2016 requires a valid license key. I have an MSDN Subscription which gives me access to Microsoft license keys for DevTest purposes.

We’re going to start from the position where you have already installed Vagrant, Hyper-V and 7-Zip on your workstation and you have a Windows Server ISO file in your file system.

Initializing Vagrant

Getting bootstrapped with Vagrant is very straightforward. Fire up PowerShell (as Administrator) and follow along:

Set-Location -Path .\Projects\WindowsServer2016
vagrant init

The init command creates a file called Vagrantfile in the current working directory. Open it up with your favorite text editor. You’ll see a lot of comments in the file explaining what the various settings represent and only three lines of live code (comments below are my own rather than those from the file).

# The beginning of the Vagrant configuration block. "2" is like a schema version.
Vagrant.configure("2") do |config|

# Box is the Vagrant term for a base image. This is the template of the VM
# that will become our file server when we activate Vagrant.
config.vm.box = "base"

# Terminates the Vagrant configuration block
end

We don’t currently have any base images, nevermind one named “base”. So creating a Vagrant Box for the server will be our first job.

Creating the Vagrant Box (Hyper-V)

Creating the virtual machine

To create our VM base image we need to create a new VM on Hyper-V. I’m going to use the Hyper-V PowerShell module for this, but you could use the new VM wizard in Hyper-V Manager instead.

Import-Module Hyper-V

# Take a look at the virtual switches you have available to provide networking
# to the new VM. I have one for my home network already, if you don't you'll
# need to create one with New-VMSwitch.
Get-VMSwitch | Format-Table

# The configuration of our new VM.
$newVmArgs = @{
  "Name" = "WindowsServer2016";
  "MemoryStartupBytes" = 4GB;
  "BootDevice" = "VHD";
  "Path" = "D:\Hyper-V";
  "NewVHDSizeBytes" = 20GB;
  "NewVHDPath" = "D:\Hyper-V\WindowsServer2016\Virtual Hard Disks\WindowsServer2016.vhdx"
  "Generation" = 2;
  "Switch" = "Home Network"; # Substitute with the name of your VMSwitch.
}

# Create the VM.
$vm = New-VM @newVmArgs

# Mount our installation media (.iso) on the VM.
Add-VMDvdDrive -VMName WindowsServer2016 -Path "D:\ISOs\en_windows_server_2016_x64_dvd_9718492.iso"

# Start VM
$vm | Start-VM

From this point, you can use Hyper-V Manager to connect to the new VM and run through the standard Windows Server installation wizard once it has booted. If the VM fails to boot you probably missed the “Press any key to boot from CD or DVD” prompt, just restart the boot sequence and watch for it.

Installing Windows Server

Given the choice between taking the Desktop Experience version of Windows Server or not, I choose not to. What that means is that you don’t get the Windows GUI. “Why would you not want the Windows GUI?” – I hear you ask. Because the whole point of this series to drive the server configuration I need through automation, not manually with a GUI. Not having the crutch of the Windows GUI means that I cannot cheat.

This isn’t some form of technical sadomasochism. Jeffrey Snover, lead architect of Windows Server at Microsoft, says it best: “Local Admin GUIs on a server are poison. It’s like heroin, you know your first shot oh so nice and then all of a sudden your life is ruined and you end up dead.” – say “No” to drugs readers. If you want to explore GUI vs. non-GUI further take a look at this Channel 9 video about Windows Nano Server.

Okay, back to work! You’ll be asked to enter a password for the built-in Administrator account at some point in the process. Once the installation process is finished, log in as that Administrator and you’ll be presented with a command prompt, or Server Manager if you opted for the Desktop Experience.

Configuring the virtual machine for Vagrant

Vagrant has a couple of prerequisites for Windows boxes that we need to set up. The following components need to be disabled:

  • User Account Control.
  • Password complexity requirements.
  • The shutdown reason dialog*.
  • Server Manager startup on login*.

It’s also useful to enable Remote Desktop Connections*.

* GUI users only.

# Start PowerShell if you were logged into a cmd.exe prompt
powershell.exe

# Disable UAC by setting the correct registry property.
New-ItemProperty -Path HKLM:Software\Microsoft\Windows\CurrentVersion\policies\system -Name EnableLUA -PropertyType DWord -Value 0 -Force

# Disable password complexity requirements. There isn't a PowerShell-like way of doing this at the moment, so we have to use secedit to export the
# configuration, replace a value and reapply it.
secedit /export /cfg .\secedit.cfg

(Get-Content -Path .\secedit.cfg).Replace("PasswordComplexity = 1", "PasswordComplexity = 0") | Out-File -FilePath .\secedit.cfg

secedit /configure /db C:\Windows\security\local.sdb /cfg .\secedit.cfg /areas SECURITYPOLICY

# Now that we've disabled password complexity, change the Administrator password to "vagrant". The docs recommend you use this standard password for boxes
# that you want to share with others.
net user Administrator vagrant

# If you've opted out of using the GUI version of Windows Server you can ignore the below commands.

# Disable shutdown reason dialog.
New-ItemProperty -Path "HKLM:Software\Microsoft\Windows\CurrentVersion\Reliability" -Name ShutdownReasonOn -PropertyType DWord -Value 0 -Force
New-ItemProperty -Path "HKLM:Software\Microsoft\Windows\CurrentVersion\Reliability" -Name ShutdownReasonUI -PropertyType DWord -Value 0 -Force
New-ItemProperty -Path "HKLM:Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Reliability" -Name ShutdownReasonOn -PropertyType DWord -Value 0 -Force
New-ItemProperty -Path "HKLM:Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Reliability" -Name ShutdownReasonUI -PropertyType DWord -Value 0 -Force

# Disable ServerManager startup on login.
New-ItemProperty -Path HKLM:Software\Microsoft\ServerManager -Name DoNotOpenServerManagerAtLogon -PropertyType DWord -Value 1 -Force

# Enable Remote Desktop Connections
Set-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Terminal Server" -name fDenyTSConnections -PropertyType DWord -Value 0 -Force
Enable-NetFirewallRule -DisplayGroup "Remote Desktop"

If you’re following along with a different version of Windows Server, the above registry keys can change between versions. So if that didn’t work, have a Google for the registry keys specific to the version you’re using.

Next, we need to configure WinRM, this is the Windows remoting protocol. The majority of the Vagrant documentation refers to SSH for remoting, some users have been able to use Windows boxes with SSH via the VirtualBox provider and PuTTY, but the Hyper-V provider uses WinRM (which is at least native to Windows).

The following commands configure WinRM to auto-start and allow unencrypted basic authentication.

winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="512"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
Set-Service -Name WinRM -StartupType Automatic
Note: The large part of what I’ve instructed you to do above strips security on this base image back to a bare minimum for it to work well with Vagrant. Vagrant positions itself as a DevTest tool, a domain where security requirements tend to be less stringent than in production (and environments often sandboxed for this reason). Please don’t use base images intended for Vagrant when creating production server instances, there are other automation tools that cover that space very well. It’s about using the right tool for the right job.

Updating Windows Server

Since we’re going to the effort of building our own base image we might as well bring the OS up to date. We’re going to use a community PowerShell module called PSWindowsUpdate for this because, in my opinion, it’s the nicest interface for working with Windows Update in PowerShell.

# As this is a clean Windows Server we should install NuGet first.
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force

# Make sure the PowerShell Gallery is a trusted repository
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted

# Install the module from PowerShell Gallery
Install-Module -Name PSWindowsUpdate

# Run Windows Updates and reboot
Get-WUInstall -AcceptAll -AutoReboot

Once you’ve rebooted back in, you’re good to go.
 

Installing the Provisioner Agent

Note: If you’re only interested in this post as a standalone piece, you can skip to the next section. This is mainly for Part 2 in the series.

The provisioner is the mechanism by which Vagrant drives the VM created from the box (base image) into a useful state for your dev testing. Boxes are supposed to be as close to the base OS as possible for maximizing reuse of images between different dev test scenarios. The only exception to this is when you have a dependency that is really time-consuming to install and configure dynamically. In that case, you might want to optimize for speed and include the dependency in the base image, but the trade-off is that you’re starting your automation from a “dirtier” state. Normally it’s the provisioner that takes care of getting you from vanilla OS to the desired state.

Vagrant has a shell provisioner that executes commands defined by the user in the Vagrantfile and this provisioner does not require you to install anything into your base image. That’s where most new users of Vagrant start, but we’re going to use the Chef provisioner instead, which does require us to install the Chef Client in advance.

To acquire and install the Chef Client, execute the following:

# Note that the URI is specific to the version of Chef and OS. Browse for the appropriate URI in your host OS web browser.
# There isn't an MSI specific to Windows Server 2016 on the Chef website so this one is for 2012 R2.
Invoke-WebRequest -UseBasicParsing -Uri https://packages.chef.io/files/stable/chef/12.19.36/windows/2012r2/chef-client-12.19.36-1-x64.msi -OutFile .\chef-client-x64.msi

# Install the downloaded MSI, specifically we want the Chef Client and the associated PowerShell modules.
# Logging is output to .\msiexec_log.log if you need the troubleshoot.
$result = Start-Process msiexec.exe -Wait -PassThru -ArgumentList "/qn /L*v msiexec_log.log /i $(Get-Location)\chef-client-x64.msi ADDLOCAL=`"ChefClientFeature,ChefPSModuleFeature`""

# Check msiexec returned 0 (success)
$result.ExitCode

Generalizing the base image

The final step in preparing our base image is to generalize it with Sysprep. This removes instance specific identifiers from the base image, allowing you to instantiate it multiple times, use it on different hardware, etc.

First we need to create an unattend.xml file so that when the base image is instantiated it can get through Windows setup without any user interaction.

# Create unattend.xml at an obvious location in the file system.
Set-Location -Path C:\
New-Item -Name unattend.xml

# Open it in Notepad (this is still available in GUI-less Windows Server).
notepad .\unattend.xml

Paste the following into Notepad, then save and exit:

<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend">
    <settings pass="generalize">
        <component name="Microsoft-Windows-Security-SPP" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <SkipRearm>1</SkipRearm>
        </component>
        <component name="Microsoft-Windows-PnpSysprep" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <PersistAllDeviceInstalls>false</PersistAllDeviceInstalls>
            <DoNotCleanUpNonPresentDevices>false</DoNotCleanUpNonPresentDevices>
        </component>
    </settings>
    <settings pass="oobeSystem">
        <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <InputLocale>en-US</InputLocale>
            <SystemLocale>en-US</SystemLocale>
            <UILanguage>en-US</UILanguage>
            <UserLocale>en-US</UserLocale>
        </component>
        <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <OOBE>
                <HideEULAPage>true</HideEULAPage>
                <ProtectYourPC>1</ProtectYourPC>
                <NetworkLocation>Home</NetworkLocation>
                <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
            </OOBE>
            <TimeZone>UTC</TimeZone>
            <UserAccounts>
                <AdministratorPassword>
                    <Value>vagrant</Value>
                    <PlainText>true</PlainText>
                </AdministratorPassword>
                <LocalAccounts>
                    <LocalAccount wcm:action="add">
                        <Password>
                            <Value>vagrant</Value>
                            <PlainText>true</PlainText>
                        </Password>
                        <Group>administrators</Group>
                        <DisplayName>Vagrant</DisplayName>
                        <Name>vagrant</Name>
                        <Description>Vagrant User</Description>
                    </LocalAccount>
                </LocalAccounts>
            </UserAccounts>
        </component>
    </settings>
    <settings pass="specialize">
    </settings>
</unattend>

Acknowledgement: Credit for the unattend.xml file content needs to go to Matt Wrock. I acquired it from his packer-templates project on GitHub here. Thanks Matt!

Now, back in PowerShell:

# Set the current working directory to be the Sysprep folder.
Set-Location -Path C:\Windows\System32\Sysprep

# Generalize the image and shutdown
.\sysprep.exe /generalize /oobe /unattend:C:/unattend.xml /shutdown

Once the VM has shut down, we’re ready to turn it into a Vagrant box.

Boxing up the virtual machine

Back on our host machine, we now need to export the VM.

# Hopefully you still have the PowerShell session from earlier, if not:
Import-Module Hyper-V
$vm = Get-VM -Name WindowsServer2016

# We need to detach the DVD drive we used to mount the ISO earlier.
$vm | Get-VMDvdDrive | Remove-VMDvdDrive

# Export the VM to a known place
$vm | Export-VM -Path D:\VagrantBoxes\

# Switch to the export location
Set-Location -Path D:\VagrantBoxes

# Take a look inside the exported VM directory
Get-ChildItem -Path .\WindowsServer2016

# There should be three subdirectories: Snapshots, Virtual Hard Disks and Virtual Machines.
# Vagrant ignores VM snapshots, so there's no point in boxing any up. Delete the directory.
Remove-Item -Path .\WindowsServer2016\Snapshots\

The final component of a Vagrant box is a Metadata JSON file. Let’s create one:

# Create the metadata file
New-Item -Name .\WindowsServer2016\metadata.json -ItemType File

# Open the empty metadata file in your preferred text editor
notepad++ .\WindowsServer2016\metadata.json

The Metadata file is used by Vagrant to associate the box with the provider it was built for, we just need to add a simple key-value pair for this:

{
  "provider": "hyperv"
}

Save metadata.json and exit back to the shell. Now we can package up the box. We need to create an archive containing the remaining subdirectories and metadata.json, but not the parent directory. If the parent directory is included in the archive we won’t have the file structure Vagrant expects.

# Change location to the exported VM directory so that we avoid accidentally
# archiving the parent directory.
Set-Location -Path .\WindowsServer2016

# Use 7-Zip to archive the remaining subfolders and metadata.json into a .tar file.
# Does assume you have 7z.exe visible from $env:Path.
7z a WindowsServer2016.tar *

TAR is Vagrant’s preferred archive format and it seems to be the most reliable for adding boxes, but it provides no file compression by itself. So if you plan on distributing your box then you might want to compress it into a TAR.GZ for more efficient storage/transfer, etc.

Finally, let’s add our new box to Vagrant.

# vagrant box add {name} {archive_location}
vagrant box add WindowsServer2016 .\WindowsServer2016.tar

If you are successful you should see:

==> box: Box file was not detected as metadata. Adding it directly...
==> box: Adding box 'WindowsServer2016' (v0) for provider:
    box: Unpacking necessary files from: file://D:/VagrantBoxes/WindowsServer2016/WindowsServer2016.tar
    box: Progress: 100% (Rate: 286M/s, Estimated time remaining: --:--:--)
==> box: Successfully added box 'WindowsServer2016' (v0) for 'hyperv'!

Testing the Vagrant Box

# Go back to our original project directory
Set-Location -Path $env:UserProfile\Projects\WindowsServer2016

# Remove the first Vagrantfile we created, we're going to create a new one with the
# Hyper-V box we created.
Remove-Item .\Vagrantfile

# Creates a new Vagrantfile, only with our box configured rather than the default.
# The -m flag excludes the comments that we saw in our first Vagrantfile.
vagrant init -m WindowsServer2016

# Now open the Vagrantfile in your text editor again
notepad++ .\Vagrantfile

The minimal Vagrantfile we have created looks like this:

Vagrant.configure("2") do |config|
  config.vm.box = "WindowsServer2016"
end

We need to configure Vagrant to use WinRM for remoting and we’re also going to disable Vagrant’s default behavior of setting up a file share between the host and the VM, although you’re welcome to re-enable this later if it is useful to you. The end result looks like this:

Vagrant.configure("2") do |config|
  config.vm.box = "WindowsServer2016"
  config.vm.communicator = "winrm"
  config.vm.synced_folder ".", "/vagrant", disabled: true
end

Save Vagrantfile and exit back to the shell.

A quick word of warning before we fire up the box. The Vagrant documentation would have you believe that all you need to do now is issue the command: vagrant up provider=hyperv – but if you do that now Vagrant won’t be able to find the hyperv provider.

Rather confusingly, you have to set a VAGRANT_DEFAULT_PROVIDER environment variable as well. The only reference to this variable in the documentation is it having a lower precedence than the –provider argument whilst resolving which provider to use. But I find for Hyper-V at least, that the argument won’t work without that environment variable being set.

# Check $env:VAGRANT_DEFAULT_PROVIDER resolves to 'hyperv'
$env:VAGRANT_DEFAULT_PROVIDER

# If it doesn't, this will add the environment parameter permanently for your user.
[Environment]::SetEnvironmentVariable("VAGRANT_DEFAULT_PROVIDER", "hyperv", "User")

# Here we go!
vagrant up

On a successful vagrant up, you should see this output like this:

Bringing machine 'default' up with 'hyperv' provider...
==> default: Verifying Hyper-V is enabled...
==> default: Importing a Hyper-V instance
    default: Cloning virtual hard drive...
    default: Creating and registering the VM...
    default: Successfully imported a VM with name: WindowsServer2016_1
==> default: Starting the machine...
==> default: Waiting for the machine to report its IP address...
    default: Timeout: 120 seconds
    default: IP: fe80::d1fe:474e:b9dd:5708
==> default: Waiting for machine to boot. This may take a few minutes...
    default: WinRM address: 192.168.0.34:5985
    default: WinRM username: Administrator
    default: WinRM execution_time_limit: PT2H
    default: WinRM transport: negotiate
==> default: Machine booted and ready!

It worked! Now we can quickly spin up a clean Windows Server 2016 instance on Hyper-V with one short command.

When you want Vagrant to tear down the VM, execute:

vagrant destroy --force

Closing thoughts

Vagrant’s great strength is its simple vagrant up/vagrant destroy loop. You can repeat these concise commands as many times as you need to return to a known state.

Most of the work here was in setting up the base image, which will hopefully be a one-time operation. You can also acquire community base images from Atlas, Vagrant’s public box repository. It’s worth knowing how to create your own though, which you now do.

The Vagrantfile is intended to be checked in to source control alongside the rest of your project. This means configuration changes can be tracked as source code, with all the other benefits using source control gives you. If you share your box also, then everybody in your team gets the benefit of being able to deploy DevTest infrastructure quickly and easily with automation.

Coming up in Part 2

In Part 2, we’re going to cover:

  • Vagrant Provisioners and how they fit into the Vagrant workflow.
  • Chef Cookbooks and how they can simplify automated system configuration.
  • How they work together to dynamically transform our Windows Server 2016 instances into useful storage servers, without complicating our development loop.

See you then.

About the Author Kirk MacPhee

An experienced software developer and technical lead, specializing in automation technologies and their application.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s