Deploying Cloud Foundry on Windows 10 – 2018

Posted on June 8, 2018 by Graham Nicol

TL:DR – I can’t currently get this installation to work.

Introduction

These instructions are based on the official bosh instructions listed at https://bosh.io/docs/bosh-lite/ but also include my comments on areas that work well, don’t work, or need some gentle crowbar action to make work! As Cloud Foundry describes itself as an opinionated PaaS, I feel empowered to have my own opinions here, and so should you! Let me know in the comments or on Twitter (@digitalteddysol) if you run into issues, or if something’s been fixed or changed.

  1. Download the Windows BOSH cli from https://bosh.io/docs/cli-v2/#install

  2. Move the downloaded file to somewhere on the PATH (I have a c:\Program Files\Cloud Foundry folder, this seemed appropriate.
  3. Rename the file to “bosh” (unless of course you want to type in that long filename you downloaded every time)
  4. In a terminal window check the version:
    bosh -v
    version 3.0.1-712bfd7-2018-03-13T23:26:43Z    Succeeded
    Handily, when you look at the “additional dependencies” page for BOSH you see the following listed under Windows:

    Windows

The create-env and destroy-env commands are not yet supported on Windows. Feel free to give it a try (and let us know if you have feedback), but we would recommend using a Linux-based virtual machine to run those commands instead.

    So, at this point we’ve stepped into an area of the map marked “Here be Dragons”. That’s not to say it won’t work, and to be honest we’re here trying it to see how it can, but it’s the first step of warnings that we might see some odd behaviours compared to an installation on MacOS or Linux.

  5. The next step is to install VirtualBox. The bosh instructions so to install the base version and then “if you encounter problems” then install the Extension Pack. Personally, whenever I’ve installed VirtualBox I’ve installed the Extension Pack, it contains so many handy little necessities that I’d only not install it if space was an absolute premium.A key thing here is that there’s a known working version of 5.1.x, so if you have an existing install make sure you run

    VBoxManage --versionAs the Windows install of VirtualBox doesn’t add the VirtualBox directory to the path you’ll need to go hunting for where you installed it and run that command from that folder.

  6. Similarly to the previous `cf-release` version of Cloud Foundry there’s a presumption that every machine in the known multiverse already has Git installed on it. To be fair, most development machines do, but not all and so if you’re new to this game and don’t have Git installed then at this point it’s critical you surf on over to https://git-scm.com/downloads and install it.
  7. The next three lines in the instructions won’t work on Windows without modification. On Linux based systems (including MacOS) a tilde can be used as an alias for the User’s home folder. This will also work in Windows Powershell, but not in a bog standard command prompt. Additionally the -p parameter (that allows you to make a whole folder tree in one go) isn’t implemented, so we’ll need to iterate that.The equivalent commands for a Windows command prompt will be:git clone https://github.com/cloudfoundry/bosh-deployment %USERPROFILE%/workspace/bosh-deployment

    cd %USERPROFILE%

    mkdir deployments

    cd deployments

    mkdir vbox

    cd vbox

    As Microsoft becomes ever more Linux friendly I suspect we’ll see more and more of these little gaps closed in Windows 10, but for now it just needs that little bit of extra work to navigate the filesystem.

  8. Again, by doing the tilde shuffle we can make the `create-env` command from the bosh-lite instructions more Windows friendlybosh create-env %USERPROFILE%/workspace/bosh-deployment/bosh.yml --state ./state.json -o %USERPROFILE%/workspace/bosh-deployment/virtualbox/cpi.yml -o %USERPROFILE%/workspace/bosh-deployment/virtualbox/outbound-network.yml -o %USERPROFILE%/workspace/bosh-deployment/bosh-lite.yml -o %USERPROFILE%/workspace/bosh-deployment/bosh-lite-runc.yml -o %USERPROFILE%/workspace/bosh-deployment/uaa.yml -o %USERPROFILE%/workspace/bosh-deployment/credhub.yml -o %USERPROFILE%/workspace/bosh-deployment/jumpbox-user.yml --vars-store ./creds.yml -v director_name=bosh-lite -v internal_ip=192.168.50.6 -v internal_gw=192.168.50.1 -v internal_cidr=192.168.50.0/24 -v outbound_network_name=NatNetworkAt this point, go and make a cup of tea/coffee or go for lunch, this command downloads all the various releases from the Internet and validates them, so it takes a little bit of time.

This… partially works.

$>bosh create-env %USERPROFILE%/workspace/bosh-deployment/bosh.yml --state ./state.json -o %USERPROFILE%/workspace/bosh-deployment/virtualbox/cpi.yml -o %USERPROFILE%/workspace/bosh-deployment/virtualbox/outbound-network.yml -o %USERPROFILE%/workspace/bosh-deployment/bosh-lite.yml -o %USERPROFILE%/workspace/bosh-deployment/bosh-lite-runc.yml -o %USERPROFILE%/workspace/bosh-deployment/uaa.yml -o %USERPROFILE%/workspace/bosh-deployment/credhub.yml -o %USERPROFILE%/workspace/bosh-deployment/jumpbox-user.yml --vars-store ./creds.yml -v director_name=bosh-lite -v internal_ip=192.168.50.6 -v internal_gw=192.168.50.1 -v internal_cidr=192.168.50.0/24 -v outbound_network_name=NatNetwork
Deployment manifest: 'C:\Users\Graham\workspace\bosh-deployment\bosh.yml'
Deployment state: './state.json'

Started validating
Downloading release 'bosh'... Finished (00:00:20)
Validating release 'bosh'... Finished (00:00:00)
Downloading release 'bosh-virtualbox-cpi'... Finished (00:00:53)
Validating release 'bosh-virtualbox-cpi'... Finished (00:00:01)
Downloading release 'bosh-warden-cpi'... Finished (00:00:26)
Validating release 'bosh-warden-cpi'... Finished (00:00:00)
Downloading release 'os-conf'... Finished (00:00:00)
Validating release 'os-conf'... Finished (00:00:00)
Downloading release 'garden-runc'... Finished (00:00:36)
Validating release 'garden-runc'... Finished (00:00:00)
Downloading release 'uaa'... Finished (00:01:09)
Validating release 'uaa'... Finished (00:00:00)
Downloading release 'credhub'... Finished (00:00:38)
Validating release 'credhub'... Finished (00:00:00)
Validating cpi release... Finished (00:00:00)
Validating deployment manifest... Finished (00:00:00)
Downloading stemcell... Finished (00:01:39)
Validating stemcell... Finished (00:00:00)
Finished validating (00:05:51)

Started installing CPI
Compiling package 'golang-1.10-linux/48c842421b6f05acf88dc6ec17f7574dade28a86'...[CLI] 2018/06/08 10:33:26 ERROR – Panic: UseIsolatedEnv is not supported on Windows

********************
goroutine 1 [running]:
runtime/debug.Stack(0xdb0620, 0xc0423ea0e0, 0xc0423c24a0)
/usr/local/go/src/runtime/debug/stack.go:24 +0x80
main.handlePanic()
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/main.go:110 +0x22b
panic(0xcb7b40, 0xc0423c24a0)
/usr/local/go/src/runtime/panic.go:458 +0x251
github.com/cloudfoundry/bosh-cli/cmd.Cmd.Execute.func1(0xc04221c188)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cmd/cmd.go:47 +0xee
panic(0xcb7b40, 0xc0423c24a0)
/usr/local/go/src/runtime/panic.go:458 +0x251
github.com/cloudfoundry/bosh-cli/vendor/github.com/cloudfoundry/bosh-utils/system.execCmdRunner.buildComplexCommand(0x1337280, 0xc0421b9f60, 0xe3b5bc, 0x4, 0xc04237fe80, 0x2, 0x2, 0xc0423a8ff0, 0x1, 0xc0423d1740, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/vendor/github.com/cloudfoundry/bosh-utils/system/exec_cmd_runner.go:88 +0x204
github.com/cloudfoundry/bosh-cli/vendor/github.com/cloudfoundry/bosh-utils/system.execCmdRunner.RunComplexCommand(0x1337280, 0xc0421b9f60, 0xe3b5bc, 0x4, 0xc04237fe80, 0x2, 0x2, 0xc0423a8ff0, 0x1, 0xc0423d1740, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/vendor/github.com/cloudfoundry/bosh-utils/system/exec_cmd_runner.go:21 +0x77
github.com/cloudfoundry/bosh-cli/vendor/github.com/cloudfoundry/bosh-utils/system.(*execCmdRunner).RunComplexCommand(0xc042179ca0, 0xe3b5bc, 0x4, 0xc04237fe80, 0x2, 0x2, 0xc0423a8ff0, 0x1, 0xc0423d1740, 0x56, ...)
:9 +0x9d
github.com/cloudfoundry/bosh-cli/installation/pkg.(*compiler).Compile(0xc0423926c0, 0x1335e60, 0xc0423d1680, 0x0, 0x0, 0x0, 0x0, 0xc0424e1800, 0x0, 0x0)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/installation/pkg/compiler.go:109 +0xbb6
github.com/cloudfoundry/bosh-cli/state/job.(*dependencyCompiler).compilePackages.func1(0x1388900, 0xed2ac4465)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/state/job/dependency_compiler.go:115 +0x77
github.com/cloudfoundry/bosh-cli/ui.(*stage).Perform(0xc0424dc7d0, 0xc0424dc820, 0x4e, 0xc04237fc80, 0x2, 0xc0424dc820)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/ui/stage.go:47 +0x15f
github.com/cloudfoundry/bosh-cli/state/job.(*dependencyCompiler).compilePackages(0xc0423a8960, 0xc0423a8ab0, 0x3, 0x3, 0x132e120, 0xc0424dc7d0, 0x3, 0x0, 0x0, 0xc042436200, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/state/job/dependency_compiler.go:133 +0x30b
github.com/cloudfoundry/bosh-cli/state/job.(*dependencyCompiler).Compile(0xc0423a8960, 0xc0421d65b0, 0x1, 0x1, 0x132e120, 0xc0424dc7d0, 0x0, 0x0, 0xc04220a008, 0xc042436200, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/state/job/dependency_compiler.go:50 +0x1fc
github.com/cloudfoundry/bosh-cli/installation.(*packageCompiler).For(0xc04237f9a0, 0xc0421d65b0, 0x1, 0x1, 0x132e120, 0xc0424dc7d0, 0x13, 0xc04247cf30, 0xc042382300, 0x33, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/installation/package_compiler.go:38 +0x93
github.com/cloudfoundry/bosh-cli/installation.(*installer).Install(0xc0422f3380, 0xc042388148, 0x4, 0xc042436200, 0xe, 0xc0423d8180, 0x13, 0xc04247cf30, 0xc042382300, 0x33, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/installation/installer.go:70 +0x44b
github.com/cloudfoundry/bosh-cli/cpi/release.CpiInstaller.installCpiRelease.func1(0x132e120, 0xc0424dc7d0, 0x32262a50, 0x136bd80)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cpi/release/installer.go:37 +0xab
github.com/cloudfoundry/bosh-cli/ui.(*stage).PerformComplex(0xc04238ee10, 0xe46417, 0xe, 0xc0423a8990, 0x1327da0, 0xc0423a0a60)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/ui/stage.go:68 +0x2c9
github.com/cloudfoundry/bosh-cli/cpi/release.CpiInstaller.installCpiRelease(0x2110400, 0xc042383300, 0x1327ae0, 0xc042394420, 0x0, 0x132dfa0, 0xc0422f3380, 0xc042388148, 0x4, 0xc042436200, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cpi/release/installer.go:39 +0x1ab
github.com/cloudfoundry/bosh-cli/cpi/release.CpiInstaller.WithInstalledCpiRelease(0x2110400, 0xc042383300, 0x1327ae0, 0xc042394420, 0x0, 0xc042388148, 0x4, 0xc042436200, 0xe, 0xc0423d8180, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cpi/release/installer.go:50 +0x153
github.com/cloudfoundry/bosh-cli/cmd.(*DeploymentPreparer).PrepareDeployment(0xc0420a65a0, 0x132e120, 0xc04238ee10, 0x0, 0x0, 0x0)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cmd/deployment_preparer.go:203 +0xcc3
github.com/cloudfoundry/bosh-cli/cmd.(*CreateEnvCmd).Run(0xc04220cfd8, 0x132e120, 0xc04238ee10, 0x133a540, 0xc04224a180, 0xc042310000, 0x1364, 0x1e00, 0xc04224b800, 0x32, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cmd/create_env.go:25 +0x283
github.com/cloudfoundry/bosh-cli/cmd.Cmd.Execute(0xef6550, 0xd3389c, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/cmd/cmd.go:74 +0x1f30
main.main()
/tmp/build/734c03a2/gopath/src/github.com/cloudfoundry/bosh-cli/main.go:35 +0x764

********************
Exit code 1

This error is consistent across both the command prompt and PowerShell.

As the following line is hardcoded into both the cpi_cmd_runner.go and compiler.go source files:

UseIsolatedEnv: true,

as well as a few test files, the only assertion that I can make at this point is that it’s currently not possible to deploy a bosh-lite version of cf-deployment directly onto Windows 10.

That’s not to say we can’t get Cloud Foundry running on a Windows 10 box, just that doing it directly isn’t currently a viable option.

Posted in Uncategorized | 2 Comments

Deploying Cloud Foundry – On Vagrant, On VirtualBox, On Windows 10

Posted on May 4, 2017 by Graham Nicol

Update 8th June 2018

These instructions, based on the cf-release package, are deprecated. Cloud Foundry now uses the cf-deployment package, which I cover elsewhere.

Introduction

I set out to try and get Cloud Foundry working on my Windows 10 machine. Searching around the web I found several different articles, most of which dated from 2015 or earlier, that got me a good way through the installation process but which had become slightly outdated due to changes in the Cloud Foundry stack. As such I thought it was worthwhile publishing this version of the instructions which document the steps I followed in May 2017.

A word of warning

Both Cloud Foundry and bosh-lite were really designed to run on Linux. That’s not to say it’s impossible to get it working on Windows (after all, that’s what we’re doing here) but, as you’ll see, there are a lot of hidden gotchas that will jump out to bite you if your host machine is running Windows.

Pre-Requisites

Before installing Cloud Foundry on top of bosh-lite hosted on a Vagrant VM running on top of VirtualBox VM there are four self-evident pre-requisites

  1. Install Oracle VirtualBox (make sure you also install the Extension pack, CF networking makes use of this)
  2. Install Vagrant
  3. Install Git
  4. Install an SSH client (such as the excellent PuTTY, the remainder of this article assumes that you are using PuTTY for SSH)

There are some other needed tools and downloads, but we’ll come to those along the way.

After the pre-requisite elements have been installed we’re ready to start creating a bosh-lite Vagrant VM.

Creating the VM

  1. Open a Windows Command Prompt.
  2. Get a local copy of the bosh-lite repository by cloning https://github.com/cloudfoundry/bosh-lite

git clone https://github.com/cloudfoundry/bosh-lite

  1. Navigate to the bosh-litefolder on your Windows host and start Vagrant

vagrant up --provider=virtualbox

  1. The next step is to log into the newly created Vagrant VM. Windows doesn’t include ssh as a program so if you enter vagrant ssh in the command prompt it will throw up an error. However, it will also display a link to the SSH Private Key. Note this address and use PuTTYgen to generate a .ppk file based on this key.
  2. Set up PuTTY to connect to 127.0.0.1 on Port 2222 using the .ppk file generated above for the SSH key.
  3. Log in to the Vagrant VM as the username vagrant. The SSH key will then authenticate for this user.

From this point forward, until we have a CF deployment, you will be working in the PuTTY session opened on the Vagrant VM. It will start you off in the home directory for the user /home/vagrant/

Creating a Cloud Foundry deployment

  1. Update the package catalogue on the VM. By default, a minimal catalogue is loaded so you would be unable to find any required software packages to install.

    2. sudo apt-get update

  1. Target the Bosh Director. Log in using admin/admin

bosh target 192.168.50.4 lite

  1. The Cloud Foundry instructions include a step to run the script at /vagrant/bin/add-route to add a set of route entries to the local route table to facilitate direct Warden container access if the network changes.
    However, this folder resides on the Windows host and as such, the add-route script contains DOS line breaks. This then causes the script to fail when you try and run it from within the Vagrant box. To work around this, we need to use the dos2unix utility to change this both on the add-route script and on the provision_cf script, and then execute the add-route script:

sudo apt-get install dos2unix
cd /vagrant/bin
dos2unix add-route
dos2unix provision_cf
./add-route
cd ~

  1. Download a bosh stemcell using the bosh cli tool

bosh download public stemcell bosh-stemcell-389-warden-boshlite-ubuntu-trusty-go_agent.tgz

  1. Once downloaded to the Vagrant VM then we upload the stemcell to our bosh instance

bosh upload stemcell bosh-stemcell-389-warden-boshlite-ubuntu-trusty-go_agent.tgz

  1. Install git onto the Vagrant VM

sudo apt-get install git

  1. Clone the CloudFoundry repository https://github.com/cloudfoundry/cf-release

git clone https://github.com/cloudfoundry/cf-release

  1. Navigate into the folder that has been created

cd cf-release

  1. The sub-folder releases contains yml files for each release in Cloud Foundry. Select which release you would like to deploy and check it out from Git, we will use Release 2.57

git checkout tags/v257

  1. From the cf-release folder run the update script to update the dependencies.

./scripts/update

  1. Upload the release to bosh

bosh upload release releases/cf-257.yml

  1. Install bundler

sudo gem install bundler

  1. Install spiff

sudo apt-get install unzip

cd ~

mkdir spiff

cd spiff

wget https://github.com/cloudfoundry-incubator/spiff/releases/download/v1.0.8/spiff_linux_amd64.zip

unzip spiff_linux_amd64.zip

export PATH=$PATH:~/spiff

  1. Edit the provision_cf script. When I first tried running provision_cf I kept running into errors that it couldn’t find certain parts of the provision script. In a pure Linux world, the script itself seems fairly “self-healing” and would make a sensible choice about locations. However, this is another point that the ghost of the Windows host machine creeps out and shouts “BOO!”.

cd /vagrant/bin
sudo nano provision_cf

In the editor, change the fifth line of the file so that the CF_DIR variable points to the cf-release folder we created in step 17

On my machine this equates to:

CF_DIR="//home/vagrant/cf-release"

Exit and Save the file.

Deploying

  1. This should be as simple as typing

sudo ./provision_cf

Testing and Connecting

Of course, nothing is ever quite that simple. If step 25 has worked you now have a functioning Cloud Foundry deployment.

  1. From within the Vagrant VM we can verify the version and check that the VMs are spooled up.

cd ~/cf-release

bosh status

bosh deployments

bosh cck
To connect to Cloud Foundry and make use of this new shiny PaaS a little extra work still needs to happen on the Windows Host.

The next steps are all executed on the Windows host machine.

  1. Install the CF cli from here
  2. Open a Command Prompt with Elevated Privileges
    1. Click the Start Menu
    2. Type cmd
    3. An icon will appear for “Command Prompt”, right click that icon and select “Run as administrator”
  3. Navigate back to the bosh-lite folder that was created as a part of the git clone command in step 6
  4. Change into the bin folder and run add-route.bat.
    Note: Technically we could run this directly from Windows Explorer by right-clicking the file and selecting “Run as administrator” from the context menu. However, as it’s a .bat file we wouldn’t be able to see the success or otherwise of the command contained in the bat file, so run it from the command prompt!
  5. Set the target to the new CF instance
    cf api --skip-ssl-validation api.bosh-lite.com
  6. Login as admin and start using the new PaaS

A few final thoughts

Like many technical walkthroughs, this one is built on the hard work and knowledge of others. Without the key information published here, here, here (I know it’s a Mac install, but it had a lot of useful info that helped compare and contrast) and the great efforts of the GitHub community (especially David Sabeti in the cf-release repo) I don’t think I would have managed to get it all working.

Update 8th June 2018

If you’ve worked through these instructions you may or may not have something that *works*. Towards the end of the cf-release package’s “life” something seems to have changed and although the release that cf-release makes is a warden release, certain buildpacks seem to start trying to use the Diego bbs, which isn’t included in the deployment.

As noted above, cf-release is now deprecated, use cf-deployment instead… although as we’ll see, that’s a pathway strewn with pitfalls in itself.

Posted in Cloud Foundry | Tagged , , , , , | 5 Comments