Kitchen CI with Chef Zero and Chef Shell

A quick setup to get you started with Chef development and quick prototyping.

We will use Kitchen CI to accomplish the following,

  • Create the Chef code directory scaffolding and develop
  • Manage the Vagrant host and converge and test

And then configure Chef-Zero to enable testing locally on the Vagrant host. Chef zero is nothing but Chef in solo (being deprecated) mode with an in-memory Chef Server. Having Chef Server around will enable you to code as close to Server-based environment without having to check for the mode (eg:- if Chef::Config[:solo] … ) anywhere.

Chef-Shell is the way we will be trouble shooting logical issues in detail. It allows step-by-step execution with access to the entire state (variables and Ohai objects etc.) of execution.

Lets start…

Kitchen CI

As a pre-requisit, install Chefdk which will be installed to /opt/chefdk

exponentof:~ > ln -s /opt/chefdk/embedded/bin/gem /usr/bin/gem
exponentof:~ > /opt/chefdk/embedded/bin/gem install  test-kitchen
exponentof:~ > sudo ln -s /opt/chefdk/embedded/bin/kitchen /usr/bin/kitchen
exponentof:~ > kitchen --version
Test Kitchen version 1.7.3
exponentof:~ > mkdir kitchen-ci/cookbooks
exponentof:~ > cd kitchen-ci/cookbooks
exponentof:~/kitchen-ci/cookbooks > git init my-web
Initialized empty Git repository in /Users/nmeethal/kitchen-ci/cookbooks/my-web/.git/
exponentof:~/kitchen-ci/cookbooks > cd my-web/
exponentof:~/kitchen-ci/cookbooks/my-web >
exponentof:~/kitchen-ci/cookbooks/my-web > vi metadata.rb
exponentof:~/kitchen-ci/cookbooks/my-web > cat metadata.rb
name "my-web"
version "0.1.0"
exponentof:~/kitchen-ci/cookbooks/my-web > mkdir recipes
exponentof:~/kitchen-ci/cookbooks/my-web > touch recipes/default.rb
exponentof:~/kitchen-ci/cookbooks/my-web > git add metadata.rb recipes/default.rb
exponentof:~/kitchen-ci/cookbooks/my-web > git status
On branch master
Initial commit
Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
     new file:   metadata.rb
     new file:   recipes/default.rb

exponentof:~/kitchen-ci/cookbooks/my-web > git commit -m "Web Cookbooks - initial commit"
[master (root-commit) 58e1b7e] Web Cookbooks - initial commit
 2 files changed, 2 insertions(+)
 create mode 100644 metadata.rb
 create mode 100644 recipes/default.rb
exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen init --driver=kitchen-vagrant
      create  .kitchen.yml
      create  chefignore
      create  test/integration/default
      create  .gitignore
      append  .gitignore
      append  .gitignore
Fetching: kitchen-vagrant-0.20.0.gem (100%)
Successfully installed kitchen-vagrant-0.20.0
1 gem installed

exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen create default-centos-72
-----> Starting Kitchen (v1.7.3)
-----> Creating <default-centos-72>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'bento/centos-7.2' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Loading metadata for box 'bento/centos-7.2'
           default: URL: https://atlas.hashicorp.com/bento/centos-7.2
….
….

exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen list
Instance             Driver   Provisioner  Verifier  Transport  Last Action
default-ubuntu-1404  Vagrant  ChefSolo     Busser    Ssh        <Not Created>
default-centos-72    Vagrant  ChefSolo     Busser    Ssh        Created
exponentof:~/kitchen-ci/cookbooks/my-web(master) > git add .gitignore .kitchen.yml
exponentof:~/kitchen-ci/cookbooks/my-web(master) >  git commit -m "adding test kitchen"
[master 53d04bd] adding test kitchen
 2 files changed, 18 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 .kitchen.yml
exponentof:~/kitchen-ci/cookbooks/my-web(master) > vi recipes/default.rb
exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen converge default-centos-72
-----> Starting Kitchen (v1.7.3)
-----> Converging <default-centos-72>...
       Preparing files for transfer
       Preparing dna.json
       Preparing current project directory as a cookbook
       Removing non-cookbook files before transfer
       Preparing solo.rb
-----> Installing Chef Omnibus (install only if missing)
       Downloading https://www.chef.io/chef/install.sh to file /tmp/install.sh
       Trying wget...
       Download complete.
       el 7 x86_64
       Getting information for chef stable  for el...
       downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=&p=el&pv=7&m=x86_64
         to file /tmp/install.sh.2283/metadata.txt
       trying wget...
       sha1     5907edce1a3b0f7bd42359fe64960da8833385e9
       sha256     1c3e680f106ab6829c3713307e447116f7bb3f2d9c30fd3943638b01d6246fe2
       url     https://packages.chef.io/stable/el/7/chef-12.9.38-1.el7.x86_64.rpm
       version     12.9.38
       downloaded metadata file looks valid...
       downloading https://packages.chef.io/stable/el/7/chef-12.9.38-1.el7.x86_64.rpm
         to file /tmp/install.sh.2283/chef-12.9.38-1.el7.x86_64.rpm
       trying wget...
       Comparing checksum with sha256sum...

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       You are installing an omnibus package without a version pin.  If you are installing
       on production servers via an automated process this is DANGEROUS and you will
       be upgraded without warning on new releases, even to new major releases.
       Letting the version float is only appropriate in desktop, test, development or
       CI/CD environments.

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       Installing chef
       installing with rpm...
       warning: /tmp/install.sh.2283/chef-12.9.38-1.el7.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
       Preparing...                          ################################# [100%]
       Updating / installing...
          1:chef-12.9.38-1.el7               ################################# [100%]
       Thank you for installing Chef!
       Transferring files to <default-centos-72>
       Starting Chef Client, version 12.9.38
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: my-web::default
         * yum_package[git] action install
           - install version 1.8.3.1-6.el7_2.1 of package git
         * log[Well, that was too easy] action write

       Running handlers:
       Running handlers complete
       Chef Client finished, 2/2 resources updated in 09 seconds
       Finished converging <default-centos-72> (0m30.38s).
-----> Kitchen is finished. (0m30.64s)
exponentof:~/kitchen-ci/cookbooks/my-web(master) >
exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen converge default-centos-72  # repeat
-----> Starting Kitchen (v1.7.3)
-----> Converging <default-centos-72>...
       Preparing files for transfer
       Preparing dna.json
       Preparing current project directory as a cookbook
       Removing non-cookbook files before transfer
       Preparing solo.rb
-----> Chef Omnibus installation detected (install only if missing)
       Transferring files to <default-centos-72>
       Starting Chef Client, version 12.9.38
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: my-web::default
         * yum_package[git] action install (up to date)
         * log[Well, that was too easy] action write

       Running handlers:
       Running handlers complete
       Chef Client finished, 1/2 resources updated in 04 seconds
       Finished converging <default-centos-72> (0m6.44s).
-----> Kitchen is finished. (0m6.70s)
exponentof:~/kitchen-ci/cookbooks/my-web(master) >

exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen list
Instance             Driver   Provisioner  Verifier  Transport  Last Action
default-ubuntu-1404  Vagrant  ChefSolo     Busser    Ssh        <Not Created>
default-centos-72    Vagrant  ChefSolo     Busser    Ssh        Converged
exponentof:~/kitchen-ci/cookbooks/my-web(master) >
exponentof:~/kitchen-ci/cookbooks/my-web(master) > git add recipes/default.rb
exponentof:~/kitchen-ci/cookbooks/my-web(master) > git commit -m "default recipe"
[master 79814dc] default recipe
 1 file changed, 3 insertions(+)

Chef Zero

exponentof:~/kitchen-ci/cookbooks/my-web(master) &gt; kitchen login default-centos-72
Last login: Sat Apr 23 15:15:39 2016 from 10.0.2.2
[vagrant@default-centos-72 ~]$ which git
/usr/bin/git
[vagrant@default-centos-72 ~]$ git --version
git version 1.8.3.1
[vagrant@default-centos-72 ~]$ sudo bash
[root@default-centos-72 /]# cd /tmp/kitchen
[root@default-centos-72 kitchen]# /opt/chef/embedded/bin/gem instal chef-zero
Fetching: chef-zero-4.6.1.gem (100%)
Successfully installed chef-zero-4.6.1
Parsing documentation for chef-zero-4.6.1
Installing ri documentation for chef-zero-4.6.1
Done installing documentation for chef-zero after 1 seconds
1 gem installed
[root@default-centos-72 kitchen]#
[root@default-centos-72 kitchen]# /opt/chef/embedded/bin/chef-zero -d
[root@default-centos-72 kitchen]# ps aux | grep chef-zero
root      5545  0.0  4.2 265284 20128 ?        Sl   15:30   0:00 /opt/chef/embedded/bin/ruby /opt/chef/embedded/bin/chef-zero -d
root      5550  0.0  0.2 112644   956 pts/0    R+   15:30   0:00 grep --color=auto chef-zero
[root@default-centos-72 kitchen]#

Manually prepping to run chef-zero. the .chef/ files were copied from the local chef-zero repo cloned from https://github.com/chef/chef-zero.git

[root@default-centos-72 kitchen]# mkdir .chef
[root@default-centos-72 kitchen]# vi .chef/knife.rb
[root@default-centos-72 kitchen]# vi stickywicket.pem
[root@default-centos-72 kitchen]# knife cookbook upload -a -c .chef/knife.rb
Uploading my-web      [0.1.0]
Uploaded all cookbooks.
[root@default-centos-72 kitchen]#

Chef Shell

[root@default-centos-72 kitchen]# chef-shell -z  -c .chef/knife.rb -o "my-web"
loading configuration: .chef/knife.rb
Session type: client
Loading........[2016-04-24T11:40:15+00:00] WARN: Run List override has been provided.
[2016-04-24T11:40:15+00:00] WARN: Original Run List: []
[2016-04-24T11:40:15+00:00] WARN: Overridden Run List: [recipe[my-web]]
resolving cookbooks for run list: ["my-web"]
Synchronizing Cookbooks:
  - my-web (0.1.0)
done.

This is the chef-shell.
 Chef Version: 12.9.38
 http://www.chef.io/
 http://docs.chef.io/

run `help' for help, `exit' or ^D to quit.

Ohai2u vagrant@default-centos-72!
chef (12.9.38)>
chef (12.9.38)> run_chef
[2016-04-24T11:34:40+00:00] INFO: Processing yum_package[git] action install (my-web::default line 1)
[2016-04-24T11:34:40+00:00] DEBUG: Providers for generic yum_package resource enabled on node include: [Chef::Provider::Package::Yum]
[2016-04-24T11:34:40+00:00] DEBUG: Provider for action install on resource yum_package[git] is Chef::Provider::Package::Yum
[2016-04-24T11:34:40+00:00] DEBUG: yum_package[git] checking yum info for git
[2016-04-24T11:34:40+00:00] DEBUG: yum_package[git] installed version: ["1.8.3.1-6.el7_2.1"] candidate version: 1.8.3.1-6.el7_2.1
[2016-04-24T11:34:40+00:00] DEBUG: yum_package[git] git 1.8.3.1-6.el7_2.1 already installed
[2016-04-24T11:34:40+00:00] DEBUG: yum_package[git] is already installed - nothing to do
[2016-04-24T11:34:40+00:00] INFO: Processing log[Well, that was too easy] action write (my-web::default line 3)
[2016-04-24T11:34:40+00:00] DEBUG: Providers for generic log resource enabled on node include: [Chef::Provider::Log::ChefLog]
[2016-04-24T11:34:40+00:00] DEBUG: Provider for action write on resource log[Well, that was too easy] is Chef::Provider::Log::ChefLog
[2016-04-24T11:34:40+00:00] INFO: Well, that was too easy
 => true
chef (12.9.38)>
chef >
chef > chef_run.re
Display all 458 possibilities? (y or n)
chef > chef_run.rewind
 => 0
chef (12.9.38)> chef_run.step
[2016-04-24T11:34:58+00:00] INFO: Processing yum_package[git] action install (my-web::default line 1)
 => 1
chef (12.9.38)> chef_run.step
[2016-04-24T11:35:02+00:00] INFO: Processing log[Well, that was too easy] action write (my-web::default line 3)
[2016-04-24T11:35:02+00:00] INFO: Well, that was too easy
 => 2
chef (12.9.38)> chef_run.step
 => nil
chef (12.9.38)>
chef (12.9.38)> pp node.attributes.os
"linux"
 => "linux"
chef (12.9.38)>
exponentof:~/kitchen-ci/cookbooks/my-web(master) > vi test/integration/default/bats/git_installed.bats
exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen verify default-centos-72
-----> Starting Kitchen (v1.7.3)
-----> Setting up <default-centos-72>...
...
...
exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen list
Instance             Driver   Provisioner  Verifier  Transport  Last Action
default-ubuntu-1404  Vagrant  ChefSolo     Busser    Ssh        <Not Created>
default-centos-72    Vagrant  ChefSolo     Busser    Ssh        Verified
exponentof:~/kitchen-ci/cookbooks/my-web(master) >

Kitchen Test = Destroy + Create + Converge + Busser & Runner + Verify + Desrtoy

exponentof:~/kitchen-ci/cookbooks/my-web(master) > kitchen test

Best of Luck!!