Chef DSM Authentication and authorization

The authentication process ensures that Chef DSM only responds to requests made by trusted users or clients. Chef DSM uses public key encryption. You create the public and private keys when you configure Chef Infra Client or setup Chef Workstation.

• Chef DSM stores the public key
• Chef Workstation saves the private key in ~/.chef/
• Chef Infra Client saves the private key in /etc/chef

Both Chef Infra Client and Chef Workstation communicate with Chef DSM using the Chef DSM API. Each time that Chef Infra Client or Chef Workstation makes a request to Chef DSM, they use a special group of HTTP headers and sign the rest with their private key. Chef DSM then uses the public key to verify the headers and contents.

Public and private keys

Chef DSM key ue

The authentication process ensures that Chef DSM only responds to requests made by trusted users or clients. Chef DSM uses public key encryption. You create the public and private keys when you configure Chef Infra Client or setup Chef Workstation.

• Chef DSM stores the public key
• Chef Workstation saves the private key in ~/.chef/
• Chef Infra Client saves the private key in /etc/chef

Both Chef Infra Client and Chef Workstation communicate with Chef DSM using the Chef DSM API. Each time that Chef Infra Client or Chef Workstation makes a request to Chef DSM, they use a special group of HTTP headers and sign the rest with their private key. Chef DSM then uses the public key to verify the headers and contents.

Chef Infra Client

knife

RSA public key-pairs are used to authenticate knife with Chef DSM every time knife attempts to access Chef DSM. This ensures that each instance of knife is properly registered with Chef DSM and that only trusted users can make changes to the data.

Knife can also use the knife exec subcommand to make specific, authenticated requests to Chef DSM. knife plugins can also make authenticated requests to Chef DSM by leveraging the knife exec subcommand.

chef-validator

The private key doesn’t yet exist the first time that Chef Infra Client runs from a new node.

During the first Chef Infra Client run:

1. Chef Infra Client uses the chef-validator private key, located in /etc/chef/validation.pem to register with Chef DSM
2. Chef DSM assigns Chef Infra Client a private key for all future authentication requests to the Chef DSM
3. Chef Infra Client saves the private key on the node as /etc/chef/client.pem

If the request to communicate with Chef DSM with the chef-validator key fails, then the entire first Chef Infra Client run fails.

After the first completed Chef Infra Client run, delete the chef-validator private key at /etc/chef/validation.pem

Chef DSM key storage

Keys are stored in different locations, depending on if the location is a node or a workstation.

Nodes

Each node stores its private key locally. This private key is generated as part of the bootstrap process that initially installs Chef Infra Client on the node. The first time Chef Infra Client runs on that node, it uses the chef-validator to authenticate, but then on each subsequent run it uses the private key generated for that client by the Chef DSM.

Workstations

Each workstation stores its private key in the user’s ~/.chef directory. This private key is generated by Chef DSM and must be download from the server and copied to the ~/.chef directory manually. If you require a new private key, generate it with Chef DSM and copy it to the ~/.chef directory again.

The chef-repo is a directory on your workstation that stores everything you need to define your infrastructure with Chef Infra:

• Cookbooks (including recipes, attributes, custom resources, libraries, and templates)
• Data bags
• Policyfiles

The chef-repo directory should be synchronized with a version control system, such as git. All of the data in the chef-repo should be treated like source code.

You’ll use the chef and knife commands to upload data to the Chef DSM from the chef-repo directory. Once uploaded, Chef Infra Client uses that data to manage the nodes registered with the Chef Infra Server and to ensure that it applies the right cookbooks, policyfiles, and settings to the right nodes in the right order.

The .chef directory is a hidden directory that’s used to store validation key files and optionally a config.rb file.

Chef DSM API authentication

API requests

A knife plugin can be used to make authenticated API requests to the Chef DSM using the following methods:

For example:

module MyCommands
  class MyNodeDelete < Chef::Knife
    #An implementation of knife node delete
    banner 'knife my node delete [NODE_NAME]'

    def run
      if name_args.length < 1
        show_usage
        ui.fatal('You must specify a node name.')
        exit 1
      end
      nodename = name_args[0]
      api_endpoint = "nodes/#{nodename}"
      # Again, we could just call rest.delete_rest
      nodey = rest.get_rest(api_endpoint)
      ui.confirm("Do you really want to delete #{nodey}")
      nodey.destroy
    end
  end
end

Other options

The most common ways to interact with Chef DSM using the Chef DSM API abstract the API from the user. That said, the Chef DSM API can be interacted with directly. The following sections describe a few of the ways that are available for doing that.

curl

An API request can be made using cURL, which is a Bash shell script that requires two utilities: awk and openssl. The following example shows how an authenticated request can be made using the Chef DSM API and cURL:

#!/usr/bin/env bash

_chef_dir () {
  # Helper function:
  # Recursive function that searches for chef configuration directory
  # It looks upward from the cwd until it hits /. If no directory is found,
  # ~/.chef is chosen if it exists
  # You could simply hard-code the path below

  if [ "$PWD" = "/" ]; then
  if [ -d ".chef" ]; then
    echo "/.chef"
      elif [ -d "$HOME/.chef" ]; then
        echo "$HOME/.chef"
      fi
    return
  fi

  if [ -d '.chef' ];then
    echo "${PWD}/.chef"
  else
    (cd ..; _chef_dir)
  fi
}

_chomp () {
  # helper function to remove newlines
  awk '{printf "%s", $0}'
}

chef_api_request() {
  # This is the meat-and-potatoes, or rice-and-vegetables, your preference really.

  local method path body timestamp chef_server_url client_name hashed_body hashed_path
  local canonical_request headers auth_headers

  chef_server_url="https://api.opscode.com/organizations/my_org"
  # '/organizations/ORG_NAME' is needed
  if echo $chef_server_url | grep -q "/organizations/" ; then
    endpoint=/organizations/${chef_server_url#*/organizations/}${2%%\?*}
  else
    endpoint=${2%%\?*}
  fi
  path=${chef_server_url}$2
  client_name="chef_user"
  method=$1
  body=$3

  hashed_path=$(echo -n "$endpoint" | openssl dgst -sha1 -binary | openssl enc -base64)
  hashed_body=$(echo -n "$body" | openssl dgst -sha1 -binary | openssl enc -base64)
  timestamp=$(date -u "+%Y-%m-%dT%H:%M:%SZ")

  canonical_request="Method:$method\nHashed Path:$hashed_path\nX-Ops-Content-Hash:$hashed_body\nX-Ops-Timestamp:$timestamp\nX-Ops-UserId:$client_name"
  headers="-H X-Ops-Timestamp:$timestamp \
    -H X-Ops-Userid:$client_name \
    -H X-Chef-Version:0.10.4 \
    -H Accept:application/json \
    -H X-Ops-Content-Hash:$hashed_body \
    -H X-Ops-Sign:version=1.0"

  auth_headers=$(printf "$canonical_request" | openssl rsautl -sign -inkey \
    "$(_chef_dir)/${client_name}.pem" | openssl enc -base64 | _chomp |  awk '{ll=int(length/60);i=0; \
    while (i<=ll) {printf " -H X-Ops-Authorization-%s:%s", i+1, substr($0,i*60+1,60);i=i+1}}')

  case $method in
    GET)
      curl_command="curl $headers $auth_headers $path"
      $curl_command
      ;;
    *)
      echo "Unknown Method. I only know: GET" >&2
      return 1
      ;;
    esac
  }

 chef_api_request "$@"

After saving this shell script to a file named chef_api_request, use it similar to the following:

bash chef_api_request GET "/clients"

PyChef

An API request can be made using PyChef, which is a Python library that meets the Mixlib::Authentication requirements so that it can easily interact with Chef DSM. The following example shows how an authenticated request can be made using the Chef DSM API and PyChef:

from chef import autoconfigure, Node

api = autoconfigure()
n = Node('web1')
print n['fqdn']
n['myapp']['version'] = '1.0'
n.save()

and the following example shows how to make API calls directly:

from chef import autoconfigure

api = autoconfigure()
print api.api_request('GET', '/clients')

The previous examples assume that the current working directory is such that PyChef can find a valid configuration file in the same manner as Chef Infra Client or knife. For more about PyChef, see: https://github.com/coderanger/pychef.

Ruby

On a system with Chef Infra Client installed, use Ruby to make an authenticated request to Chef DSM:

require 'chef/config'
require 'chef/log'
require 'chef/rest'

chef_server_url = 'https://example.com'
client_name = 'client_name'
signing_key_filename = '/path/to/pem/for/client_name'

rest = Chef::REST.new(chef_server_url, client_name, signing_key_filename)
puts rest.get_rest('/clients')

or:

require 'mixlib/cli'
require 'chef'
require 'chef/node'
require 'chef/mixin/xml_escape'
require 'json'

config_file = 'c:/chef/client.rb'
Chef::Config.from_file(config_file)
Chef::Log.level = Chef::Config[:log_level]

def Usage()
  puts '/etc/chef/client.rb' # The config file location, for example ~/home/.chef/config.rb etc
  config_file = gets.chomp
  if (!File.exist?(config_file))
    puts 'config_file #{config_file} doesn't exist. Exiting.\n'
    exit
  end
  STDOUT.puts <<-EOF
    Choose options for example 1

    1 Display all nodes per environment
    2 Display all nodes in detail (can be slow if there a large number of nodes)
    9 Exit
  EOF
end

def ExecuteUserChoice()
  testoption = gets.chomp
  case testoption
  when '1'
    Execute(method(:DisplayNodesPerEnv))
  when '2'
    Execute(method(:DisplayNodesDetail))
  when '9'
    puts 'exit'
  else
    puts 'Unknown option #{testoption}. Exiting\n'
    exit
  end
end

def DisplayNodesPerEnv()
  Chef::Environment.list(false).each do |envr|
    print 'ENVIRONMENT: ', envr[0], '\n'
    Chef::Node.list_by_environment(envr[0], false).each do |node_info|
      print '\tNODE: ', node_info[0], '\n'
      print '\t\tURL: ', node_info[1], '\n'
    end
  end
end

def DisplayNodesDetail()
  Chef::Node.list(true).each do |node_array|
    node = node_array[1]
    print '#{node.name}\n'
    print '\t#{node['fqdn']}\n'
    print '\t#{node['kernel']['machine']}\n'
    print '\t#{node['kernel']['os']}\n'
    print '\t#{node['platform']}\n'
    print '\t#{node['platform_version']}\n'
    print '\t#{node.chef_environment}\n'
    print '\t#{node.run_list.roles}\n'
  end
end

def Execute(option)
  begin
    profilestart = Time.now
    option.call()
    profileend = Time.now
    timeofrun = profileend - profilestart
    print 'Time taken = #{timeofrun}'
  rescue Exception => ex
    print 'Error calling chef API'
    print ex.message
    print ex.backtrace.join('\n')
  end
end

Usage()
ExecuteUserChoice()

Another way Ruby can be used with the Chef DSM API is to get objects from Chef DSM, and then interact with the returned data using Ruby methods. Whenever possible, the Chef DSM API returns an object of the relevant type. The returned object is then available to be called by other methods. For example, the api.get method can be used to return a node named foobar, and then .destroy can be used to delete that node:

silly_node = api.get('/nodes/foobar')
silly_node.destroy

Debug authentication issues

In some cases, Chef Infra Client may receive a 401 response to the authentication request and a 403 response to an authorization request. An authentication error error may look like the following:

[Wed, 05 Oct 2011 15:43:34 -0700] INFO: HTTP Request Returned 401
Unauthorized: Failed to authenticate as node_name. Ensure that your node_name and client key are correct.

To debug authentication problems, determine which Chef Infra Client is attempting to authenticate. This is often found in the log messages for that Chef Infra Client. Debug logging can be enabled on a Chef Infra Client using the following command:

chef-client -l debug

When debug logging is enabled, a log entry will look like the following:

[Wed, 05 Oct 2011 22:05:35 +0000] DEBUG: Signing the request as NODE_NAME

If the authentication request occurs during the initial Chef Infra Client run, the issue is most likely with the private key.

If the authentication is happening on the node, there are a number of common causes:

• The client.pem file is incorrect. This can be fixed by deleting the client.pem file and re-running Chef Infra Client. When Chef Infra Client re-runs, it will re-attempt to register with the Chef DSM and generate the correct key.
• A node_name is different from the one used during the initial Chef Infra Client run. This can happen for a number of reasons. For example, if the client.rb file doesn’t specify the correct node name and the host name has recently changed. This issue can be resolved by explicitly setting the node name in the client.rb file or by using the -N option for the Chef Infra Client executable.
• The system clock has drifted from the actual time by more than 15 minutes. This can be fixed by syncing the clock with an Network Time Protocol (NTP) server.

Update a user’s key pair for authenticating with Chef DSM

You can update a user’s key pair on Chef DSM with knife using either the knife user reregister subcommand, or the knife user key subcommands.

knife user reregister

Use knife user reregister to regenerate an RSA key pair for a user. Knife will store the public key on Chef DSM and the private key will be displayed in the standard output, or use the --file option to write to a named file.

knife user reregister USERNAME (options)

knife user key

You can list, add, edit, and delete public keys using the following subcommands:

• knife user key create
• knife user key delete
• knife user key list
• knife user key edit

To update a user’s key pair:

1. Check the current keys associated with the user:

   knife user key list USERNAME
   

2. Create a new key pair:

   knife user key create USERNAME --key-name KEYNAME --expiration-date YYYY-MM-DDTHH:MM:SSZ --file FILENAME
   

   Knife will open your text editor with a data file containing the username, key name, and key pair expiration date that will be sent to Chef DSM.

   Modify the username, key name, and key expiration date to match the new key pair that you are creating, then save the file and close your editor.

   Knife will also generate a new private key (PEM file) using the specified filename.

   Note

   Specify the expiration date in ISO 8601 format.

   The expiration date is optional. User keys don’t expire if an expiration date isn’t specified.

3. Make the new user key active by placing the generated PEM file in the .chef directory on your workstation.

4. Open your config.rb file or credentials file and modify it to match the new key name.

5. Check the list of current keys associated with the user:

   knife user key list USERNAME
   

6. Delete any old or unwanted keys to reduce security risks:

   knife user key delete USERNAME OLD_KEY_NAME
   

7. Check the list of current keys associated with the user to verify that the new key has been added and any older keys have been deleted:

   knife user key list USERNAME
   

Authorization

For more information about Chef DSM Authorization, see Organizations and Groups.

Chef DSM API

For more information about using the Chef DSM API endpoints see the Chef DSM API documentation.