Since releasing a Campfire LWRP for Chef a few weeks ago, my team has evaluated and subsequently transitioned to Atlassian’s HipChat service. Luckily I was able to reuse the framework I had already created for Campfire as the basis for a HipChat LWRP.

The LWRP should work with any modern version of Chef. When you use include_recipe to access the LWRP in your own recipes, the default recipe for this cookbook will install the required ‘hipchat’ gem.

Attributes

• room - the name of the room you would like to speak into (requied).
• token - authentication token for your HipChat account (required).
• nickname - the nickname to be used when speaking the message (required).
• message - the message to speak. If a message is not specified, the name of the hipchat_msg resource is used.
• notify - toggles whether or not users in the room should be notified by this message (defaults to true).
• color - sets the color of the message in HipChat. Supported colors include: yellow, red, green, purple, or random (defaults to yellow).
• failure_ok - toggles whether or not to catch the exception if an error is encountered connecting to HipChat (defaults to true).

Usage example

1
2
3
4
5
6
7
8
9

include_recipe 'hipchat'

hipchat_msg 'bad news' do
  room 'The Pod Bay'
  token '0xdedbeef0xdedbeef0xdedbeef'
  nickname 'HAL9000'
  message "Sorry Dave, I'm afraid I can't do that: #{some_error}"
  color 'red'
end

Availability

You can find this cookbook on github or on the Opscode community site.

This weekend I decided that I’d had enough with reusing the same pattern for manipulating tags on EC2 instances across multiple recipes. Since Opscode already publishes an aws cookbook with providers for other AWS resources, I figured it would be worthwhile to create a provider for manipulating these tags and contribute it back upstream.

The result of this Saturday project is the resource_tag LWRP. Source available here, Opscode ticket here.

Actions

• add - Add tags to a resource.
• update - Add or modify existing tags on a resource – this is the default action.
• remove - Remove tags from a resource, but only if the specified values match the existing ones.
• force_remove - Remove tags from a resource, regardless of their values.

Attribute Parameters

• aws_secret_access_key, aws_access_key - passed to Opscode::AWS:Ec2 to authenticate, required.
• tags - a hash of key value pairs to be used as resource tags, (e.g. { "Name" => "foo", "Environment" => node.chef_environment },) required.
• resource_id - resources whose tags will be modified. The value may be a single ID as a string or multiple IDs in an array. If no resource_id is specified the name attribute will be used.

Usage

resource_tag can be used to manipulate the tags assigned to one or more AWS resources, i.e. ec2 instances, ebs volumes or ebs volume snapshots.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

include_recipe "aws"
aws = data_bag_item("aws", "main")

# Assigining tags to a node to reflect it's role and environment:
aws_resource_tag node['ec2']['instance_id'] do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  tags({"Name" => "www.example.com app server",
        "Environment" => node.chef_environment})
  action :update
end

# Assigning a set of tags to multiple resources, e.g. ebs volumes in a disk set:
aws_resource_tag 'my awesome raid set' do
  aws_access_key aws['aws_access_key_id']
  aws_secret_access_key aws['aws_secret_access_key']
  resource_id [ "vol-d0518cb2", "vol-fad31a9a", "vol-fb106a9f", "vol-74ed3b14" ]
  tags({"Name" => "My awesome RAID disk set",
        "Environment" => node.chef_environment})
end

When setting tags on the node’s own EC2 instance, I recommend wrapping resource_tag resources in a conditional like if node.has_key?('ec2') so that your recipe will still run on Chef nodes outside of EC2 as well.

Like many small teams, Needle uses 37signals’ Campfire chat platform for collaborating online. Along with messages exchanged between coworkers, we also use Campfire for announcing new git commits, jira tickets and successful application deployments.

Since the code I’ve been using to send messages from Chef recipes to Campfire is virtually identical between a number of our cookbooks, I decided to turn that code into a LWRP that anyone can use in their own recipes. The cookbook for this LWRP is available on github.

Requirements

• a Campfire API token (these are unique to each Campfire user, so if you want your messages to come from a particular user, get their token)
• the tinder gem (installed by the campfire::default recipe)

Attributes

• subdomain - the subdomain for your Campfire instance (required)
• room - the name of the room you would like to speak into (requied)
• token - authentication token for your Campfire account (required)
• message - the message to speak. If a message is not specified, the name of the campfire_msg resource is used.
• paste - toggles whether or not to send the message as a monospaced “paste” (defaults to false)
• play_before - play the specified sound before speaking the message
• play_after - play the specified sound after speaking the message
• failure_ok - toggles whether or not to catch the exception if an error is encountered connecting to Campfire (defaults to true)

A list of emoji and sounds available in Campfire can be found here: http://www.emoji-cheat-sheet.com/

Usage examples

1
2
3
4
5
6
7
8
9

include_recipe 'campfire'

campfire_msg 'bad news' do
  subdomain 'example'
  room 'Important Stuff'
  token '0xdedbeef0xdedbeef0xdedbeef'
  message "I have some bad news... there was an error: #{some_error}"
  play_after 'trombone'
end

Chef’s deploy and deploy_revision resources provide a useful mechanism for deploying applications as part of a chef-client or chef-solo run, without depending on an external system (e.g. Capistrano.) Many Chef users learning to use these resources for the first time will find that they also need to install an SSH deploy key and an SSH wrapper script for Git before they can make effective use of these deploy resources, and that the Chef wiki doesn’t provide much documentation around this issue.

Enter deploy_wrapper: a Chef definition which handles the installation of an SSH deploy key and SSH wrapper script to be used by a deploy or deploy_revision resource.

Before deploy_wrapper, a recipe to configure the required resources to make an automated deploy or deploy_revision possible might look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

directory '/root/.ssh' do
  owner "root"
  group "root"
  mode 0640
end

directory '/opt/myapp/shared' do
  owner "root"
  group "root"
  mode 0755
  recursive true
end

deploy_key = data_bag_item('keys', 'myapp_deploy_key')

template "/root/.ssh/myapp_deploy_key" do
  source "deploy_key.erb"
  owner "root"
  group "root"
  mode 0600
  variables({ :deploy_key => deploy_key })
end

template "/opt/myapp/shared/myapp_deploy_wrapper.sh" do
  source "ssh_wrapper.sh.erb"
  owner "root"
  group "root"
  mode 0755
  variables({
    :deploy_key_path => "/root/.ssh/myapp_deploy_key"
  })
end

deploy_revision "/opt/myapp" do
  repository node['myapp']['repository']
  revision node['myapp']['revision']
  ...
  ssh_wrapper "/opt/myapp/shared/myapp_deploy_wrapper.sh"
end

Not counting the source to template files for these resources, thats almost 30 lines of code just to set the stage for a deployment. It didn’t take long for me to grow tired of reusing this rather verbose pattern across a growing number of recipes.

Here’s how I accomplish the same thing with the deploy_wrapper definition:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

deploy_key = data_bag_item('keys', 'myapp_deploy_key')

deploy_wrapper "myapp" do
  ssh_wrapper_dir "/opt/myapp/shared"
  ssh_key_dir "/root/.ssh"
  ssh_key_data deploy_key
  sloppy true
end

deploy_revision "/opt/myapp" do
  repository node['myapp']['repository']
  revision node['myapp']['revision']
  ...
  ssh_wrapper "/opt/myapp/shared/myapp_deploy_wrapper.sh"
end

Much better, right? Well, a lot shorter anyway. Now let’s talk about what the deploy_wrapper parameters used in the above example are doing.

The ssh_key_dir and ssh_wrapper_dir parameters specify directories which will be created by Chef. In the case of ssh_wrapper_dir, the git SSH wrapper script will automatically be created in this directory following the pattern “APPNAME_deploy_wrapper.sh”, using the value of the name parameter (in this case, myapp) in place of “APPNAME”.

Similarly, an SSH key file containing the data passed to the ssh_key_data parameter will be created in the directory specified as the value for the ssh_key_dir parameter. The key file will be named following the pattern “APPNAME_deploy_key”, using the value of the name parameter (myapp) in place of “APPNAME”.

The sloppy parameter is the only optional one. Because the default configuration of most most ssh installations is to require manual verification when accepting a remote host’s key for the first time, the sloppy parameter allows one to toggle key checking (StrictHostKeyChecking) on or off.

When the value for sloppy is true, the wrapper script will accept any host key without prompting. The default value for sloppy is false, meaning that additional Chef resources, or … *gasp* … manual intervention, will be required in order to set up a known_hosts file before deployments can run successfully.

Monitoring with Pingdom

Swedish firm Pingdom offers a flexible, affordable service for monitoring the availability and response time of web sites, applications and other services. At Needle we provision an instance of our chat server for each partner we work with, and as a result I’ve found myself creating a Pingdom service check to monitor each of these instances. As you might imagine, this is a rather repetitive task, and the configuration is basically the same for each service check – a process ripe for automation!

Thankfully Pingdom provides a REST API for interacting with the service programatically, which has made it possible for me to write a Chef LWRP for creating and modifying Pingdom service checks. Source available here: http://github.com/cwjohnston/chef-pingdom

Requirements

Requires Chef 0.7.10 or higher for Lightweight Resource and Provider support. Chef 0.10+ is recommended as this cookbook has not been tested with earlier versions.

A valid username, password and API key for your Pingdom account is required.

Recipes

This cookbook provides an empty default recipe which installs the required json gem (verison <=1.6.1). Chef already requires this gem, so it’s really just included in the interests of completeness.

Libraries

This cookbook provides the Opscode::Pingdom::Check library module which is required by all the check providers.

Resources and Providers

This cookbook provides a single resource (pingdom_check) and corresponding provider for managing Pingdom service checks.

pingdom_check resources support the actions add and delete, add being the default. Each pingdom_check resource requires the following resource attributes:

• host - indicates the hostname (or IP address) which the service check will target
• api_key - a valid API key for your Pingdom account
• username - your Pingdom username
• password - your Pingdom password

pingdom_check resources may also specifiy values for the optional type and check_params attributes.

The type attribute will accept one of the following service check types. If no value is specified, the check type will default to http.

• http
• tcp
• udp
• ping
• dns
• smtp
• pop3
• imap

The optional check_params attribute is expected to be a hash containing key/value pairs which match the type-specific parameters defined by the Pingdom API. If no attributes are provided for check_params, the default values for type-specific defaults will be used.

Usage

In order to utilize this cookbook, put the following at the top of the recipe where Pingdom resources are used:

1

include_recipe 'pingdom'

The following resource would configure a HTTP service check for the host foo.example.com:

1
2
3
4
5
6

pingdom_check 'foo http check' do
  host 'foo.example.com'A
  api_key node[:pingdom][:api_key]
  username node[:pingdom][:username]
  password node[:pingdom][:password]
end

The resulting HTTP service check would be created using all the Pingdom defaults for HTTP service checks.

The following resource would configure an HTTP service check for the host bar.example.com utilizing some of the parameters specific to the HTTP service check type:

1
2
3
4
5
6
7
8
9
10
11

pingdom_check 'bar.example.com http status check' do
  host 'bar.example.com'
  api_key node[:pingdom][:api_key]
  username node[:pingdom][:username]
  password node[:pingdom][:password]
  check_params :url => "/status",
               :shouldcontain => "Everything is OK!",
               :sendnotificationwhendown => 2,
               :sendtoemail => "true",
               :sendtoiphone => "true"
end

Caveats

At this time I consider the LWRP to be incomplete. The two major gaps are as follows:

• Changing the values for check_params does not actually update the service check’s configuration. I have done most of the initial work to implement this (available in the check-updating branch on github), but there are still bugs.
• The LWRP has no support for managing contacts.

Future

• Add update action for service checks which modifies existing checks to match the values from check_params
• Add enable and disable actions for service checks
• Add support for managing contacts (pingdom_contact resource)
• Convert TrueClass attribute values to "true" strings
• Validate classes passed as check_params values
• One must look up contact IDs manually when setting contactids in check_params

Introduction

Recently I have been experimenting with the logging-as-a-service platform at Loggly. It seems pretty promising, and there’s a free tier for those who are indexing less than 200MB per day.

Since I am using Chef to manage my systems, I decided I would take a crack at writing a LWRP that would allow me to manage devices and inputs on my Loggly account through Chef. This makes it possible for new nodes to register themselves as Loggly devices when they are provisioned, without requiring me to make a trip to the Loggly control panel. The resulting cookbook is available here: http://github.com/cwjohnston/chef-loggly

Requirements

• Valid Loggly account username and password
• json ruby gem

Required node attributes

• node['loggly']['username'] - Your Loggly username.
• node['loggly']['password'] - Your Loggly password.

In the future these attributes should be made optional so that usernames and passwords can be specified as parameters for resource attributes.

Recipes

• default - simply installs the json gem. Chef requires this gem as well, so it should already be available.
• rsyslog - creates a loggly input for receiving syslog messages, registers the node as a device on that input and configures rsyslog to forward syslog messages there.

Resources

loggly_input - manage a log input

Attributes

• domain - The subdomain for your loggly account
• description - An optional descriptor for the input
• type - The kind of input to create. May be one of the following:
  • http
  • syslogudp
  • syslogtcp
  • syslog_tls
  • syslogtcp_strip
  • syslogudp_strip

Actions

• create - create the named input (default)
• delete - delete the named input

Usage

1
2
3
4
5
6

loggly_input "production-syslog" do
    domain "examplecorp"
    type "syslogtcp"
    description "syslog messages from production nodes"
    action :create
end

loggly_device - manage a device which sends logs to an input

The name of a loggly_device resource should be the IP address for the device. Loggly doesn’t do DNS lookups, it just wants the device’s IP.

Resource Attributes

• username - Your Loggly username. if no value is provided for this attribute, the value of node['loggly']['username'] will be used.
• password - Your Loggly password. if no value is provided for this attribute, the value of node['loggly']['password'] will be used.
• domain - The subdomain for your loggly account
• input - the name of the input this device should be added to

Resource Actions

• add - add the device to the named input (default)
• delete - remove the device from the named input

Usage

1
2
3
4
5

loggly_device node[:ipaddress] do
    domain "examplecorp"
    input "production-syslog"
    action :add
end

UPDATE: as of lately (spring 2012) I am not actively using Loggly, so I am not actively improving this code. Please let me know if you are interested in updating it or taking over the project.