The Missing AWS OpsWorks CLI

We at Plivo need to constantly deploy updates to our infrastructure that consists of large number of servers spread across multiple regions and continents worldwide. Unfortunately, the complexity of this infrastructure requires very specific and scalable deployment tools that were not already available.

The OpsWorks Command Line Interface (CLI) is a deployment tool for managing AWS OpsWorks. We built the OpsWorks CLI because the standard AWS CLI UI does not have any way to perform simultaneous operations across multiple stacks, a logical group of servers that you want to manage together. We have been using our own OpsWorks CLI extensively for the last year and we’re happy to be opening it up to everyone. We hope that DevOps teams already using AWS and especially those that are also using AWS OpsWorks can use our open source CLI to easily manage multiple server instances at once.

What is AWS OpsWorks?

If you’re familiar with configuration management, then you likely already know Chef, the automation platform. OpsWorks is a free configuration management service provided by AWS that uses Chef, but exposes a friendly user interface on top of it. OpsWorks uses Chef to automate how servers are configured, deployed, and managed across your Amazon Elastic Compute Cloud (Amazon EC2) instances or on-premises compute environments. OpsWorks splits your architecture into stacks. For example, for a simple wordpress setup you could have a production and a staging stack, each with a database and a web layer.

Why OpsWorks?

There are many configuration management solutions, such as Chef, Puppet, Salt, Ansible, and the list goes on… But none of those actually come with a user-friendly UI in their community editions. Most of those also require setting up a central server to take care of delivering the configuration to your servers, which is another point of failure that you need to make highly available too.

OpsWorks is free, runs on both EC2 and on premise, and brings you a powerful UI to inspect and maintain your stacks. You simply point OpsWorks at a Git repository or S3 Bucket with your cookbooks (Chef jargon for the files that actually define how your servers are provisioned) and define which files and servers should run. And because this is built on top of Chef, you can use some of the +3,000 existing cookbooks.

Finally, it has an easy to use auto-scaling feature based on CPU, Memory and Disk Usage, and can automatically start new instances to accommodate for higher loads.

What is OpsWorks CLI?

OpsWorks comes with a neat user interface that allows you to manage a stack: start and stop instances, perform configuration updates, deploy code, check the status of a load-balancer, etc. But the UI does not have any way to perform those operations across multiple stacks at once. What if you wanted to run a specific configuration update on all your production stacks? Or update all your database layers across all stacks?

This is where our CLI comes in: with a simple command line, it allows you to do everything you would be able to do on the UI. In addition, you can run your deployments across your whole infrastructure with a smart and intuitive filtering system. Our OpsWorks CLI also comes with an interactive prompt so that you do not have to access the AWS Console for trivial tasks.

Features and Benefits

OpsWorks CLI is built with scalability in mind. For example, with the OpsWorks CLI, we can easily run a simultaneous deployment on 20 stacks, while constantly pulling the AWS API to retrieve the deployments’ statuses in real time.

Simple Commands

Relying on the AWS API, we built a set of simple commands to help you easily run commands across stacks and regions. These commands can be used in conjunction with smart filters to quickly identify, setup, update, configure and deploy instances. You can also achieve more complex functionality including looking at past deployments of each stack and monitoring elastic load balancers, check for the health of instances, and much more.

Smart Filtering

Any Opsworks CLI command accepts three basic filters: layer, stack, and region. These smart filters also support wildcards and regexes to enable complex filtering logic. For example the command below would match all stacks whose name contain wordpress, and only include their database layer:

opsworks instances -f 'stack:*wordpress*,layer:database'

Another example is using regexes to check ELBs of two wordpress stacks at once:

opsworks elbs -f 'stack:(prod|staging)-wordpress'

Interactive GUI and Interactive Prompt

The OpsWorks CLI also offers an interactive prompt if called without any arguments.

How to use OpsWorks CLI?

The OpsWorks CLI accesses the AWS API using your AWS credentials. Once the credentials are configured, you can run opsworks with no arguments to access the interactive prompt. You can use this interactive prompt to accomplish simple tasks including listing stacks, listing instances, inspecting ELBs, updating cookbooks, deploying code, as well as running configurations and setups.

Visit the OpsWorks CLI Github repository for instructions on how to install the OpsWorks CLI: https://github.com/plivo/opsworks.

What’s next for OpsWorks CLI?

We use our OpsWorks CLI extensively at Plivo, so we’re always looking to build out new features and improvements. One feature on our roadmap is to build a bot and connect it to a real-time collaboration platform like Slack and Hipchat. With a few commands in a chat room, engineering teams can monitor and deploy instances without leaving the conversation. This way, the whole team can be instantly updated with all the changes and deployments across the organization.

Another feature we are working on is rolling deployments. Currently the CLI applies a given command to all the instances of a layer at once (just like what the AWS console). The idea would be to do the deployment instance by instance, taking them out of their respective load balancers first, and then putting them back online.

How do I contribute?

Please visit the OpsWorks CLI github page to report issues and help us make this tool even better.