The recent launch of Chef Automate 1.5 brought with it the GA release of a new Compliance view to the Chef Automate UI. This update brings with it some new detailed compliance reporting dashboards, as well as a means to easily install and manage InSpec Profiles without needing to deploy a separate Compliance Server. To take advantage of this new functionality, the Audit Cookbook has seen some updates as well, making it easier than ever to detect and correct issues across your fleet — all in one place.
Hands On:
Throughout this post, I’ll be calling out some resources in our documentation and Learn Chef Rally learning modules for suggestions on where to get more thorough insights on the topics being covered in each section.
What’s New?
Back in November, Victoria Jeffrey put together a fantastic run-down of how the Audit cookbook is put together. Since then, there have been a few small changes that we’ll be making use of today.
First, some attribute names and values have changed:
The collector attribute has changed to reporter. This controls where the audit cookbook will send the results of the profiles we run.
The chef-server-visibility and chef-visibility options for that attribute have changed to chef-server-automate and chef-automate. These reporters will send profile data to your Automate Server, either by authenticating through your Chef Server, or by sending data directly to the Automate Server.
Most importantly for today’s example, the fetcher attribute now has the ability to pull profiles directly from a Chef Automate Server!
Where to Start
Easily the most difficult challenge when starting to enforce and audit organizational standards is figuring out where to start. Thankfully, Chef Automate’s new Compliance View gives us an easy way to pull in some pre-made profiles that can serve as a roadmap for where we might be able to improve our systems’ security. In this post, we’ll see just how quickly we can go from identifying an issue that needs to be solved, to crafting a remediation for that issue, all while tracking our progress in a unified dashboard along the way.
Before we start making changes to our servers, we need to understand their current running state. To kick my environment off, I have a single CentOS 7.3 node bootstrapped and sending data to my Automate Server. Right now, it’s simply checking in, and does not yet have any cookbooks in its run list. In other words, I’m not making any changes to this node at this point, just capturing basic system information.
Now that I have a server to manage, I’ll need to determine how securely it’s configured. To help get me started, I’ll be using one of the profiles included with a Chef Automate installation called the Dev Sec Linux Security Baseline. This profile contains a number of controls to validate some generalized linux security best practices, and can help me get an idea of how my image is performing out of the box.
Profiles can be downloaded from the Compliance View in Chef Automate by navigating to the “Profile Store” option on the left-hand menu, selecting the desired profile(s) from the “Available” tab, and clicking the “get” button.
If I dive into the profile details, I can see a brief description of each control it contains. From there I can further dive into the InSpec code behind that control and determine the methodology used to validate. Whatever level of detail is required, the Profile Store provides an easy place to find everything we need to know about the profiles in our organization.
Hands On:
If you’d like to get familiar with the Automate UI, the newly released Chef Automate Pilot is a great way to get started in a snap!For more on the basics of bootstrapping a new node and more, be sure to check out the Infrastructure Automation Track.
Enter the Audit Cookbook
As I mentioned earlier, recent updates to the Audit Cookbook included the ability to fetch profiles from Chef Automate’s profile store, and I’ll illustrate that today by creating a simple wrapper cookbook to start evaluating the InSpec profile I just installed.
(For more on wrapper cookbooks, check out Jerry Aldrich’s recent blog post)
It all starts with a simple chef generate:
Simply put, a wrapper cookbook is a way to invoke and configure other Chef cookbooks without needing to alter any of their code. My example cookbook contains only a few small modifications from the defaults provided by the chef generate command.
recipes/default.rb
include_recipe 'audit::default'
attributes/default.rb
default['audit']['fetcher'] = 'chef-server-automate'
default['audit']['reporter'] = 'chef-server-automate'
default['audit']['profiles'] = [
{
name: 'Linux Baseline',
compliance: 'admin/linux-baseline',
},
]
metadata.rb
depends 'audit'
Most Chef Automate installations are configured with data collection provided by a Chef Server, in which case we’d use the chef-server-automate fetcher and reporter as seen above. In some cases — for example, when using chef-solo — there may be a need to send node data directly to an Automate server, in which case the chef-automate fetcher and reporter would be used instead.
The ['audit']['profiles'] attribute takes an array of profiles that we’d like our node to execute, each defined as a hash containing at minimum a name and source. In this case, to pull from my Automate Server, I’m using the compliance source, which will pull via whichever fetcher I’ve specified. The Compliance View in Chef Automate also provides a convenient helper button to copy the proper profile name for use in this attribute.
Now all I need to do is add that cookbook’s default recipe to my node’s run list, and I’ll be able to get my first peek on how much work I have ahead of me.
Hands On:
The Writing Cookbooks track is a great place to get comfortable with the process of creating new cookbooks and running them on your nodes.Also, be sure to check out the Audit Cookbook’s README for a detailed overview of the configuration options available.
Step 1: Detect
After I update my node’s run list and it checks in, I can see in Chef Automate’s Nodes View that chef client completed successfully, that my new cookbook was applied, and the audit attributes are set as expected.
However, if I hop over to the Compliance View, I can see that my node did not pass the Linux Baseline audit. Clicking on “Profile Status” shows me that of 51 total controls evaluated, 26 of them came back with the “failed” status. Looks like I have some problems to solve!
To determine the specific nature of those problems, the “Nodes” and “Profiles” tabs give me a few options for looking through my scan results. I can see details for each failed control, as well as what specifically failed within that control. For example, let’s have a look at the first failure:os-02: Check owner and permissions for /etc/shadow
The profile reports that the file /etc/shadow is not readable or writeable by its owner, which it expects to be true. The source code provides a more detailed accounting of what permissions in particular it’s expecting to find:
describe file('/etc/shadow') do
it { should exist }
it { should be_file }
it { should be_owned_by 'root' }
its('group') { should eq shadow_group }
it { should_not be_executable }
it { should be_writable.by('owner') }
it { should be_readable.by('owner') }
it { should_not be_readable.by('other') }
end
Now that I’ve identified a problem, and familiarized myself with the expectations defined in my InSpec profile, I have everything I need to implement a lasting solution.
Hands On:
For more on writing InSpec controls, check out the Compliance Automation track.
Step 2: Correct
If InSpec is the language of detecting issues in our environments, chef is the language of correcting them. Chef’s DSL consists of a variety of easy to read resources that can be used to define how our nodes should be configured. To keep things simple, in this case we’ll just concern ourselves with one: the File resource.
The file resource allows us to create and optionally define content for files on our systems. It also allows us to define ownership modes, which is what I’ll need to do to resolve the issue my audit uncovered. To that end, I’ve created a new recipe in my example cookbook:
recipes/shadow.rb
file '/etc/shadow' do owner 'root' mode '600' action :create end
This will ensure that root is the owner of my file, and that only the defined owner can read or write to that file. Finally, I’ll need to update my default recipe as well, to make sure this new resource is included in my chef client run:
recipes/default.rb
include_recipe 'audit::default' include_recipe 'szechuan::shadow'
Now I’m ready to re-run, and see whether or not I’ve solved my issue.
First things first though; let’s make sure the new resource was applied in the Nodes View.
Looks good! I see that the new resource ran successfully. Now for the moment of truth — did it correct my failing control?
Success! 26 failed controls have become 25, and I can see that my os-02 check is now in the “Passed Controls” category! I still have plenty of work ahead of me, but now I have a way to focus my future efforts, and as I address each problem, I can always come back to Chef Automate to ensure my changes are having the effect I’m intending on the nodes I’m responsible for.
Hands On:
Of course, this is just a very simple example, and not meant to serve as production-quality code. For a great OS hardening resource built in tandem with the Linux Baseline profile, check out the Dev Sec Chef OS Hardening cookbook.
Wrapping Up
That’s all for now! I hope this exercise has given you some ideas on how the Audit Cookbook in concert with the new Compliance View in Chef Automate can help you detect and correct problems in your environments. As ever, if you have any questions, or stories you’d like to share, join us in Chef Community Slack, where Chef engineers and community members are always happy to talk shop. Happy automating!
Links:
Chef Automate 1.5 Announcement
Audit Cookbook on Chef Supermarket
Blog: The Audit Cookbook: A How-To
Learn Chef Rally