As a security company, there’s a lot of pressure to keep our data secure while still moving fast and innovating on product development. I find the intersection of security and speed the most interesting challenge as an infrastructure security professional. The unique thing about Threat Stack is that our Security and Engineering teams have learned how to work together to automate security into our day-to-day processes — making them simultaneously more secure, efficient, and effective.
I’m a firm believer that an effective SecOps organization involves people, processes, and tools, in that order. The tools we’ve built in-house are meant to make people’s lives easier, and ease some of the processes that make security a natural part of the workflow if you’re trying to get a job done quickly.
We’ve open-sourced a lot of the tooling we’ve developed to make our operations more secure, and hope you’ll find this information useful when you’re thinking about automating security in your own organization.
In this post, I’ll describe three of the tools we’ve developed (and then open-sourced) at Threat Stack in order to integrate automated security processes into our workflow. (I’ve also included a description of a fourth tool that we developed — an automated SOC 2 compliance checking bot. We use it internally, but to date, it’s not available outside Threat Stack.)
1. Deputize: On-Call Role-Based Access Automation
One of the major practices we implemented to secure our organization is role-based access control, embracing the principle of least access to make sure that permissions are only granted to those who need them. But, in an on-call rotation, you may need to temporarily grant access to the appropriate hosts for the responsible on-callers for a short amount of time.
At Threat Stack we decided to automate the changing of the guard for on-call duty by writing a tool called Deputize. This tool updates LDAP with the proper people who need to be on call, pushing a message into Slack with who’s on duty as the rotation changes. Using PagerDuty, people on call will get the appropriate information for permissions in a few minutes. Find Deputize on Github.
2. Authkeys: Public Key Authentication
At Threat Stack, we authenticate our public keys with SSH, which is tied to a hardware device that requires multi-factor authentication.We decided to create a centralized store for SSH keys in a good, known database like LDAP, rather than relying on local storage.
To make sure people weren’t arbitrarily making SSH keys without really needing them, we created a tool called Authkeys that queries LDAP for SSH keys when you log in. As a result, our team no longer needs to add SSH keys to every machine (as a part of Chef). Find Authkeys on Github.
3. VPNNotify: Slackbot to Report Unauthorized Logins
Our team tries to make it as easy as possible for employees throughout the organization to recognize and report unusual security events. If an unauthorized attacker gets access to a VPN account, they can move quickly to gain access to systems containing sensitive information.
We created a tool called VPNNotify to automatically send a Slack message to employees when someone logs into their VPN account. From there, the user can easily notify the operations or incident response team if they didn’t initiate the VPN session. VPNNotify is just one simple way to automate a common security concern and generate overall user awareness on how to communicate a security incident. Find VPNNotify on Github.
Bonus: sockembot for SOC 2 Change Management
Spoiler alert. This tool isn’t available outside Threat Stack (yet), but we wanted to include a brief write-up on it anyway to give you insight into how we added visibility and automation to one of our key processes.
Threat Stack is Type 2 SOC 2 compliant, and to maintain compliance, our operations team created sockembot — an automated SOC 2 compliance checking bot — to provide automation, speed, and visibility in the development cycle as outlined by our SOC 2 change management procedure. By ensuring that code is compliant before it gets pushed to Production, the tool produces stronger code quality and better security.
Before any code is released into Production, sockembot displays a helpful message tying together everything we need to know about the code’s compliance status in a digest. If the code is compliant, sockembot outputs a digest that allows the user to inspect the changes that were about to be shipped into Production. If the code is not compliant, sockembot blocks the noncompliant code from being shipped to Production. Read more about sockembot in this post.
Earning the Trust of Engineering
When there are a lot of projects in play, automating security can help earn the trust of your engineering teammates. Sure, you can always require that teams take on an additional security step in their development process, but these workflow hiccups often frustrate people and lead to workarounds that can compromise the best of security intentions. By creating new systems that work easily into an operational workflow, you become the one who makes life easier through security — rather than slowing people down.
To learn more about how Threat Stack does security, listen to our recent webinar Who Watches the Watchman: How a Security Company Does Security, and for great insights into how Threat Stack does DevOps, take a look at this blog post series by Pete Cheslock.