I’m a big fan of Chris Gates’ publications on DevOops and From Low to Pwned. The content reflects a lot of issues that we also experience in many assessments in general and assessments in agile environments in particular. In addition, we were supporting several projects recently that were organized in an agile way. In this post, I want to summarize some thoughts on how security work can/should be integrated into agile projects. The post was also a result from the preparation of our upcoming Troopers workshop on Docker Security & Devops, which of course also covers organizational aspects, but not to the degree this post describes them.
The definition of agile development results in certain relevant aspects for security work, in particular the focus on
- individuals and interactions over processes and tools,
- working software over comprehensive documentation, and
- the general approach of adaptive planning.
While the specific implementation of the agile principles depends on the used agile framework/methodology/model, the aspects listed above almost always have certain effect on security work and can be discussed in a way independent from the specific agile model. One important differentiation however must be made between
- effects on security work that result from the agile approach and
- effects that result from suboptimal implementation of agile methods.
Issues like confusing stand-ups with reporting sessions, the mix-up of different roles or not sticking to roles, and not being able to establish the blame-less feedback culture impact the effectiveness of the agile approach significantly. Unfortunately those issues are common to many projects or environments which just recently started to adopt agile methods or even still struggle to follow some high level management decision to “go agile”. The resulting impediments (see what I did there? 😉 ) can of course impact the results and perception of agile projects in a negative way, however, they must also be clearly distinguished from issues of the agile approach itself.
Having ruled out implementation issues, we can now discuss the inherent effects of agile methods on security work.
If we have to perform a security assessment for/in an agile project, we could basically approach it like any assessment task and just demand a finalized target of evaluation (TOE) to perform the assessment against. While this may even be feasible for assessments (after all, agile projects work with some type of “releasese” as well), it is difficult to achieve for ongoing consulting work/security support of the project or software (environments) which already transitioned into a continuous deployment mode. In addition, successful ongoing integration of security work may even make final approvals aided by technical security assessments less relevant. The following paragraphs describe potential approaches to integrate security work into agile projects and the effects on traditional security work (they do not necessarily present one single, coherent approach but rather different pieces that should be considered when coming up when defining your own agile security approach).
Commitment
Agile projects require commitment from the project members. While this type of commitment has a special context, it still contributes to the building of a strong team, especially in combination with other agile principles. This in turn means that “security people” should commit to the project; one way of doing so is to integrate into the team as permanent project members that do not only show up for random security reviews where you have “assessment requirements” that break the agile workflow. The following paragraphs will come back to the idea of having one or more permanent security project members and what their tasks should comprise.
Adaptive Planning
One of the core agile principles is the flexible adaption of the project to new insights, customer requirements, or changes in general, no matter where they originate from, which then results in an ever-evolving (from the security perspective: ever-changing 😉 ) environment.
This continuous evolvement should be supported by the project security members by providing ongoing consulting and review for changes from a security perspective.
At the same time, you may want to differentiate between the project security members and a central security group issuing approvals: the project security members provide security knowledge within the project, however, an independent (re)view of the environment is still crucial to avoid “organizational blindness”. Thus we would recommend the project security members being the counterparts of the central security group within the project. This may also result in the need to re-organize “your security department” into distinct groups of “project security people” and “central security (governance) guys”.
Understand Agile Projects
If you have made the step to have established dedicated project security members, you most likely also understood agile projects to a certain extent (else one would not have seen the need to do so…). Obviously, for dedicated project security members it is crucial to understand agile projects, however, it is even more crucial to do so when you cannot perform this transition (e.g. due to having a small security department).
Once there are dedicated security project members, the following points are imperative (even if you do’ne implement them, strongly consider them):
- Learn the agile vocabulary
- Use the same tools (e.g. if you have security findings, have the product owner integrate them in user stories, use the same issue tracker)
- Contribute evil user stories
- Participate in stand-ups
- Have interactions with individuals rather than providing 80 pages of abstract security requirements
- Integrate “security alignment” into DoD/acceptance criteria
Technology Evaluation
As a central security department, you should provide the evaluation of technology as a service. We constantly have plenty of findings in agile environments that have serious security impact (unauthenticated docker sockets exposed to the world only being one of them) which often result from new technologies that (can) provide great functional benefits but are not thoroughly analyzed/understood from a security perspective yet.
Provide the evaluation of those technologies as a service and then provide configuration blueprints (and commit them here 😉 ) and training (instead of huge word documents listing settings and the abstract need to encrypt every communication).
Provide Training
Follow the interaction between individuals over processes and tools-mantra and provide security trainings instead of 80 pages of abstract security requirements. This is always a relevant idea to consider but especially relevant in agile projects.
Documentation
Here’s a difficult one. The agile principle working software over comprehensive documentation results in a relevant challenge for security work. I do not think that comprehensive documentation following established standards into every detail is necessary, however, we have seen too many projects where documentation was almost completely neglecte. This is not only difficult for security work, but constantly working under assumptions about the overall design/infrastructure can also have dangerous impact on the overall development (and agile projects are not only about software anymore since the DevOps age, right?). Being integrated into a team as a security member can give you relevant infrastructure insights, however, I would still recommend to insist on keeping a high-level infrastructure overview up to date. The entities included in this overview depend on your level of abstraction. For example, if you already transitioned to an abstract docker swarm, there may not be the need to document individual systems but only “logical entities”. However, I would still encourage to document those logical entities and their communication relationships (and paths, potentially) with other entities. In the ideal case, you also have a clear mapping to subnets/prefixes. Many projects that we asked for such an overview identified a lot of open security issues by themselves just by seeing how different the view on the architecture was or how many aspects were not clear at all. I also encourage you to identify suitable ways of maintaining such an infrastructure overview — there is nothing more painful than constantly re-arranging visio charts that are manually versioned on some network shares.
This architecture overview should also be the basis to maintain a constantly updated threat model in case you have project security members.
Continuous Micro-Assessments
I mentioned that the concept of “approval pentests before release” may become less relevant. When agile environments transform into a Continuous Deployment (CD) environment, comprehensive approving pentests of a certain version/state of the application become close to impossible and hinder the effective implementation of a principle that can result in promising benefits. For example, have you ever had the discussion with application owners why they needed at least three months to roll-out a critical security fix? In a CD environment, there is no such thing like an out-of-band patch.
The direct conclusion could be that we do not perform security assessments against fixed releases but also in a continuous way. This can be integrated by having a chaos pentester who is part of the development team. After all, when performing security assessments, non-familiarity with the target of evaluation often improves results and results in insights that cannot be gained from someone closely working on the TOE.
Continuous Micro Assessments should be performed on a regular basis within the target environment (there are a lot of sloppy configurations which are “just in place during development” but can result in serious security impact) which can be easily detected by a human pentester and should be corrected in an early phase ([privileged] account management [of functional user accounts] is a classical example which is often neglected until it is too late).
In addition, the project security member should have a good overview about when security-relevant code was changed and an assessment (of specific components) should be carried out. However, as described above, a continuous assessment of the whole target environment is still necessary even when spot assessments of changed components are performed. Often multiple non-critical findings can be combined into a single more comprehensive issue (this presentation is a good example for that).
Automated Security Checks
One of the most-covered aspects of agile security is the integration of automated security checks into the continuous integration pipeline. I absolutely agree on the relevance of such efforts, still, even given that I may be biased here, I want to emphasize the need for human participation in the development process as described above. It will be a while until we have an automated check for “I forgot authentication for this service” 😉 Still, it is worth to include evaluated potential for automated security check into your DoD/acceptance criteria.
Some of the described concepts may sound a bit abstract if you haven’t worked in agile projects yet (not that I would consider myself an expert on agile development), but I’m looking forward to potentially dive deeper into some of the described ideas — maybe after a discussion with you at our Troopers Roundtable on the topic? 😉
Cheers,
Matthias