The in-flight software that ran during the US National Aeronautics and Space Administration’s (NASA’s) Apollo 11 mission in 1969, of which the purpose was to safely take humans to the moon and back to Earth, contained approximately 145,000 lines of code and was developed by the software engineering division at the Massachusetts Institute of Technology’s (MIT’s) (Cambridge, Massachusetts, USA) Draper Laboratory in the 1960’s.1 Fast forward to 2022 and the software that powers Google’s services is built on 2 billion lines of code supported by 25,000 Google engineers around the world.2 Google’s services are used by billions of people daily and are rapidly evolving to meet the changing needs of its extremely diverse customer base.
This comparison demonstrates how software has changed from targeted code used to solve specific problems to platforms that evolve, adapt and continuously improve.
It is only natural that the software development process has also changed. Initially, programmers did not have a systematic approach to development activities, which meant that they were the only people able to debug or maintain the programs they had written. In the 1970s and 1980s, software engineering methods such as structured programming, system development methodology and structed system analysis and design were created to develop repeatable processes, making it easier for programmers to collaborate and continue the work of previous programmers more effectively. Software development methodologies of this era were collectively known as waterfall models. These software development models were not flexible because they followed a linear path from start to finish. In the 1990s and 2000s, software engineers created more flexible models for software development such as Agile and DevOps. These models are designed to enable rapid development and maintenance of software according to changing requirements of end users.
If adopted properly, Agile and DevOps practices can result in improvements in objective software development metrics such as deployment frequency, lead time for changes, change failure rate and mean time to recover (MTTR). IT auditors and governance professionals can play important roles in ensuring that such leading practices are adopted in the right manner to fully realize the potential benefits. COBIT® can be leveraged to easily understand Agile and DevOps practices and determine which controls should be implemented.
If adopted properly, Agile and DevOps practices can result in improvements in objective software development metrics such as deployment frequency, lead time for changes, change failure rate and MTTR.
Understanding Agile
A group of 14 programmers published the Manifesto for Agile Software Development in 2001.3 The manifesto states:
We are uncovering better ways of developing software by doing it and helping others do it. This way of working values:
- Individuals and interactions more than processes and tools
- Working software more than comprehensive documentation
- Customer collaboration more than contract negotiation
- Responding to change more than following a plan
That is, while there is value in the latter items, the former items are often valued more.4
The Agile manifesto has gained recognition and acceptance around the world. A recent study indicates growth in Agile adoption within software development teams, increasing from 37% in 2020 to 86% in 2021.5 According to the study, respondents believe that adoption of Agile principles has led to many benefits including accelerated software delivery and enhancement in fulfilling customer requirements. However, despite the benefits, the study also indicates that many organizations continue to face challenges in successfully adopting Agile principles, such as inconsistencies in processes and practices, cultural clashes, general organizational resistance to change, lack of skills and experience, absence of leadership participation, and inadequate management support and sponsorship.
DevOps and Agile
DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption and evolution of Agile practices in the context of a system-oriented design approach. It involves changing, establishing, and nurturing a culture and environment in which planning, developing, coding, building, testing, releasing, deploying, operating, and monitoring are designed to be performed rapidly, frequently, and more reliably (figure 1).
Figure 1—The DevOps Life Cycle
Source: Devopedia, "DevOps," Version 8, 15 February 2022, http://devopedia.org/devops. Reprinted with permission.
DevOps has emerged as a popular approach for software development and maintenance among organizations according to a 2021 study.6 Results indicate that 83% of IT decision makers report their organizations are implementing DevOps practices, yet the results also indicate that the vast majority of organizations have been unable to attain a high level of maturity in their DevOps practices and are, therefore, unable to reap the full benefits of DevOps.
COBIT and DevOps
Over the years, best-practice frameworks such as COBIT have been developed and promoted to assist in the process of understanding, designing and implementing enterprise governance of IT (EGIT).
For IT audit professionals, COBIT resources such as the COBIT® Focus Area: DevOps Using COBIT® 2019 publication can be used to plan and execute IT audits, including the audit of teams that have adopted DevOps. The key aspects relevant to DevOps include:
- DevOps stakeholder interests—It is important to know who are the potential stakeholders at the enterprise, how DevOps benefits each of them and what considerations for adoption should be kept in mind. IT auditors may believe that only programmers and system administrators are stakeholders; however, other possible stakeholders include the architecture board, security officer, project manager and service manager. By knowing what interest each of these stakeholders can potentially have in DevOps, the IT auditor can execute an audit that caters to their concerns.
- Key aspects of DevOps—The Culture, Automation, Lean, Measurement and Sharing (CALMS) framework7 is used to assess whether an organization is ready to adopt DevOps processes and how an organization is progressing in its DevOps transformation. This can also be mapped to COBIT concepts, which is helpful for IT auditors who are likely more familiar with COBIT than CALMS.
- DevOps continuous activities—Many IT auditors are familiar with auditing traditional software development life cycles, but it is necessary for them to also understand the additional activities that are continuously performed in a DevOps model and make audit plans to address such activities accordingly.
- Governance and management objectives—Planning and executing an audit using COBIT governance and management practices can be very effective. There are several COBIT governance and management practices that are relevant to DevOps including:
- APO01 Managed I&T Management framework
- APO1 Managed Quality
- BAI02 Managed Requirements Definition
- BAI03 Managed Solutions Identification and Build
- BAI07 Managed IT Change Acceptance and Transitioning
- BAI08 Managed Knowledge
- BAI10 Managed Configuration
- BAI07 Managed IT Change Acceptance and Transitioning
- MEA01 Managed Performance and Conformance Monitoring
- Organizational structures—The assignment of DevOps roles and responsibilities varies greatly from one enterprise to another. COBIT resources can provide guidance on what roles and responsibilities stakeholders in the DevOps process should be performing. At many organizations, different players in the process adopt roles and responsibilities that are not best suited from a controls point of view. IT auditors should be able to effectively communicate to management what the leading practices are and how to ensure that the right people perform the appropriate roles.
- Principles, policies and procedures—There are specific principles, policies and procedures (e.g., business agility principle, change management policy, test acceptance procedure) that should be updated or created to support the implementation of DevOps processes. IT auditors should review existing policies, procedures and standards to see if relevant updates have been made to align with DevOps practices.
- Tool types—It is helpful for IT auditors to know what types of tools support DevOps processes so they are prepared before performing IT audit field work. Also, many organizations do not need to use all DevOps tools available. Knowledge of all the different tools that support DevOps processes enables an IT auditor to provide recommendations to improve DevOps processes through further automation when necessary.
Conclusion
IT auditors across industry sectors and regions can acknowledge that the organizations they serve are at various stages in the process of implementing DevOps practices. The IT audit community should consider using COBIT to equip itself with the knowledge and skills to audit DevOps processes effectively and to ensure that DevOps is able to deliver its true potential in a secure and effective manner.
Endnotes
1 Woos, D.; “Introduction,” Computer Science at Brown University Lecture, Providence, Rhode Island, USA, 2020
2 Metz, C.; “Google Is 2 Billion Lines of Code—And It's All in One Place,” Wired, 16 September 2015
3 Agile, Manifesto for Agile Software Development, 2001
4 Ibid.
5 Digital.ai, 15th State of Agile Report, USA
6 Puppet, 2021 State of DevOps Report
7 Buchanana, I.; “CALMS Framework,” Atlassian
Syed Salman, CISA
Is passionate about enabling enterprises to see the upside of technology risk and find ways to sustainably optimize technology performance by implementing leading practices. Salman has helped organizations around the world optimize benefits from their technology governance, technology management, cybersecurity and privacy practices.