Agile vs Traditional: Managing Chaos or Maintaining Order?
As technology, business needs, and consumer demands are constantly evolving, firms must adapt to changes in order to develop competitive advantages to stay ahead of the competition. While the meaning of digital revenues and processes is open to interpretation, it’s clear that digital business is a reality now and expected to be a very significant aspect of achieving competitive advantage and differentiation using information and technology. Software systems are fundamental in the management of data, which in turn determines the quality of information available to support decision making in strategies for growth and innovation.
The role of technology within large organisations sits between the need for innovation and the nature of the business model adopted by the firm to mediate the link between technology and firm performance, and define the business model as a system that: solves the problem of identifying who are the customers, engaging with their needs, delivering satisfaction, and monetising the value.
Project managers are “change agents” focused on project goals who create the environment for others to deliver unique business results. Central to the role of the project manager is the ability to get the best performance from the project team, and this is reliant on the project manager’s ability to effectively communicate with, motivate and inspire all stakeholders. Increasingly, projects are no longer contained within the organisation, but are collaborative, open, and must include a diversity of stakeholders. Gartner identified that the increased acceptance of digital into mainstream business means that lines are becoming increasingly blurred, and boundaries ‘semiporous’ — both inside and outside the enterprise — as multiple networks of stakeholders bring value to each other by exploiting and exploring platform dynamics.
Defining project success
Delivery of concrete and usable results on time, scope and budget defines a successfully completed project. According to Standish Group, large projects rarely complete successfully within the triple constraint of time, cost, and performance — in contrast to small projects, which have a greater than 70% chance of performing well. They argue that large IT projects are “unnecessary” and to increase the chance of success, large projects should be divided into smaller projects that can be performed in parallel if necessary.
A project that has a clear definition of success and is monitored consistently and able to adapt to changes as they arise will have better chance of success. Many projects are initiated without a clear definition or vision of what would be considered a successful outcome. Defining success among key stakeholders before the start of a project and at several review points during the project’s lifecycle, and creating a vision of the project outcome is in itself a significant driver of project management success. Communicating a “shared project vision” is a significant factor to project success, and needs to be supported by all project stakeholders, sponsors, and management throughout the lifecycle of the project. To be successful, a project vision must be understood, motivational, credible, demanding and challenging.
Researchers at Google conducted a study of its employees, interviewing 180+ active Google teams over two years, to try to pinpoint the characteristics and skills that are found in high performing teams. The researchers were surprised to discover that: “Who is on a team matters less than how the team members interact, structure their work, and view their contributions.” Five key dynamics were found to influence the success of teams and set them apart from others: Psychological safety; dependability; structure and clarity; meaning of work; impact of work. Of these five dynamics, Google analysts found that psychological safety is the most important and is essential to the proper functioning of the other four.
Psychological safety is a belief that one will not be punished or humiliated for speaking up with ideas, questions, concerns, or mistakes. In a high performing project environment, this involves framing the work as a learning problem instead of an execution problem, acknowledging ones own fallibility, and sharing ideas and asking questions. In her TedX Talk, Rozovsky infers that individuals on teams with higher psychological safety are less likely to leave, collaborate with teammates, and generate more revenue through their projects.
Managing for change
Traditional approaches to software development follow a plan-driven product-line approach using a systemised process where the project is directed, from planning to completion, through a predetermined course of predictable controlled steps. Similar to conventional project management, waterfall, follows on a sequential path, through: requirements; design; development; testing; deployment. This technique is most suitable when project requirements and outcomes are known in advance, allowing all aspects of the project to be planned before work is commenced. A project following a waterfall process tries to eliminate all uncertainty about what is being built before tackling the uncertainty of how it will be built — the end product is fully defined.
Agile development follows an iterative approach to implementing projects, where uncertainty is central to the planning and implementation process — in direct contrast to traditional software development methodologies. The key differences between agile and traditional (or waterfall) are outlined in the “Agile Manifesto” key principles, which state:
“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.”
Agile software development grew out of the need to improve the end product, shorten the development life cycle, reduce coding errors, and to facilitate scope changes as the project develops. Scrum, as a variant of agile project management, also developed from the need to adapt from a linear approach to an integrated trial and error approach necessary to develop new products quickly and flexibly. Scrum is now the most used agile methodology, while lean-kanban is probably the fastest growing agile methodology.
A combination of waterfall and agile methodologies is becoming more prevalent, possibly due to desire to implement the fluidity of agile in organisations with traditional project management processes that are ingrained in the organisational culture. This increased tendency towards ‘hybrid’ models may indicate a convergence in project management, where market forces increasingly dictate the organisation’s product development strategy.
Gartner identify “digitisation” as a core competency of modern businesses, and organisations that cannot operate effectively in the digital domain will struggle to: deliver services; attract and retain talent; and have their products/services perceived as value-adding by customers.
On the other hand, a disorganised mix of traditional and agile methodologies may indicate aversion to change, as is seen when the organisational culture is resistant to change and reverts to old familiar practices. The most obvious difference between waterfall and agile project management is found in the company’s attitude to change. Agile thrives on the unknown, and seeks to reduce barriers to change by eliminating the bureaucracy and ‘red-tape’ that slow the pace of the project. Teams self-regulate, are responsible for their own budgets and timelines, and monitor or adapt the project scope as new information is discovered.
Waterfall, on the other hand, invests heavily in planning and documentation in order to prevent change, and make it difficult to alter the requirements once the project is underway. It is essential for waterfall project management to include wording into the contract to discourage change, where: the client introduces no unnecessary changes; all changes must be authorised; changes are subject to the triple constraint; any change is documented in detailed reporting; changes must be compensated for in extra costs and increased schedule.
Features of Agile
Glaiel, Moulton, and Madnic (2013) presented the “Agile Genome” — a framework for understanding Agile development practices — which identifies seven representative characteristics of agile software development projects. (See Table 1)
Examining several variations of agile project management methodologies in use in software development they found that they shared common characteristics.[14] In particular, they note that the five most popular methodologies [Scrum; eXtreme Programming; Test Driven Design; Feature Driven Design; Crystal] all have Feature Driven and Iterative-Incremental characteristics in common. (See Table 2)
After completing a case study of software projects at Intel Shannon over a three year period, Fitzgerald concluded that contemporary agile methods such as XP and Scrum are not anti-method, and require an equally disciplined approach, and as much tailoring as any traditional method.[12] Similarly Špundak asserts that it is possible, and in some cases even preferable, to combine traditional and agile methods to fit the project characteristics and characteristics of the organisation.[32]
However, the simplicity and low overheads that make using Agile methodologies so appealing also make it highly susceptible to misapplication.[8] Any tailoring of methodologies must be deliberate and continuously assessed to ensure that it is focused on project goals and not a result of poor implementation. Schwaber cautions that:
“Scrum’s simplicity itself—its lack of prescription—can be disarming, and new practitioners often find themselves reverting to old project management habits and tools and yielding lesser results.”
When a new mythology such as agile is introduced into a company mainly as a cost reduction measure, its success or failure will be judged on the amount of money saved over a period of time. Although there are a myriad of legitimate reasons to introduce new methods for managing projects to achieve organisational goals, the justification should not be for financial considerations alone. There needs to exist a ‘shared vision’ within the organisation’s senior management, between client, project sponsor and project champion, as well as the employees working directly within the project.
Agile is a commitment to cultural change as well as a methodology to increase efficiencies and reduce costs. Senior management evaluating the effectiveness of project teams by financial reports alone may underestimate other positive performance impacts within the company. Agile project management is conducive to innovation, and benefits the long-term strategy of the company, and requires active support from top management to build innovation into the organisational culture. But embedding agile project management “within a larger setting of a traditional management of hierarchical bureaucracy leads to inevitable tensions. Usually the prevailing culture of hierarchical bureaucracy is triumphant”.[10]
Success of iterative project management depends on trust and support from the highest levels, and may involve educating senior management of the value and necessity of this approach. The transition towards integrating self-managing teams in an organisation structure takes time and resources, and is often difficult to implement in practice. Until key management become enthusiastic sponsors and accept the ‘change mindset’ with conviction, agile or scrum project managers will struggle to exert the authority needed to ensure success of their projects.
Agile, and in particular scrum, must allow teams the freedom to operate autonomously within the larger company framework to allow the system to function efficiently and effectively. Agile projects are self-contained, highly skilled teams in close communication, and do not put accent on an extensive documentation, which means project knowledge is mainly tacit. Agile does not rely on the processes or reporting associated with traditional project management, and as a result progress may be unclear to those outside the project. Senior management, without the reassurance of familiar measurement tools to assess progress, risk unsettling the project by making decision based on incorrect information or assumptions. Because of the self-contained nature of agile teams, other management in the organisation tend to underestimate the resource requirements of agile teams, and either try to overload them with work or redeploy members mid-project. As a result, unrealistic or over-optimistic targets may be set, and without historic performance data to reference against, performance may seem poor, and consequently, iterative processes might not be allowed the opportunity to develop within the company. Management outside the project team should try to be more aware of the day-to-day activities of the project team by participating in regular team meetings. The net effect would be a more nimble, productive project orientated environment that thrives on change — essential for an organisation trying to embed innovation at the core of its strategy. Adopting Agile techniques can have a positive influence on the organisation as a whole. It can stimulate new ways of thinking and act as a change agent, where the energy and motivation the effort produces can spread throughout the company and begin to break down some of the rigidities that have set in over time.