...Or is conventional planning failing business?
There's no doubt about it: Agile works. There are many stories of companies that have benefitted from going agile. However, agile is failing to meet expectations in larger organizations; in many cases, it is failing because established strategic planning conflicts with the notion of a lean enterprise and agile planning (with very short time horizons).
In my experience, capital allocation decisions are typically made six to twelve months in advance. Investors and management teams want to carry out strategic planning for an entire year. Financial targets are yearly targets. Planning for a year in the future is not a time horizon that fits the scheme of agile.
Agile emphasizes responding to change over planning. Rapid response is a core tenet of the agile method. It, however, does not mean that agile doesn't value planning — it just values responding to change more. Therefore, a lot of agile development focuses less on planning and more on individuals and interactions, on getting working software delivered and on collaboration between stakeholders.
Agile originated from a need to understand how development teams are formed in order to execute software projects. Agile methodologies we know today, such as SCRUM and Kanban, were developed as a response to many software teams failing to deliver on time and budget.
Ask an agile practitioner to hold a yearly planning exercise and be prepared to receive a weird look in repsonse. Agile has become a multibillion-dollar industry with more experts and certifications than you can imagine. Examples of startups and individual teams benefitting from agile are abundant. However, there are far fewer examples of agile working well for larger organizations, especially when it clashes with strategic, long-term planning.
So what aspect of strategic planning conflicts with agile? A strategic plan is typically a collection of goals and expectations from the business that spans a time period, such as a year — or even five. Most strategic plans include delivery timelines for new and upgraded products that correspond with financial goals, such as revenue and profitability. Providing timelines to planners six to twelve months in advance is a wasteful exercise due to the uncertainty of software effort estimation. To extend agile to large enterprises, new frameworks have been promoted to fill the gap. The scaled agile framework is one example that large organizations can adopt to become a lean, agile-enabled enterprise.
Scaled agile (SAFe), along with a focus on concepts such as continuous long-term product road mapping are ways that companies have tried to make agile function within long-term strategic planning. SAFe contains the notion of strategic themes leading to the creation of a portfolio of projects planned and managed together through agile release trains (continuous delivery or new software).
So is the problem really that agile doesn't work well for long-term planning — or does the issue lie with the practice of long-term strategic planning?
Strategic planning needs to evolve. If you're putting together a plan for the following year, you have to remember that what is relevant in September may not be appropriate when the new year begins. Planning should be more dynamic. It should not be a point-in-time exercise. One way to achieve this is through optionality. If option A is not successful, then ensure that option B is executed, or maybe even option C. Can all of these options achieve the revenue numbers expected? If not, then how should the options be grouped to achieve the desired results? Frameworks must be put in place to evaluate the options and pick each one around the time that the work begins. Delivery teams should have an open mind when evaluating options and providing sizing estimates to the planning process.
Maybe I am a pessimist, but I don't envision investors, board members, CEOs and CFOs giving the go-ahead to start planning and reporting revenue every two weeks any time soon. Budgeting typically follows strategic planning and will either start from a revenue target with a path to success, or with costs that then point to revenue and profitability numbers. I don't think the commitment to financial goals built on annual (or semi-annual) software and operational delivery targets will ever go away.
So how do larger organizations enjoy the benefits of strategic long-term planning while maintaining agility? By taking a dynamic approach to strategic planning. At the same time, agile proponents need to stop focusing only on the team unit and start focusing more on the enterprise with practical, realistic methods, instead of fancy frameworks.
