Is Agile The Antidote To Your Horrible Boss?
If you've toiled in a big organization for any time, you're familiar with the whips and scorns of an insolent supervisor and the sneer of cold command. In the midst of multiple put-downs and slights of your best efforts, it's little consolation that your horrible boss is just as unhappy as you are. You may even suspect that those at the very top are just as frustrated by the constraints imposed on their position.
If you think about it for a bit, late one evening after a hard day over a glass of wine, or perhaps something stronger, you might surmise that you are all imprisoned in the bureaucratic predicament depicted in Franz Kafka's novel, The Castle, where no one knows what the heck is going on or why you are all acting this way. You might even begin wonder: is this the best that the human race can manage? Is there no antidote to soul-destroying bureaucracy?
One possibility that has recently emerged is Agile–a way of running organizations that has become a vast global movement. In principle at least, it sounds attractively anti-bureaucratic, as explained here. In Agile, people and interactions are emphasized ahead of process and tools. Customers and staff constantly interact with each other. Finished work is delivered frequently so that the people can see the impact of their work in days or weeks rather than years. Communications take the form of conversations, not top-down commands. There is close daily cooperation between business people, developers and designers. Continuous attention is paid to technical excellence and good design and regular adaptation to changing circumstances. Teams routinely achieve high-performance and operate in the state of "flow" identified by Mihály Csíkszentmihályi, where those doing the work enjoy energized focus, full involvement, and deep satisfaction in the process of the activity.
Too good to be true? In 2016, a group of firms set out through a series of site visits to find out. They formed a non-profit Learning Consortium and visited eight firms in the US and Europe that said they were operating entrepreneurially at scale. The site visits showed that some of these large organizations, or large units of these organizations, have been successfully operating in an Agile manner at scale for at least five years. They include Cerner, Ericsson's network management unit, Microsoft's Developer Division, Spotify and Riot Games. The operations in question are not isolated experiments in those firms. In each case, they are part of large-scale implementations of Agile that operate non-bureaucratically with continuous innovation.
Where Did Agile Come From?
Where did Agile come from? It's a movement that took off in software development in 2001 with the publication of the Agile Manifesto. It is driven both by the passion of those who love working this way and by managers who recognize that survival in an unpredictable and rapidly changing marketplace requires a capacity to adapt equally rapidly.
As Harvard Business Review noted in its article, "Embracing Agile," in April 2016, Agile is now spreading rapidly to all parts, and all kinds, of organizations. There are already hundreds of thousands of Agile practitioners around the world and tens of thousands of organizations that have embraced Agile.
But the stark reality is that most big organizations remain stubbornly bureaucratic—that is to say, very non-Agile.
Humanity's Fatal Attraction To Bureaucratic Management
As analyst Gary Hamel has pointed out, bureaucracy still "constitutes the operating system for virtually every large-scale organization on the planet… Strategy gets set at the top. Power trickles down. Big leaders appoint little leaders. Individuals compete for promotion. Compensation correlates with rank. Tasks are assigned. Managers assess performance. Rules tightly circumscribe discretion."
The modern organization is built "to minimize risk and keep people in their boxes and silos," as business school professor John Kotter in his book, Accelerate (2014), writes. People "are working with a system that is designed to get today's job done—a system that asks most people, usually benignly, to be quiet, take orders, and do their jobs in a repetitive way." It's not much fun, the thinking goes, but it is necessary. That's the way the world is.
Why Is Bureaucracy So Hard To Dislodge?
Why is bureaucracy still rampant? For one thing, it's been around for an awfully long time. In fact, as early as the invention of writing in ancient Sumer around 3500 BC, bureaucracy was used to administer the harvest and allocate its spoils. It was deployed to build the pyramids of ancient Egypt. It ran the Roman and the Chinese Empires for more than a thousand years. It is the foundation of the modern military machine and the basis for running giant organizations like steel mills, railways and mass production factories.
Imperial bureaucracy in ancient China. (Image: Public Domain, Wikimedia)
Given bureaucracy's track record since the very beginning of civilization, is it any wonder that traditional managers continue to believe that bureaucracy constitutes the rational way to organize any large-scale human activity? Top-down processes and organized hierarchies are seen as necessary to maintain order and maximize efficiency.
Instead of individuals doing whatever they happen to feel like doing, inanimate systems, processes, procedures, files, hierarchy, and rules drive disciplined decision-making and systematically eliminate waste in order to force relentless progress towards the organization's goal. The logic is inexorable. Bureaucracy is simply the way things are—the only way to get things done.
The Flaws Of Bureaucracy
Bureaucracy of course has had its critics. One downside is its human cost. Embedded in bureaucracy is the requirement for all to obey the system unquestioningly and habitually, even when no explicit orders have been given. Thus we know what our office, our factory, or our horrible boss, wants and we deliver it to them. Our complicity with these unstated dictates is what makes modern bureaucracy possible. The economy depends on it. Our material prosperity flows from it. Efficiency requires it. The modern economy is bureaucracy. Any attempt to deny this fact is to deny the nature of modern society itself.
It was German sociologist Max Weber who reminded us what poets have always known: bureaucracy makes us miserable. It traps us in an impersonal "iron cage" of rule-based behavior. "The system" effectively coerces us to slip seamlessly into the demands of repetitive production. We adjust our natural characteristics as human beings to the systems of the organization. In the process, we willingly reprogram ourselves according to the requirements of the task at hand.
The situation may be dressed up in rah-rah terms like career, promotions, advancement, titles, status, rewards. merit increases, bonuses and the like. But behind the HR hoopla, the process does not encourage human flourishing. We are being manipulated as things, as the ubiquitous term "human resources" makes explicit. We should therefore hardly be surprised that today only one in five workers is fully engaged in his or her work, and even fewer are passionate about it.
Weber himself that saw one possible escape from bureaucracy through charismatic leadership, in which demagogues might rise "above the rules" and excite us by persuading us that they can make us, the organization—or the country—great again, through sheer force of personality. In the political sphere, positive figures like Gandhi and Nelson Mandela emerged, along with more harmful incarnations like Adolf Hitler. In the business sphere, people like Jack Welch and Steve Jobs have also claimed to "break the old rules" and assert a new order through the force of personality. Yet when the dust has settled on their efforts, the new order has often looked remarkably like the old one, albeit with different labels. Charisma is not a promising approach to solve the problems of bureaucracy. .
But Weber was not alone in seeking alternatives to bureaucracy. Throughout the 20th century, theorist after theorist suggested that humanistic management with an emphasis on teams and collaboration would be a better way to get work done. It began with Mary Parker Follett in the 1920s, and continued with Elton Mayo and Chester Barnard in the 1930s, Abraham Maslow in the 1940s, Douglas McGregor in the 1960s, Peters and Waterman in the 1980s, on to Smith and Katzenbach in the 1990s.
As part of these efforts, multiple fixes like task forces, special project groups, strategy departments, tiger teams, skunk works, R&D, dual operating systems, knowledge funnels, design thinking and so on were tried.
In The Age of Heretics (1996) Art Kleiner celebrated the corporate mavericks of the 1950s, '60s and '70s who pioneered self-managing work teams, responsiveness to customers, grassroots organizing and other ways to imbue corporations with a sense of the value of human relationships.
The problem is that these experiments, some of which were strikingly successful at the time, didn't seem to last or spread. "Over the decades," says Hamel, "many organizations have run successful experiments with post-bureaucratic practices, but these little efforts are either soon aborted, or marginalized."
Despite all the talk of corporate change, Hamel says that we have killed the cancer of bureaucracy. "We've flattened corporate hierarchies, but haven't eliminated them. We've eulogized empowerment, but haven't distributed executive authority. We've encouraged employees to speak up, but haven't allowed them to set strategy. We've been advocates for innovation, but haven't systematically dismantled the barriers that keep it marginalized. We've talked (endlessly) about the need for change, but haven't taught employees how to be internal activists. We've denounced bureaucracy, but haven't dethroned it."
Bureaucracy thus seems unstoppable. It's what the modern organization, and the modern economy, rests on. Raging against bureaucracy is like lamenting the weather— it's beside the point. In a modern organization, process trumps individual initiative; if it didn't, it wouldn't be a modern organization.
If this is true, replacing your horrible boss won't get you far. He will be replaced by another, perhaps slightly more sympathetic and possibly less obnoxious, perhaps wearing jeans and sporting a smile. But he too will be chained to the same bureaucratic treadmill, another victim of "the system." Once again, system, process, procedure, files, hierarchy, and rules will be driving the decision making, no matter what any individual thinks. Wishing away bureaucracy is a fool's errand: it's like wishing for a world without gravity.
Or is it?
Why Agile Might Defeat The Cancer Of Bureaucracy
There are grounds for thinking that Agile might succeed in defeating the cancer of bureaucracy, where earlier efforts have failed.
Thus we now know that the cost of bureaucracy is high. Gary Hamel and Michele Zanini suggest that in the USA it's $3 trillion annually. There is thus a strong business case for eliminating bureaucracy. Firms that don't pursue that business case may have trouble flourishing.
We also know that the world has changed in a way that makes bureaucracy the wrong choice. The basic assumption of bureaucracy is that the world is a stable and predictable place. Today we live in a VUCA world—a volatile, complex, uncertain and ambiguous. In such a world, an inability to be nimble in the face of rapid change can be a fatal flaw. Already, the life expectancy of a US corporation has declined drastically—from 75 years to just 10 years. Bureaucracy, instead of being the basis of our material prosperity, is now a constraint. Organizations have to find a solution to the problem of bureaucracy. Otherwise they won't survive.
By contrast, Agile responds to the particular way in which the world has changed. As a result of globalization, deregulation and new technology, power in the marketplace has shifted decisively from the seller to the buyer. Now the customer is the center of the firm's universe. This is more than an increased attention to customers: it is a fundamental shift in the goal of the organization—a veritable Copernican revolution in management.
Unlike the conventional picture of bureaucracy as a top-down pyramid of boxes, the Agile organization has an interactive relationship with customers, who are conceptually very much part of the organization. The explicit focus on delivering continuous new value to customers makes Agile particularly relevant to the challenge of modern management
Thus Agile thinking goes deeper than those 20th century attempts to reform bureaucracy through humanistic management. In retrospect, they were mere tweaks to the concept of the organization as a static machine. Big bosses continued to appoint little bosses, and so on down the line. Having a few teams that worked pretty well didn't make much difference to the firm as whole. The organization continued to operate like a giant warship—big and disciplined but slow and hard to maneuver.
By contrast, Agile transforms the very concept of an organization. When the whole organization embraces Agile, the organization functions less like a giant warship, and more like a flotilla of tiny speedboats. Instead of a steady-state machine, the whole organization is an organic living network of high-performance teams that is in constant flux to exploit new opportunities and add new value for customers. In such organizations, managers recognize that competence resides throughout the organization and that innovation can come from anywhere. The whole organization, including the top, is obsessed with delivering more value to customers. Agile teams can take initiative on their own, and interact with other Agile teams to solve common problems. In effect, the whole organization shares a common mindset in which organization is viewed and operated as a network of high-performance teams.
Unlike the individual reform efforts of the 20th century, Agile has now become a huge global movement. There are tens of thousands of organizations embracing Agile and hundreds of thousands of Agile practitioners. At this scale, it hardly matters whether any individual firm succeeds or falters. What's more important is that the Agile movement as a whole advances.
Do Agile Organizations Exist In Practice?
The final objection to Agile from traditional managers is usually that Agile is a fantasy. It sounds good, but in practice, it just doesn't work at scale: big firms can't be Agile. Agile, the argument goes, may occasionally work with tiny teams or small firms, but when firms grow up and become real firms, they find they need bureaucracy. Agile is no solution for big organizations. It can't handle complex operations. It doesn't operate at scale. And so on.
In 2016, the site visits of the Learning Consortium—a non-profit corporation comprising eight firms in the US and Europe that are sharing their experiences in operating entrepreneurially at scale—suggest the opposite. The site visits showed that a number of large organizations, or units of large organizations, have been operating in an Agile fashion at scale for at least five years.
The member firms range in age from 8 years to 326 years. The firms operate globally and have been on their various Agile journeys for periods ranging from 15 months to 15 years. Some of the firms were "born Agile" while others are engaged in transformation from top-down bureaucracy. Some are in rapidly growing sectors, while others are in mature sectors where there is intense competition and pressure to cut costs.
In the site visits, we saw a combination of the revolutionary and the evolutionary. For instance, at Microsoft, they started with one team in 2008; several teams in 2009, 35 teams in 2010, several hundred teams in 2011 and then a decision to carry across the whole organization 2014. There was revolutionary intent, but it was implemented in an evolutionary fashion. They didn't start out with big structural changes. Those changes came later, but in an organic fashion.
Each of the site visits included more than ten people learning what each company has done and then exploring in greater depth the issues that were uncovered. After the visits, the member firms got together for several days to review what had been learned and identify common themes. Each firm is sharing what is learned within its own organization in order to spur enhanced implementation of entrepreneurial goals, principles and practices. The full report of the findings will be available here in November 2016.
The Drucker Forum 2016 And Beyond
A discussion of these issues will also take place at the Drucker Forum in Vienna Austria on November 17-18, 2016. I will be chairing a session with Gary Hamel entitled "The Entrepreneurial Organization At Scale." The sessions will be live-streamed: you can enroll in the live-streaming here.
So there are reasons to think that relief from your horrible boss may be at hand. Agile is now a vast global movement that is spreading rapidly. If you look round your own organization, no matter how bureaucratic it may be, you will probably find one or more Agile teams, perhaps working under cover, waiting for the day when your organization yields to the inevitable and opts to embrace Agile. A management revolution is under way, perhaps coming soon to a theater near you.
Disclosure: I am an unpaid pro bono adviser and director of the SD Learning Consortium.
And read also:
Follow Steve Denning on Twitter at @stevedenning
Enviado desde mi iPad