The recent turmoil in British politics has led much of the political establishment and commentariat to agree on one thing: what the country needs now is competence, the more boring the better. The last thing we need is risky ideas.
After the erratic administrations of Boris Johnson and Liz Truss, this is understandable. But it risks misreading both the needs of the times and the lessons of history. Profound crises demand leaders who are also imaginative and agile. Competence alone rarely turns out to be enough.
That imagination is now often seen as an alternative to competence, rather than its complement, shows what a serious state British politics is in – as the combined costs of austerity, stagnant productivity, Brexit, weakened public institutions and a diminished standing in the world become ever more apparent.
So serious are these problems, they have prompted one of British politics’ rare sea-changes, with Labour well ahead in the polls after the arrival of yet another Conservative prime minister. But although it’s clear the country needs a period of rebuilding after the mistakes of recent years, it’s much less clear where the necessary long-term ideas, strategies and plans will come from.
Prime Minister Rishi Sunak’s reputation for competence may be overrated as a result of comparisons with his short-term predecessor Truss. Few would describe Sunak as a visionary and one policy he is closely associated with – subsidies to eat out with other unvaccinated people in the summer of 2020 – now looks rather daft.
Meanwhile Labour leader Keir Starmer has promised that under him, “Britain will get its future back”. But he has said little about his vision or longer-term plans for the country, and there is none of the buzz of earlier periods in his party’s history when it pioneered everything from the NHS and Open University to equality laws and constitutional reform.
I believe this shrinking of ambition may be a local symptom of a much bigger problem in global politics, particularly in the west, which needs addressing alongside the immediate pressures of sky-rocketing energy bills and living costs.
The gravity of our current “polycrisis”, spanning war, climate and health as well as finance, obscures a less obvious and visible crisis – of our collective imagination.
A deficit of options
These days, we are good at envisioning future ecological disasters or technological threats. Yet much more than our predecessors 50, 100 or 200 years ago, we struggle to picture or describe how our society could be significantly better a generation from now.
Just ask your friends. Unless they are quite unusual, they will find it much easier to imagine dystopias than utopias, things getting worse rather than getting better. They may be able to paint a picture of a future world full of drones, robots and AI, but will find it much harder to describe what elderly care might look like when they are 90, or a day in the life of the House of Commons a generation from now.
This shrunken view of the future makes our times different from previous eras, whether the 1960s or 1940s, or much of the 19th century, when many had a strong sense of how their societies could get better, fuelled by everything from utopian literature to social movements.
This story is part of Conversation Insights
The Insights team generates long-form journalism and is working with academics from different backgrounds who have been engaged in projects to tackle societal and scientific challenges.
Today’s imagination gap has been very evident in the US amid the (now diminishing) prospect of another presidential election between a septuagenarian and an octogenarian in 2024. But it’s also a long time since we’ve heard a British party leader speak convincingly about what our country might be like even ten years from now, let alone 20 or 30 years ahead.
Some of the blame for this lies with the main parties. Where once they had large research teams working on future options – for example, under Chris Patten for the Tories in the 1970s or Michael Young for Labour in the 1940s – now their staff appear to concentrate mainly on campaigning and tactics.
Some of the blame lies with UK universities, which have largely given up the job of designing radical options and what I call “exploratory social sciences” – focusing instead on analysis of the present and past, or reviving old ideas. Here there is a stark contrast with the sciences, from genomics to AI, where speculative design is enthusiastically encouraged.
And some of the blame lies with our media, which tends to live in an eternal present – with extraordinary attention paid to daily Westminster intrigues but far less to longer-term patterns. Social media, in particular, strips away the depth both of historical perspectives and possible pathways to the future in favour of a feverish present.
The net result is a gap where imagination should be. It means that at a time of intensifying crises, we have a deficit of options. This fuels the pessimism of large majorities of people in many countries, who now expect life to be worse for their children than it was for them.
‘Shrinking the future’
Today’s polycrisis is very much a material crisis involving viruses, war, energy and food. But it is also, in some respects, a psychological crisis. Fascinating recent research surveying the patterns of sentiment in all books published in English, German and Spanish over the last 150 years (as gathered on Google) identifies symptoms of a collective depression in recent decades on a scale greater than during the world wars.
The Indiana University authors write of an upsurge of “cognitive distortions” since around 2000, leading them to comment that “large populations are increasingly stressed by pervasive cultural, economic and social changes” linked with “the rising prevalence of depression and anxiety in recent decades”. The research concludes that “catastrophising” ways of thinking have risen sharply, as utopias have been displaced by dystopias in our collective mind.
This “shrinking of the future” has had an obvious political effect. It has fuelled what German sociologist Andreas Reckwitz describes as the switch from positive politics, which emphasises the openness and possibilities of the future, to a negative version which is defensive, sceptical and nostalgic, convinced that the best years lie in the past.
Donald Trump’s promise to make America great again; Giorgia Meloni’s invocation of traditions of family, God and nation in Italy; Jair Bolsonaro’s nostalgia for the days of military rule which was almost enough to get him reelected – all are examples of playing to a large part of the public homesick for the past.
Read more:
Militant optimism: a state of mind that can help us find hope in dark times
Even leaders who (some of the time) promise a bright new future increasingly link this to stories of national exceptionalism, whether in the form of Narendra Modi’s Hindutva ideology or Xi Jinping’s increasingly strident nationalism. One fascinating symptom is that most of the Chinese Politburo’s recent monthly study sessions (yes, they do have them) have apparently focused on Chinese history and its lessons.
The result of these trends is to narrow down what could be possible – our options for the decades ahead. I often hear people say there is no shortage of ideas out there, and this is partly true. But if you work with mayors or governments around the world, you quickly discover that these ideas are often only half-formed, not yet ready to be put into practice at any scale.
It’s not hard to see, broadly, what transitions are required to a zero carbon economy amid an ageing society in a world full of powerful technologies. Yet there’s a surprising lack of good ideas on exactly what needs to be done in practice. For example, much has been done on the priorities for cutting carbon, but the detail of how to implement these sector by sector, how to sell them politically and how to sequence them is much harder.
Adam Price, leader of Wales’s nationalist political party Plaid Cymru, recently described this as “the tenth chapter problem”, pointing to many of the books he had read that offered brilliant diagnoses of what is wrong with our economy or society in their first nine chapters, only to present answers and prescriptions that are bland and unconvincing in the final one.
Widening the possibility space
Whether there are options is, in part, a matter of choice – of whether societies have invested in the hard graft of preparing themselves for possible futures. For example, the preparatory work done over many decades on a universal health service expanded the UK’s possibility space, making it much easier to create the NHS once the political conditions were right in the 1940s.
More recently, decades of work on how a circular economy might work has made it at least possible that we might retreat from fast fashion or the mountains of e-waste that are such an ugly side of contemporary consumer capitalism.
Social scientists throughout history have believed they had a duty to widen this space. For some of the greatest, from Adam Smith to Karl Marx, diagnosis and prescription were tightly interwoven. Marx’s tomb in Highgate Cemetery in London is inscribed with this famous comment:
The philosophers have only interpreted the world in various ways. The point, however, is to change it.
Yet now such “possibility thinking” has faded from prominence.
Search out well-informed proposals for how welfare, democracy and taxation could be organised a generation or two from now and you’ll find surprisingly little. And on the rare occasions that mainstream media – including the BBC – turn to the future, they tend to call on elderly establishment figures to share their wisdom, rather than listening to the pioneers of novel ideas.
Few of today’s social scientists would share the view of HG Wells a century ago that “sociology is the description of the Ideal Society and its relation to existing societies”. There are some exceptions, such as the attempts to map out pathways of societal transformation that are prominent in the Netherlands and Scandinavia. But this kind of work has become harder and it’s not encouraged by the main funders of social science research, who in my experience tend to be nervous of novel ideas or supporting speculative work that goes beyond incremental change.
Read more:
Futurology: how a group of visionaries looked beyond the possible a century ago and predicted today’s world
Healthy pressures to attend to hard data and evidence have had the unintended consequence of squeezing out attention to the future since, by definition, evidence and data refer to the past and present. Similarly, a well-intentioned focus on research impact has encouraged incremental work on policy – how to tweak a little here or there, ideally aligned with the interests of the government of the day – while discouraging the serious work of long-term policy design. A brilliant idea that will flourish in 30 years time won’t show up in the Research Excellence Framework (Ref), which is now the main way UK academics are assessed.
An equally healthy commitment to rigour in higher education has made it hard, or even career threatening, to be too creative, since any genuinely new idea risks sounding flaky, vague or half-baked (as any radical idea will be in its infancy). This is one reason why the most prominent public intellectuals prefer to revive ideas from half a century ago rather than proposing new ones.
The reliance on peer review as a near-universal assessment method for academic publication globally also discourages the boldest, most speculative thinking – favouring safe proposals over more radical ones that tend to get a mix of very high and very low scores. As a result, many of the brightest academics opt either for analytical work or for the safer space of commentary and critique, and steer clear of the riskier space of saying what they think should be done.
How crisis can accelerate change
Milton Friedman – not everyone’s favourite economist – argued why this reluctance to generate bold new ideas matters as well as anyone. In the 1982 preface to his book Capitalism and Freedom, Friedman writes:
Only a crisis – actual or perceived – produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies [and] to keep them alive and available until the politically impossible becomes the politically inevitable.
Whether or not you agree with a single other word penned by Friedman, he was surely right about this. Our problem now, as we face an accumulation of crises, is that we don’t have enough of those options available, enough ideas ready to make the transition from impossible to inevitable.
Crises can be powerful accelerators of change. Indeed, one definition of leadership is the ability to use the smallest crisis for the largest effect. The COVID pandemic has pointed to how we might remake the economy with less travel and revived neighbourhoods. Through furlough, it opened up new thinking about welfare. It also boosted attention for issues such as mental health.
The pandemic could have prompted serious debate about pay too, since key workers were often paid the least while non-essential workers earned far more. In the US, 47% of the lowest paid were deemed essential workers – a far higher percentage than among the highest paid. In retrospect, it’s remarkable how little our politicians used the pandemic as a springboard for change. In this sense, the COVID crisis looks like a wasted opportunity.
Not all politics is short-termist, however. I work with several Scandinavian governments that take quite a long view, helped by relatively consensual politics and a system that encourages collaboration. Many non-democracies around the world also remain quite good at thinking and acting over decades rather than months and years (but not always for the better).
In the US, by contrast, the dramatic shifts from one regime to another make long-term policymaking very hard. The only thing US politicians can often agree on is massive spending – one luxury of having the dollar as a reserve currency.
The UK is prone to similar short-termism. But one counter-example that I worked on 20 years ago stands up reasonably well to scrutiny and shows what’s needed now. Various UK government departments, as well as the No 10 strategy unit I was running at the time, prepared long-term plans for energy and renewables generation and use as part of a roadmap to radically reduce carbon emissions. Later governments roughly stuck to what was proposed, and cross-party mechanisms were put in place to monitor actions and results.
Since then, the UK has cut its emissions by 40-50% even if you include imports – far more than most people expected. We have shifted to an energy system with a big percentage of renewables (usually around 40%), again much more than was expected.
Read more:
UK target to cut emissions 78% by 2035 is world-leading – but to hit it, action is needed now
There are comparable examples from the social field. Major societal challenges like teenage pregnancy have been dramatically reduced, just as the last Labour government dramatically reduced child poverty and youth unemployment and, for a time, sharply cut street homelessness too.
Yet even activists can be unaware of these facts, and so drift into an unrealistic fatalism – an assumption that nothing really works. A much-circulated picture shows CO₂ emissions relentlessly rising while documenting the various summits that came and went: Kyoto, Copenhagen and Paris. The facts here are not in doubt. But by not showing the significant number of countries which did sharply reduce their emissions, the picture gives a misleading account, and one that is as likely to spur a resigned shrug of the shoulders as a leap into action.
Conditions for more imaginative policymaking
Having worked with many strategy units around the world, including those of Australia, Singapore and the European Commission as well as the UK, I can offer some common threads about what conditions encourage more imaginative (and effective) policymaking.
First, to have a sense of vision and direction, it helps to know what you care about.
Rather obviously, it’s then important to gather evidence on what worked or didn’t work in the past. This won’t always be reliable, not least because the world changes. But you have to be very sure of your ground to go against strong bodies of evidence, as Truss and her chancellor, Kwasi Kwarteng, did in September 2022.
The optimal teams for doing long-term strategic work are diverse – in background, mindset and experience – and should ideally always involve people with frontline experience. The teams who were briefly at the top in the Truss government were almost the opposite: ideologically close-knit and inexperienced with backgrounds in PR and lobbying. Labour faces a parallel problem, with little direct experience of business or technology among their MPs and advisers.
Another lesson is to experiment. Nothing works in practice as it works on paper or on a screen. So wise governments try things out if they can, ideally on a small scale and fast. When Franklin D Roosevelt was elected at the height of the Great Depression in 1933, he promised lots of experiments and said from the start that some would fail, but that not to experiment would be a dereliction of duty. Roosevelt, of course, was re-elected a further three times.
The dozens of universal basic income experiments underway around the world and the experiments of the UK’s Behavioural Insights Team are good examples of the same approach now.
In the early 2010s the Conservatives embraced this experimental approach, which chimes with their traditional scepticism of big government, “top-down” plans. But since David Cameron’s time, they have largely reverted to government by speech and press release – perhaps a symptom of collapsing time horizons. If you may not survive to the end of the year, you are unlikely to start experiments that may take a couple of years to generate usable findings.
In contrast, some governments have deliberately encouraged imagination. The UK in the 1940s was full of commissions trying to think ahead to the post-war world of health, education, welfare and town planning.
Often, prime ministers from both sides encouraged a hinterland of more expansive thought because it opened up new spaces they could move into. The Thatcherites used their sometimes-crazy thinktanks to expand the boundaries of the possible. Tony Blair encouraged a network of thinktanks and gatherings, including regular meetings of progressive leaders such as Luiz Inacio “Lula” da Silva, Helen Clark, Gerd Schröder and Bill Clinton, and futures work within Whitehall which fed into five- and ten-year strategies. Cameron engaged with ideas from environmentalism, behavioural science and the science of happiness.
Read more:
Technology will not save us from climate change – but imagining new forms of society will
Elsewhere in the world, leaders such as Angela Merkel and Emmanuel Macron have invested a lot of time in keeping up to speed with emerging ideas. Singapore even has a Centre for Strategic Futures in the prime minister’s office to institutionalise this kind of horizon scanning. All understood social theorist Max Weber’s famous comment that, although material interests “may drive the trains down the tracks … ideas are the switchmen”.
In the UK over the last decade, however, there has been much less enthusiasm or intellectual confidence among its big political parties. Some attribute this to lack of curiosity in the cases of Theresa May as well as Johnson and Truss. Others argue that backgrounds in law may explain why London’s Labour mayor, Sadiq Khan, and Starmer appear so uninterested in ideas or the future.
There are many methods that they, and their successors, could be using to expand their policymaking options. Foresight exercises and scenarios help to map out what might happen in the future – for example, to the shape of cities. Games, thought experiments and mass participatory exercises can all open up new options.
In my new book, I show how to rethink anything from the design of public libraries to benefit systems, public health schemes to parks, democracy to food – using methods I call “extension”, “inversion” and “grafting” to generate new options.
Extension simply means extending some aspect (like keeping libraries open longer, or extending the idea of human rights to a new field); inversion means inverting roles, for example, as the Grameen Bank turned farmers into bankers or the Expert Patients Programme turned patients into sources of knowledge; while grafting means taking an idea from a very different field, as the idea of the jury was grafted on to democracy in the form of citizens’ juries.
Read more:
Is it possible for everyone to live a good life within our planet’s limits?
These approaches can then contribute to bigger-picture alternatives. What would a truly zero-carbon economy look like in terms of skills, regulations and taxes? What would a radically transformed care system be like? How could cities work in an age of driverless cars, micromobility and greatly increased cycling and walking?
Often there are harbingers of the future in the present, so another place to look for imaginative ideas is in everyday life. Witness the extraordinary vitality of social innovation movements from food to ethics, the work underway to apply “doughnut” concepts to sustainability, and the subcultures around data or health, which tend to thrive at some remove from orthodox politics and the mainstream media.
I always advise politicians to spend time with these innovators and to feed off their energy. Centre-left governments in countries as different as Finland and Portugal have been good at doing this. But you get little sense of anything comparable from the UK’s main parties right now.
Labour’s challenge of imagination
With Labour now likely to win the next election, Britain’s challenge of imagination is closely tied up with the party’s capacity to think and imagine. And here the signs are mixed at best. It’s understandable why Labour is prioritising seriousness and moderation in contrast to a government that has lost the plot. But I believe it risks misreading the historical moment we’re in.
The UK in 2024 will be a very different place from the UK of 1997 or Australia earlier in 2022, when its Labor party secured a majority. These were countries enjoying stability and prosperity. Voters saw little risk in a change of government.
Crisis times, by contrast, demand commensurate boldness. Threats tend to drive the public to seek a haven of safety – which they may find in their sense of nation or in the promise of protective welfare. But they also react subliminally against parties that appear not to grasp the gravity of the situation. Indeed, there is quite strong evidence that if electors believe they are likely to lose out in the years ahead (as many in the UK do now), they are willing to take quite big risks in the present – including voting for populist parties.
This could become a problem for Labour if the Conservatives shape-shift once again. So too could their caution over methods. As far as I can see, Labour has learned little from the newer parties around the world, such as Podemos, M5S and En Marche, with their often radically novel ways of mobilising people online.
Labour’s senior figures struggle to come across as interesting and engaging in the way that social media demands. One symptom of this is that the party currently has only 3,640 followers on Tiktok, compared with Duolingo’s 5 million followers or Manchester United’s 20 million. Look at the few videos the party has put up and you appreciate just how much of a problem this is: excerpts from parliamentary speeches that feel like they could have been made in 1972 or 1992. It’s striking that the best recent social media outputs from Labour have simply cut together speeches by Conservatives (which are, admittedly, funny and powerful).
On policy, too, the approach is perhaps deliberately traditional. Many of its ideas echo mainstream European and east Asian policy of half a century ago, rather than of the world of the 2020s and beyond.
Labour has, for example, now returned to industrial policy of the kind that was normal in countries like Germany, Japan or South Korea in the second half of the last century as they tried to shape, guide and direct their economies. This is welcome in comparison with the alternatives of the Corbynite left and the free market Tory right. But the problem with this return to the mainstream is that it leaves glaring gaps such as what this industrial policy would mean for services, which now dominate the UK economy, or for emerging technologies such as quantum.
Labour’s big proposal for creating a new public organisation, Great British Energy (GBE), also takes its inspiration more from post-war Europe rather than the 2020s. Some of the language echoes Germany’s Energiewende in the 1980s, with its strong emphasis on moving to decentralised energy systems as well as away from fossil fuels. But the design of GBE – focused on production rather than seeing energy systems in the round – seems unsuited to this goal.
An even bigger challenge is how to be progressive without much cash. The next UK government will have none of the fiscal leeway that Labour had after 1997. It’s not hard to design generous, progressive policies if there’s plenty of money around; much harder to do so in times of austerity.
But money isn’t everything, and with a different mindset it’s still possible to plan changes that don’t require lots of money. In the recent past there have been many good examples, from raising money from dormant bank accounts and insurance to fund social projects, to shifting regeneration to brownfield sites. Many digital platforms now mobilise everything from underused parking spaces and spare rooms to household goods, part of a 21st-century way of thinking that is quite common in the green movement and among the digital natives, but still much less familiar in Whitehall and Westminster.
With a bit more confidence, there are many fields where Labour and other parties of the centre-left could be projecting radical alternatives – in a second or even third term, if not immediately. A decade ago, for example, Labour promised a National Care Service – and this is certainly a field where the status quo is not plausible. While in the long run this is bound to need a lot of money, there is much that could be done to prepare the ground now.
Similarly, there is growing interest in mental health policies that address anxiety and depression among large parts of the population, not just small minorities. Many of these are as much about what employers should do as about state provision.
And then there is democracy itself. Evidence of declining public confidence in democratic systems has sparked an extraordinary wave of innovation globally, from digital parliaments and citizen assemblies to new public roles such as Wales’s future generations commissioner. These are just a few examples where Labour could combine pragmatic realism in the short term with bolder accounts of where it wants to go longer term.
There are no fair winds for those who don’t know where they are going. The best leaders combine sharp diagnoses of problems of the present (and plausible responses) with a vision and roadmap for decades into the future. In fields such as engineering, it is assumed that progress depends on combining far-reaching imagination with highly competent implementation.
Yet over the last few years, Britain has too often suffered from the opposite – stunted imagination and incompetent implementation. As we grapple with the polycrisis for (likely) years to come, we will need both imaginative creativity and good implementation. Competence alone will not save us.