“On no one process, on no one country, on no one route, and on no one field must we be dependent. Safety and certainty in oil lie in variety and variety alone,” said Winston Churchill, aware of the risks he was running in converting the Royal Navy to run on oil in 1913. Over a century later, energy supply shocks triggered by geopolitical turmoil make that ethos truer than ever.

Ideological rigidity in the face of a heating planet and war in Europe is seen across the political spectrum: from the failure of Joe Biden’s “Build Back Better” bill, the incoherence of its opponents, and what passes for a federal Australian energy policy, to Germany’s energy response to Russia, and the new British Energy Security Strategy.

Typically, the political right, when it pretends to take climate seriously, puts too much faith in the free market and technology to cut emissions, without actually offering supportive policies. It focuses on boosting fossil fuel production and – a rare glimmer of light – nuclear power, while ignoring energy efficiency and the possible need for conservation in an emergency.

While some doubt the capability of democracy to tackle the climate challenge, authoritarian states such as China have shown that in a pinch economic growth always overrides the environment.

“Green” policies, on the other hand, correctly support renewables but demonise nuclear power and dismiss carbon capture or fossil-fuel derived “blue” hydrogen. They overemphasise command-and-control instead of market-based emissions reduction via carbon pricing. And, resolutely opposed to any oil and gas production, they are unable to think strategically about meeting the continuing, even if diminishing, need for fossil fuels in the most secure and cost-effective way.

Both prescriptions fail, for different reasons, in addressing energy poverty, whether domestically or globally. Composing, so Wagner said, is the “art of transition”; energy policy should likewise be the art of transition from today to utopia without catastrophe in the middle. On the contrary, the real world’s energy transition has delivered tremendous improvements in non-fossil energies and stern commitments on net-zero carbon – but is so far a transition to higher emissions, higher costs and geopolitical peril.

The energy transition conversation in the Middle East and Africa sounds quite different. The region will host the next two UN climate conferences: COP27 in Egypt at the end of this year, COP28 in Abu Dhabi next year. In the build up to this, during the recent MENA climate week, key themes emerged offering an energy transition that is more politically and economically feasible, more socially just, and more secure.

Speaking at the event, Dr Sultan Al Jaber struck the right note, as he has often done before: “We cannot and must not unplug the current energy system before we have built the new one.” He wears several hats covering all aspects of the energy transition including CEO of Abu Dhabi National Oil Company (ADNOC), chairman of clean energy investor Masdar, UAE Minister of Industry and Special Envoy for Climate Change.

Masdar, in which ADNOC and two of Abu Dhabi’s sovereign wealth funds, Mubadala and ADQ, share ownership, set an aim in January to become the world’s biggest renewable company with 100 gigawatts of capacity by 2030, and eventually 200 GW, as well as a world leader in “green” (renewable) hydrogen. German vice-chancellor Robert Habeck was in Doha and Abu Dhabi on 20-21 March to ask for more gas to replace Russian supply, and to sign accords on hydrogen cooperation.

The second theme is that there is an essential place for oil and gas in the transition. Of course, leading hydrocarbon producers prefer such a narrative, and some have been guilty of watering down diplomatic progress on climate before.

OPEC secretary-general, Mohammad Barkindo, said: “The role of oil and gas is guaranteed for the foreseeable future. It will continue to account for 50% of the global energy mix to 2045.” In net-zero scenarios such as that of the International Energy Agency, the share of fossil fuels is much less – but not nothing, still around a third.

The third theme is that of energy access and affordability – particularly for those countries and people with the lowest incomes. Such ideas may resonate more from Barkindo, a Nigerian, than from the wealthy Gulf states. There is a vein of hypocrisy in Western countries’ ceasing to fund fossil-fuelled energy in Africa, a trivial greenhouse gas emitter, while continuing to use and profit from petroleum, and offering only nugatory amounts of “climate” finance to poorer countries.

Cynical arguments to prolong the life of the fossil economy is one thing. Pragmatism grounded in the hard realities of climate, technology, economics and politics is another. The energy transition will obviously be disruptive – it has to be. But to be feasible at all, it must be as smooth, flexible and as diverse as possible.

Robin M. Mills is CEO of Qamar Energy, and author of The Myth of the Oil Crisis.