By Jim Muir
17 October 2017
Rising To the Challenge
Just as the 9/11 Twin Towers atrocity in 2001 had left the US and its allies with no alternative but to strike back with extreme force, the challenge mounted by IS through its numerous provocative outrages in the region and around the world could not be ignored - nor was it meant to be.
But rooting out the militants from Iraq and Syria was always going to be much more complex than the campaign unleashed more than a decade earlier against al-Qaeda and its Taliban hosts in Afghanistan.
There was never any real question of a wholesale Coalition invasion of either country. Air strikes were one thing, but putting in vast numbers of troops on the ground would be something else.
Syria was embroiled in a bewilderingly complicated civil war. Iraq was also in turmoil. US forces had only pulled out three years earlier. There was no appetite for a repeat of an invasion which was almost universally seen as a disastrous blunder with lasting consequences.
But uprooting the militants could not be done by air power alone. Reliable, motivated partner forces on the ground would be needed. In both countries, they were in short supply.
In Syria, the US and its allies were already engaged with some of the motley array of Syrian rebel groups. But they never amounted to the kind of focused, cohesive forces that the Coalition was looking for - they were fragmented, fractious, increasingly Islamist in tone, and largely bent on overthrowing the Assad regime rather than turning on IS.
In Iraq, the regular armed forces were in disarray after collapsing like a house of cards in the face of the onslaught on Mosul by a few hundred IS militants. Much work would have to be done to weld them into a motivated and competent fighting force.
In both countries it was to the Kurds in the north that the Americans and their Coalition partners first turned as their most hopeful immediate allies.
The Kurdish Peshmerga forces in northern Iraq had a long relationship with the Western Coalition powers. In 2003, US Special Forces fought alongside them to clear Islamist militants from mountain positions near the Iranian border just before the invasion against Saddam Hussein, in which they were partners.
Humiliated by the loss of vast areas of territory to the lightning IS assault in early August 2014, the Peshmerga needed no encouragement to hit back. Less than two weeks later, with Coalition air support, they recaptured the strategic Mosul dam.
In the months that followed, they slowly clawed back much of the ground they had lost, including Sinjar in November 2015, in operations closely co-ordinated with Coalition air strikes.
But in the rest of Iraq, the picture was much messier. Iraqi government forces were able to halt the IS thrust southwards towards the capital, Baghdad, but made little early progress pushing back into the mainly Sunni areas where the militants had dug in.
It was Iranian-backed Shia militias, whose emergence was consolidated under the IS threat, that made the running in the initial phases of the campaign to oust the militants, driving them out of areas north-east of Baghdad near the Iranian border and pursuing them westwards until the eventual recapture of Saddam Hussein's hometown Tikrit in the Tigris river valley in March 2015.
"If it weren't for Iran, the democratic experiment in Iraq would have fallen," said Hadi al-Ameri, leader of the Iranian-backed Badr Organisation, one of the biggest Shia fighting groups, known as the Hashd al-Shaabi or Popular Mobilisation (PM), that rose in defence of Baghdad and the south.
"Obama was sleeping, and he didn't wake up until IS was at the gates of Irbil. When they were at the gates of Baghdad, he did nothing. Were it not for Iran's support, IS would have taken over the whole Gulf, not just Iraq."
But in the Sunni heartlands of Anbar province to the west of Baghdad the Iraqi state forces were struggling.
In May 2015, IS militants captured the provincial capital, Ramadi. At the same time, they spread their grip in Syria, taking the iconic city of Palmyra (Tadmur) in the desert east of Homs. It was a clear signal that while the organisation may have been dealt some setbacks it was still at the height of its powers and capable of exploiting any weaknesses it found.
But the Americans were quietly busy preparing for the real war on IS. They had poured billions into rebuilding the Iraqi army after 2003, only to see much of it go to pieces under pressure because so much was wrong with its foundations.
Now, putting in several thousand advisers, they focused on training and equipping the key Iraqi special forces - the Counter-Terrorism Service (CTS), otherwise known as the Golden Division - that was to play a crucial role spearheading the drive against the militants.
The systematic flashback began with Anbar province and the eventual recapture of Ramadi at the end of 2015 and early 2016, a grinding and costly operation with close Coalition air support. There was huge destruction and the displacement of most of the city's largely Sunni population. There were also tensions over the deployment of Iranian-backed, mainly Shia militias in core Sunni areas, though they were largely kept out of the city itself.
The recapture of Ramadi tightened the noose around the even more iconic centre of Sunni sentiment at nearby Falluja, where militants had fought two major battles with the Americans in 2004. Government and PM forces began closing in around the city in early 2016, soon after taking Ramadi.
By now, after a lot of testing and assessing, the Americans and their allies, working closely with local forces, were clearly embarked on a comprehensive strategy to "degrade and ultimately destroy" IS wherever it could be found.
The broad lines were spelled out in February 2016 by Lt Gen Sean McFarland, then the commanding general of the Coalition's Combined Joint Task Force - Operation Inherent Resolve.
"The campaign has three objectives: to destroy the ISIL parent tumour in Iran and Syria by collapsing its power centres in Mosul and Raqqa; two, to combat the emerging metastases of the ISIL tumour worldwide; and three, to protect our nations from attack."
Next step on the road to Mosul, the battle for Falluja was joined in late May 2016 and took just over a month of assaults, bombardments and air strikes to complete, with civilians again paying a heavy price. IS fighters deployed their full range of battle tactics, with snipers, booby-traps and suicide bombers making advance difficult and dangerous for the attacking forces.
Despite the attrition suffered by the Iraqi forces, especially the elite CTS forces, it was not long before eyes turned to the big prize - Mosul. With President Obama due to bow out early in the New Year, the Americans were getting impatient.
In peacetime roughly 10 times the size of either Falluja or indeed the IS "capital" Raqqa, Mosul was never going to be an easy nut to crack. It had fallen to the militants in a matter of hours in 2014. By the time the government and Coalition campaign to dislodge them was finally launched in mid-October 2016, IS had had nearly two-and-a-half years to dig in and prepare its defences.
Inside IS's Ruined 'Capital'
Figures walk through devastated street in Raqqa
There is a moment in the journey into Raqqa when you leave the real world behind. After the bombed-out Samra bridge, any signs of normal life vanish.
Turn right at the shop that once sold gravestones - its owner is long gone - and you are inside the city.
Ahead lies nothing but destruction and grey dust and rubble.
This is a place drained of colour, of life, and of people. In six days inside Raqqa, I didn't see a single civilian.
The City Fit for No-One
Marshalled against a band of militants variously estimated at between 2,000 and 12,000 with a hard core probably closer to 4,000 or 5,000, was a vast assembly of government and auxiliary forces, estimated at around 100,000, counting regular army and federal police units, Kurdish Peshmerga forces, and the Popular Mobilisation, including some Sunni and minority volunteers as well as Shia militias.
It was an epic battle that was to last nine months. Government forces slowly battered their way in around the city through dozens of outlying villages and finally into Mosul's eastern half, which was declared "liberated" towards the end of January 2017. Then it was on to the western side of the river and the old city, with its dense warren of narrow streets.
The area around the ancient Great Mosque of al-Nuri, where Abu Bakr al-Baghdad had made his only public appearance as the newly-anointed "caliph", was the last to fall. The mosque was blown up, almost exactly three years after the declaration of the IS "caliphate". The evidence was that IS itself had destroyed it, rather than let it fall into "infidel" hands. The Iraqi Prime Minister, Haider al-Abadi, saw this as "official recognition of defeat" by the militants. It signalled, he said, "the collapse of the IS caliphate and a grand military victory".
It was indeed a monumental achievement, a massive body-blow to the IS state project. The city had been, as a senior Western official in northern Iraq put it, "the beating heart of IS", in real terms much more important than Raqqa in Syria.
But it came at huge cost. The years of preparation and months of battering that it took to dislodge a few thousand militants stood in stark contrast to the ease with which they had rolled into the city virtually unopposed in June 2014. Some estimates put the number of civilians killed at more than 40,000. At least 1,200 Iraqi security personnel also died in battle. The damage was estimated at $50bn. Hundreds thousands of mainly Sunni inhabitants were displaced, and there were widespread allegations of abuses.
The weeks that followed saw a further rapid erosion of the group's territorial position in Iraq. By the end of August 2017, it had been driven out of Tal Afar, an important town controlling the main road from Mosul towards Syria. A month later, another long-standing IS pocket in and around the mainly Sunni town of Hawija, west of Kirkuk, was also largely overrun by government and auxiliary forces.
In Iraq, that left IS in control only of the area in and around the town of al-Qaim, on the River Euphrates by the Syrian border in the far west of Anbar province.
Its presence in the country had been very radically degraded. But the organisation was still far from destroyed.
Presentational Grey Line
Across in Syria, the overall picture by the autumn of 2017 was roughly the same, despite the country's very different, extremely complex and fast-moving situation.
In 2014 and 2015, IS had substantial holdings in Syria - it looked as though someone had spilled a bottle of black ink on its map. They controlled virtually the whole of Syria's stretch of the Euphrates, from Albu Kamal on the Iraqi border, up through Deir al-Zour and Raqqa, the Tabqa dam, and on to Jarablus on the border with Turkey, one of several crossings that were lifelines for the militants. They held towns and villages to the east and north of Aleppo (including the near-mythical Dabiq) and other pockets further south, including Palmyra and nearby oilfields.
But this heyday did not last. In Syria, IS found itself facing, not one motley coalition of enemies as in Iraq, but three.
The Americans, despairing of Syrian rebel groups whose interests lay elsewhere, adopted the experienced Syrian Kurdish fighters of the YPG, the Popular Protection Units, as their main on-the-ground partners. They added on as many Arab and minority fighters as they could find to produce the Syrian Democratic Forces (SDF), their favoured vehicle against IS in Syria.
This did not go down at all well with the US's Nato ally Turkey, which sees the YPG and their parent political body, the Democratic Unity Party (PYD), as terrorists because of their affiliation to Turkey's own Kurdish rebels, the Kurdistan Workers' Party (PKK).
Often seeming to be working at cross-purposes to the Americans, Ankara set up its own coalition from among the Syrian opposition groups it had been backing and facilitating for years.
IS also received attention from a third coalition, grouping Syrian government forces, Iran and Russia. The Russians carried out air strikes against IS positions partly with the same kind of motivation as the Western Coalition - revenge for the downing by IS of the Russian airliner over Sinai in October 2015 - but their real strategic interest was different.
The Russians and Iranians were bent on securing their investment in the Assad regime, helping it restore its authority in the country, and pre-empting any move by the Americans and their allies to block a link-up between eastern Syria and western Iraq that would give Tehran a direct land connection from Iran all the way to Lebanon.
That thought may have been somewhere in Washington's head, but its primary concern was hitting and destroying IS.
Turkey for its part had a different overriding preoccupation: to prevent the Syrian Kurds from expanding and joining up the areas they controlled along the southern side of the Turkey-Syria border.
Despite these conflicting motivations, for IS the result was the same - pressure on many fronts. The US-backed SDF made the most headway, systematically whittling away IS assets step-by-step, but Turkey and Syrian government forces were also on the march.
As early as September 2014, US air strikes had helped prevent IS taking over the Kurdish-held town of Kobane on the border with Turkey. After that, the SDF were pushing forward. By October 2016, IS had lost all access to the Turkish border, a huge logistical setback - virtually all foreign jihadists bound for both Syria and Iraq had come that way, apart from other advantages.
The Turkish-backed Syrian rebels also played their part, taking over IS-held areas north and east of Aleppo - including Dabiq, which fell in a mundane enough manner with none of the apocalyptic "burning of the Crusader armies" heralded in IS mythology.
Gradually the militants' tide receded further and further down the Euphrates valley, with the SDF, backed by perhaps 2,000 US special forces with artillery and air power, doing most of the pushing. By November 2016, the SDF were in a position to launch a campaign to isolate and eventually attack the IS "capital", Raqqa. The assault on the city itself began in early June 2017.
By then, much of the population and many IS fighters had already fled, with most of the militants heading further downstream to the final major city largely under their control - another provincial capital, Deir al-Zour.
But before the recapture of Raqqa had been completed, Russian- and Iranian-backed Syrian government forces were already thrusting towards Deir al-Zour, where a military base and airport had held out against the IS militants for almost three years. To get there, the government forces had recaptured Palmyra (for the second time) in March and pushed eastwards along the road to take Sukhna in August before reaching Deir al-Zour itself in September.
US-backed SDF forces had already bypassed the yet-to-fall Raqqa and moved into oil-rich areas to the east of Deir al-Zour, laying claim to oil and gas fields that would greatly bolster the Kurds' ambitions to run their own area in the north of the country - or at least strengthen their hand in eventual negotiations with Damascus.
So as 2017 moved into its final months, IS territorial control in Syria seemed to be as doomed as it was in Iraq - bottled up in a small patch straddling the border around al-Qaim in western Iraq and Albu Kamal in Syria. Rooting the militants out from there would not necessarily be a simple task in a fiercely tribal area prone to defying governments.
But to all intents and purposes, the IS dream of administering and expanding a thriving Islamic state in the two countries as the core of a growing global caliphate had crashed in flames.
So had IS been virtually destroyed as well as severely degraded? Was it time to celebrate its demise?
It had certainly been dealt crippling blows in terms of functioning as a territorial state. That project lay in ruins. The administrative structures it had set up had been swept away. Major sources of finance had been cut off - from the oilfields it had controlled, taxes imposed on the millions of Iraqis and Syrians it had ruled, and cash coming in across the border with Turkey. It could no longer attract thousands of foreign jihadists or recruit locals to replace the 60,000 IS fighters the Americans believe have died since the heady days of June 2014.
With virtually all of their urban holdings lost, the militants were shrinking back into the shadows and deserts. Many of their leaders had also been killed, although Abu Bakr al-Baghdadi himself apparently eluded the hunt, issuing a defiant audio statement in September 2017. Among the known dead were Mohammed Emwazi, "Jihadi John", killed by a drone strike in Syria in late 2015.
But all of this had been long foreseen by the leadership. In May 2016, the man regarded as the organisation's real Number Two, spokesman and head of external operations Abu Muhammad al-Adnani, made a lengthy statement already foreshadowing the next phase of IS existence.
"You will never be victorious. You will be defeated," he told the Americans. "Do you think that victory comes by killing one leader or another? That would be a fake victory!"
"Or do you consider defeat to be the loss of a city or the loss of territory? Were we defeated when we lost the cities in Iraq [in 2007-8] and were in the desert without any city or land? And would we be defeated, and you victorious, if you were to take Mosul or Sirte or Raqqa, or even all the cities, and we were to return to our previous situation? Absolutely not! True defeat is the loss of will and desire to fight."
Adnani, who as spokesman had announced the birth of "Islamic State" and the caliphate in June 2014, may not have lost the will and desire to fight. But he did lose his life. He was killed by an American air strike near Aleppo in August 2016.
He was right to suggest that the targeted killing of militant leaders has in general had little discernible impact on the course of movements, psychologically important though it may be.
But the Iraqi expert on radical movements, Hisham al-Hashemi, believes the case of Abu Bakr al-Baghdadi might be different. The Americans are unlikely to rest until they have killed Baghdadi, not least because of their belief that he personally repeatedly raped an American NGO worker, Kayla Mueller, and then had her killed in early 2015.
"IS's future depends on Baghdadi," Hashemi said, speaking at a time when IS still controlled huge swathes of territory in Iraq and Syria.
"If he is killed, it will split up. One part would stay on his track and announce a new caliphate. Another would split off and return to al-Qaeda. Others would turn into gangs following whoever is strongest."
"The source of his strength is that he brought about an ideological transformation, blending jihadist ideas with Baathist intelligence security methods, enabling him to create this quasi-state organisation."
Baghdadi's death might not have such an impact in the new situation. But his special talents could come into their own again should conditions in either or both of those countries, or elsewhere, once again favour a comeback by the militants.
That is not so fanciful, even amid the smoking wreckage of the IS above-ground infrastructure.
In Iraq, the burning Sunni grievances against the Shia-dominated Baghdad government that allowed IS to make its astonishing advances in 2014 have not been seriously addressed. American officials are already referring to the danger of an "AQI-2" emerging - a second al-Qaeda in Iraq. It has happened before.
The wholesale destruction of Sunni towns and cities in the drive against IS and the displacement of hundreds of thousands of Sunnis may even have aggravated those grievances, providing fertile ground for militants to infiltrate back.
There is an optimistic argument that the united cross-sectarian effort that it took to drive the militants out may have sown the seeds of a new Iraqi nationalism. But the fact remains that the Sunni community has been crushed, and the Shia majority dominates both politically and through the proliferation of its powerful militias. Political leaderships in Baghdad have signally failed to promote any serious national reconciliation since 2003.
"IS was essentially an Iraqi creation," said a senior Western intelligence official, speaking shortly before the militants were ousted from Mosul.
"The tragic reality is that at the moment, it is the main Sunni political entity in Iraq. From the West, it's looked at as a kind of crazed cult. Here in Iraq it represents an important constituency. It represents a massive dissatisfaction, the alienation of a whole sector of the population."
Putting IS On Trial
A young man wearing a shabby, brown prisoner's outfit stands before three black-robed judges in a tiny, provincial courtroom, shaking nervously.
After sipping some water, he confirms his name: Abdullah Hussein. He is accused of fighting for so-called Islamic State (IS).
"The decision of the court has been taken according to articles 2 and 3 of the 2005 Counterterrorism Law," states the judge. "Death by hanging."
And then Hussein - who, like many suspects here, was picked up on the Mosul frontline - breaks down crying.
Much work remains to be done if that sobering assessment is to be reversed. And Baghdad meanwhile faces multiple challenges that might help tip it back into the state of chaos in which the militants could thrive.
The state's coffers are empty at a time when the reconstruction bill is estimated conservatively at $100bn. Corruption is endemic. Apart from the Sunni issue, Baghdad politics is in turmoil, with deep rivalries between Shia factions with greater or lesser attachment to Iran. And the threat of secession by the Kurds in the north, after the September 2017 independence referendum, could bring further instability.
The very least that could be expected post-Mosul was an indefinite period of low-level insurgency, with bomb blasts and suicide attacks in different parts of the country, especially aimed, as before, at mainly Shia areas. It was already happening. The US military's Combating Terrorism Centre reported that by mid-2017 there was already an average of 130 insurgent attacks taking place monthly in the "liberated" eastern half of Mosul.
And it was already happening next door in Syria too, with IS carrying out random deadly explosions in Damascus and elsewhere despite its stark loss of territory. Although it was obvious by late 2017 that the Assad government with its Russian and Iranian backers were prevailing, the final pattern of control and governance country-wide remained uncertain.
There were many ways in which further turmoil could ensue, not least the fact that, with US support, Kurdish forces had spread well beyond their own demographic comfort zone, causing huge resentment among some Sunni elements. As in Iraq, many Syrian Sunnis were bound to feel aggrieved by everything that had happened since the civil war broke out in 2011.
And where there were grievances, IS or its successors would surely be there to build on them.
Capitalising On Chaos
IS had in any case been busy spreading its bets and developing other territorial possibilities beyond the "parent tumour" of Iraq and Syria.
Libya initially proved the most promising. It had just the kind of failed-state anarchy, the state of "savagery", that left room for the jihadists to move in, forging alliances with local militants and disgruntled supporters of the overthrown regime of Muammar Gaddafi, just as they had done in Iraq.
IS signalled its arrival there in typical style, issuing a polished video in February 2015 showing a group of 21 bewildered Egyptian Christian workers in orange jumpsuits being beheaded on a Libyan beach, their blood mingling with the waters of the Mediterranean as a warning to the "Crusader" European countries on the other side of the sea that radical Islam was on the way.
The man who voiced that warning was believed to be the IS leader in Libya, an Iraqi called Wissam al-Zubaydi, also known as Abu Nabil. By coincidence, Zubaydi was killed in a US air strike on the same day IS struck in Paris, 13 November 2015.
The militants took over a big stretch of the coast and the central city of Sirte, which was to Muammar Gaddafi what Tikrit was to Saddam Hussein. Another American air strike in February killed (among nearly 50 other people) Noureddine Chouchane, reputedly an IS figure responsible for the deadly attacks on Western tourists in his native Tunisia.
But the IS presence in Libya soon came under pressure, with militias loyal to the Government of National Accord, which been born out of UN efforts, dislodging the militants from Sirte in the second half of 2016.
But they did not disappear. The Americans launched an air strike on IS elements south of Sirte in October 2017, their first such action in Libya since President Trump took over.
The militants were undoubtedly on the defensive in Libya. But it remained a deeply fragmented country, and its new government far from powerful or universally accepted. There would likely continue to be pockets of chaos there for the jihadists to exploit.
And there was no shortage of other possibilities being actively explored and developed - Yemen, Afghanistan, Pakistan, Somalia, the Philippines, Nigeria... wherever there were dysfunctional states and angry Muslims, there would be opportunities for IS, competing strongly with a diminished al-Qaeda as a dominant brand in the jihadist market.
Adding the extra risk for the West, that that competition could be another spur for spectacular terrorist attacks which they know are being actively plotted.
Long before IS lost almost all its territorial gains in Iraq and Syria, it was clear that the organisation, while trying its best to hold ground wherever it could, was trying to stay alive and assert itself ("Remaining and Expanding") in other ways - renewed insurgency in the lost lands, constant propaganda and proselytising on the internet, and above all, a revitalised push to carry out acts of terror in the West, the more shocking the better.
As early as September 2014, just as IS was embarking on its provocative confrontation with the West, spokesman Abu Muhammad al-Adnani urged the movement's followers in graphic detail to kill the hated infidels.
"If you can kill a disbelieving American or European - especially the spiteful and filthy French - or an Australian, or a Canadian, or any other of the unbelievers waging war, including the citizens of the countries that entered a coalition against the Islamic State, then rely upon Allah, and kill him in any manner or way however it may be. Smash his head with a rock, slaughter him with a knife, run him over with your car, throw him down from a high place, choke him or poison him."
In his statement in May 2016, al-Adnani signalled the organisation's new stress on attacks inside Western countries in response to the drive against it in Iraq and Syria.
"Truly, the smallest act you do in their lands is more beloved to us than the biggest act done here; it is more effective for us and more harmful to them... Know that your targeting those who are called "civilians" is more beloved to us and more effective, as it is more harmful, painful, and a greater deterrent to them."
Attacks in Europe and elsewhere intensified almost in direct proportion to IS' loss of terrain in Iraq and Syria. Among the more notable were the truck assault in Nice, France, in July 2016 (at least 86 killed) and a similar attack in Berlin in December (12 killed), the Istanbul nightclub bombing in January 2017 (39 killed), and two vehicle and knife attacks in London in March and June 2017 (15 killed).
The attacks in London broke new ground for IS, and even more so the suicide bomb attack at the Manchester Arena in May 2017 in which more than 20 people died. A van attack in Barcelona in August in which 13 people were killed was also seen as an advance for the militants.
These kinds of attacks, often carried out by "lone wolves" and involving everyday implements that are impossible to control, were clearly a nightmare for security services.
Their job was made all the more challenging by studies showing that fewer than 20% of "successful" attacks in Europe and North America were carried out by jihadists who had gone to Syria or Iraq, and that most were perpetrated by citizens of the countries under attack.
While IS had managed to plant a small number of activists in the crowds of refugees and asylum seekers besieging the gates of Europe and the West, simply vetting them carefully would obviously not be enough.
Clearly, IS internet messaging was still powerful, resonating among the disaffected, the resentful, the alienated and disempowered.
The Americans, focusing largely on the military defeat of the militants, seemed taken aback by IS's ability to keep up and even expand the challenge.
"When I consider how much damage we've inflicted and they're still operational, they're still capable of pulling off things like some of these attacks we've seen internationally, we have to conclude that we do not yet fully appreciate the scale or strength of this phenomenon," said Lt Gen Michael Nagata, one of the top US Special Operations officers, quoted by the Combating Terrorism Centre.
"We spend an inordinate amount of time and resources as the United States, but also as our partners, trying to not only defeat ISIS and their control of the physical caliphate, but their virtual space that they own," added Thomas P Bossert, President Trump's adviser on homeland security and counter-terrorism adviser.
"They're proselytizing. It's troubling."
Clearly, tackling such a vague, ethereal and pervasive threat was a far cry from the clear-cut task of confronting IS fighters on the battlefield. The threat was not going to go away. Cyber-inspired lone wolf attacks could materialise at almost any time, any place. The price of relative safety would be eternal vigilance.
Battle for Minds
The US and its allies were starting to come to grips with the cyber dimension of the challenge - and giving much less priority to the crucial problem of poor governance in the countries whose dysfunctionalities provided opportunities for the radicals to take root.
But there were other parts of IS' complicated root system that were even harder to get at.
The New York-based security consultancy Soufan Group estimated that more than half of the 27,000 foreign jihadists making their way to Syria and Iraq in the first 18 months of the "Islamic State" were from the Middle East and North Africa.
Clearly, the "Caliphate" had appeal, despite - perhaps in some cases, because of - its graphically publicised brutality. While vowing to degrade and destroy the organisation, President Obama put his finger on the real challenge:
"Ideologies are not defeated with guns, they are defeated by better ideas, a more attractive and more compelling vision," he said.
The Complex Art Of IS Propaganda
On a relatively normal day... there was a total of 50 distinct pieces of propaganda. The photo reports and videos included depictions of an IS offensive in northern Syria and eulogies for the dead in Salahuddin... Overwhelmingly, though, the propagandists were preoccupied with a carefully refined view of 'normal' life.
Fishing and Ultra Violence
The problem is that when disenchanted people in the region look around them - especially the young, the idealistic, or the hopeless unemployed who have no future - they see scant evidence of "better ideas" or attractive and compelling visions.
They see the ruins of an "Arab Spring" which raised hopes only to dash them cruelly.
The brutal, corrupt dictatorships which it shook have either fragmented into chaos and sectarian and tribal upheaval, like Syria itself, Libya, Yemen and (with Western intervention) Iraq, or the sinister "deep states" of their former regimes came back, even more harshly in the case of Egypt, less so in the case of Tunisia.
While many of the European jihadists may have heeded the call for other reasons, socio-economic factors play an important part in radicalising some of the Arab jihadists, and will continue to do so unless addressed.
Mohammed is giddy with excitement as he films the discovery of three IS fighters on his mobile phone.
"Shoot him," he shouts, gesturing at one of them.
His nervousness gives him away as the unit's cook. He's unarmed, but his fellow soldiers from the Iraqi police Special Forces - known as the Emergency Response Division - are armed to the teeth and are not taking any chances.
The Secret Lives Of Young IS Fighters
One of the biggest contingents is from Tunisia, where a detailed survey in the poorest suburbs of the capital Tunis showed clearly that the radicalisation of young people there had far less to do with extreme Islamic ideology as such than it did with unemployment, marginalisation and disillusion after a revolution into which they threw themselves, but which gave them nothing, and left them hopeless.
IS is also filling a desert left by the collapse of all the political ideologies that have stirred Arab idealists over the decades. Many used to travel to the Soviet Union for training and tertiary education, but communism is now seen as a busted flush. Arab socialism and Arab nationalism, which caused such excitement in the 1950s and 1960s, mutated into brutal, corrupt "republics" where sons were groomed to inherit power from their fathers.
In this vacuum, IS took up the cause of punishing the West and other outsiders for their actions in the region over the past century:
the carve-up by the colonial powers 100 years ago, drawing a border between Iraq and Syria which IS has now erased
the creation of Israel under the British mandate for Palestine, and its subsequent unswerving political and financial support by the US
Western (and indeed Russian) backing for corrupt and tyrannical Arab regimes
the Western invasion and destruction of Iraq on the flimsiest of pretexts, with the death of uncounted thousands of Iraqis
the Abu Ghraib and Guantanamo prisoner abuse scandals....
The roots of IS also lie in a crisis within Islam.
"Isil is not Islamic," said President Obama, echoing statements by many Western leaders that "IS has nothing to do with Islam".
But it has.
"It is based on Islamic texts that are reinterpreted according to how they see it," says Ahmad Moussalli, professor of politics at the American University of Beirut. "I don't say they are not coming out of Islamic tradition that would be denying facts. But their interpretation is unusual, literal sometimes, very much like the Wahhabis."
Hisham al-Hashemi, the Iraqi expert on radical groups, agrees.
"Violent extremism in IS and the Salafist jihadist groups is justified, indeed blessed, in Islamic law texts relied on by IS and the extremist groups. It's a crisis of religious discourse, not of a barbaric group. Breaking up the religious discourse and setting it on the right course is more important by far than suppressing the extremist groups militarily."
Because ancient texts can be interpreted by extremists to cover their worst outrages does not implicate the entire religion, any more than Christianity is defined by the Inquisition, where burning at the stake was a stock penalty.
Extremist ideas remain in the dark, forgotten corners of history unless their time comes. And IS time came, with Afghanistan, Iraq, and everything that followed.
"Salafism is spreading in the world, in Afghanistan, Pakistan, the Arab countries," says Prof Moussalli.
He blames the Saudis for stifling the emergence of a moderate, democratic version of Islam, the "alternative Islamic discourse" to Salafism that President Obama wanted to see.
"A moderate Islamic narrative today is a Muslim Brotherhood narrative, which has been destroyed by the Gulf states supporting the military coup in Egypt," says Prof Moussalli, referring to the Egyptian military's ousting of the elected President Mohammed Morsi, a senior Muslim Brotherhood figure, in July 2013.
"We lost that opportunity with Egypt. Egypt could have paved the way for real change in the area. But Saudi Arabia stood against it, in a very malicious way, and destroyed the possibility of changing the Arab regimes into more democratic regimes that accept the transfer of power peacefully. They don't want it."
Saudi Arabia's ultra-conservative Wahhabi religious establishment and its constant propagation have raised ambiguity over its relations with radical groups abroad. Enemies and critics have accused it of producing the virulent strain of Wahhabism that inspires the extremists and even of supporting IS and other ultra-Salafist groups.
"We are at war with IS, which sees us as corrupt Wahhabis." he says.
"IS is a form of Wahhabism that has been suppressed here since the 1930s. It resurfaced with the siege of the Grand Mosque in Mecca in 1979 and spread here and there. But Saudi Arabia didn't back it at all, it saw it as a threat. So it's true that salafism can turn radical, just as the US right-wing produces some crazy lunatics."
Hundreds of people died in a two-week siege when extremist salafists took over the Grand Mosque, the holiest place in Islam, in protest at what they saw as the Kingdom's deviation from the true path.
More recently, Saudi Arabia's security forces and its Shia minority have in fact been the target of attacks by IS, and the kingdom has executed captured militants. It has an active deradicalisation programme.
But the Kingdom's extreme conservatism, its distaste for democracy, and its custodianship of the shrines in Mecca and Medina to which millions of Muslims make pilgrimage every year, have made it one of the main targets for calls for a hard-to-achieve reformation within Islam as part of the battle to defeat IS and other extremist groups.
The winds of change do now seem to be starting to blow through the desert kingdom, with cautious reforms behind which the young Crown Prince, Muhammad Bin Salman, is seen as the driving force.
Although the 2017 decree allowing women to drive caused a stir, it remains to be seen how deep such changes can go in a country where the grip of the ultra-conservative clergy is still strong, and where official primary schoolbooks still teach that all religions apart from Islam are "false".
Changing some of the fundamental interpretations of religion is a monumental undertaking that could take generations. But it cannot come fast enough for a senior Sunni politician in Iraq.
"We must accept the fact that Islam has a crisis," he says. "IS is not a freak. Look at the roots, the people, the aims. If you don't deal with the roots, the situation will be much more dangerous. The world has to get rid of IS, but needs a new deal: reformation, in Saudi Arabia, Afghanistan, al-Azhar [the ancient seat of Sunni Islamic learning and authority in Cairo]."
"You can't kill all the Muslims. You need an Islamic reformation. But Saudi and Qatari money has been blotting out the voices so we can't get anywhere. It's the curse of the Arab world - too much oil, too much money."
Sunni Axis v Shia Crescent
The demise of the IS as a "state" straddling Iraq and Syria has huge geo-strategic implications, few of them positive for the US and its regional allies.
While President Trump railed and tweeted against the evil of Iran and its sponsorship of terrorism, Tehran was quietly benefitting from everything the Americans were doing against IS.
They were non-speaking, arms-length partners in the campaign against the militants in Iraq. US air power, training and hardware directly helped the official Iraqi forces. But the Iranian-backed Shia militias played their part in a series of campaigns ending in a victory which greatly expanded their presence. The rise and fall of the IS phenomenon in Iraq left Iran's influence there stronger than ever.
While the distance between the American and Iranian roles in Syria was even bigger, with Iran on the side of the regime and the US originally backing the rebels, both were fighting IS, and its defeat would ultimately benefit the Assad regime and its Iranian backers. The Americans and their Coalition allies will not be around forever. Iran and its local allies will.
The US had little choice but to do what it did. The superpower could not allow itself to be goaded and provoked with impunity.
But it had earlier opened the floodgates to Iranian influence through a war that was purely elective, a war of choice and ambition rather than necessity: the invasion of Iraq in 2003, launched on pretexts (Saddam Hussein's possession of weapons of mass destruction and his support for terrorism) that turned out to be totally spurious.
By dismantling the Iraqi state in 2003, the US-led coalition was breaking down the bulwark that had contained Iran, the region's Shia superpower, seen as a threat by the Saudis and most of their Sunni Gulf partners since the Islamic Revolution in 1979.
The empowerment of the majority Shia community in Iraq after 2003 gave Iran unrivalled influence over Iraqi politics. For years, it had been backing anti-Saddam Iraqi Shia factions in exile. Now they came home to rule the roost.
The arrival of the IS threat led to even more Iranian influence, through the arming, training and directing of the burgeoning Iraqi Shia militias.
For Saudi Arabia and its allies, Iranian penetration in Iraq threatens to establish, indeed largely has, a Shia crescent linking Iran, Iraq, Syria under its minority Alawite leadership, and Lebanon dominated by the Iranian-created Shia faction, Hezbollah.
From the outset of the war in Syria in 2011, the regional Sunni powers - the Saudis and their Gulf partners, and Turkey - backed the Sunni rebels in the hope that the overthrow of Assad would establish Sunni majority rule.
A north-south Sunni axis running from Turkey through Syria to Jordan and Saudi Arabia would split the Shia crescent and foil the Iranian project, as they saw it.
That is essentially what IS did in 2014, when it moved back into Iraq, took Mosul and virtually all the country's Sunni areas, and established a Sunni entity that straddled the suddenly irrelevant border with Syria, blocking off Shia parts of Iraq from Syria.
Now that Sunni stake through the heart of the Shia crescent has been removed by the military defeat of IS and the failure of the Saudis, Turks and others - all American allies - to bring about regime change in Damascus.
It's intriguing to wonder what would have happened had IS halted its advances and provocations after the seizure of Mosul in 2014, toned down its rhetoric, perhaps made some local compromises and accommodations.
Chart showing estimated distribution of Sunni Muslims in the Middle East
Just after the fall of Mosul, I asked the Iraqi Kurdistan President, Masoud Barzani, whether he could live with his new neighbours.
"You don't choose your neighbours," was his reply. But five weeks later, IS attacked and overran his areas, threatening the Kurdistan capital Irbil, and inflicting genocide on the Yazidis.
"Had they not become international terrorists and stayed local terrorists, they could have served the original agenda of dividing the Arab east so there would be no Shia crescent," says Prof Moussalli.
We may never fully understand why IS did it. Perhaps its virulent strain of Salafism just had to keep pushing outwards: "Remaining and Expanding".
The Islamic State in its original form has been smashed. But for all the reasons that brought it about, IS and everything it represented will still be around in one form or another for a very long time to come.
Author: Jim Muir
Editor: Raffi Berg
Production: Ben Milne, Susannah Stevens