Twenty years ago, the U.S. invaded Afghanistan to get rid of the Taliban. Today, American forces, battered by one of the bloodiest attacks of the war, are relying for their own security on that same group, whose members they were trying to kill just weeks earlier.

Fighters of the Taliban’s elite Badri 313 unit, dressed in the latest tactical gear, patrol the same Kabul airport parking lot as U.S. Marines, separated by a few coils of razor wire. Farther away, Taliban foot soldiers pat down Afghans seeking to enter the facility and disperse crowds with whips and occasional gunfire in the air.

The Taliban’s key mission around the airport in the final days of the chaotic withdrawal is to hold off Islamic State, an even more radical organization, which killed 13 U.S. troops and nearly 200 Afghans in a suicide bombing on Thursday.

In this arrangement, the 5,200 American forces in Afghanistan “use the Taliban as a tool to protect us as much as possible,” Gen. Frank McKenzie, head of the U.S. Central Command, said after Thursday’s attack. The Taliban and the U.S., he added, now share a “common purpose.”

More than that: The Taliban checkpoints on the way to the airport—in coordination with the U.S.—are screening Afghans whose past work with Western forces puts them in danger of Taliban retribution. The Taliban are “providing the outer security cordon” for American forces, Gen. McKenzie said, and have closed some roads at U.S. request, extending the checkpoints’ perimeter.

To many Americans who had been involved in the 20-year war against the Taliban, one that resulted in some 2,465 American military deaths, this new reality in Kabul is nothing short of mind-boggling.

“The U.S. mission in Afghanistan is ending in the worst possible way imaginable,” said Lisa Curtis, an Afghanistan expert who served in the State Department, the Central Intelligence Agency and the National Security Council under the Bill Clinton, George W. Bush and Donald Trump administrations. The way the withdrawal is taking place “has resulted in tremendous costs to U.S. national security for a long time to come,” she said.

Taliban militants at the entrance of Afghanistan's international airport in Kabul on Aug. 17, two days after the Taliban swept into the capital.

Photo: Kyodo News via Getty Images

The Taliban, of course, have little love for the American forces who hunted their leaders with drone and air strikes and imprisoned many in Guantanamo Bay or Bagram air base detention facilities. Yet now that they are in power, the Taliban seek international recognition for their administration, renewed access to the global financial system and potentially foreign aid.

For Washington, there are good reasons to keep up a dialogue with the Taliban even after the last American troops fly out of Kabul on Aug. 31. Counterterrorism operations against Islamic State are one priority. Another stems from the rushed evacuation mission that is leaving behind thousands of Western citizens and many more Afghans who face Taliban wrath.

Though roughly 110,000 people have been flown out since Aug.14, that includes only a fraction of the Afghans who fear Taliban persecution because they helped the U.S. and its NATO allies.

“There are a lot of people who worked with and supported the U.S. government who are not going to get out when this airlift is completed, and it will require some pragmatic transactional cooperation with the Taliban in order to facilitate getting the rest out,” said Laurel Miller, who served as acting U.S. Special Representative for Afghanistan and Pakistan under former presidents Barack Obama and Donald Trump and is now Asia program director at the International Crisis Group, a conflict-resolution organization.

Mullah Abdul Ghani Baradar, leader of the Taliban delegation, signed an agreement with U.S. envoy Zalmay Khalilzad at a ceremony in Doha, Qatar, on Feb. 29, 2020.

Photo: ibraheem al omari/Reuters

Taliban leaders and senior U.S. government officials became familiar with each other during negotiations ahead of the February 2020 agreement in Doha, Qatar, that committed the U.S. to withdraw its forces. Some Trump administration officials held the view no peace settlement was possible without Taliban buy-in, so they engaged with the militants in the hope they would agree to a power-sharing deal. Close contacts have continued since then, with that coordination enabling the current arrangements around the airport.

In previous conflicts, such as the Soviet occupation of Afghanistan or the American war in Vietnam, years passed between the withdrawal of foreign forces and the fall of capital cities to insurgents. Washington was expecting a similar scenario as President Biden in April ordered the full withdrawal by September. As recently as June, the direst U.S. intelligence assessments predicted Kabul would fall six to 12 months after the last American forces left.

Instead, the stunning collapse of the Afghan republic on Aug.15 swept the Taliban into power while U.S. forces remained on the ground in numbers so small they were dependent on the insurgents’ good graces.

Overnight, Taliban troops who had been targeted by U.S. airstrikes the previous week turned into the external layer of protection for American troops in the Kabul airport. These Taliban fighters managed to foil previous Islamic State attack plans, Gen. McKenzie said, but weren’t able to prevent Thursday’s tragedy.

The Taliban have fought Islamic State since 2015, and they gunned down its senior leader, Abu Omar al Khorasani, hours after they seized Kabul, along with eight other jailed Islamic State militants. Now, the terrorist group, whose branch in Afghanistan is called Islamic State-Khorasan, or ISIS-K, is using the latest events to paint the Taliban as treasonous stooges of the infidels.

“It is to be noted that, for more than a week, the American forces, in partnership with the Taliban militia, have evacuated hundreds of foreign staff, translators and spies who worked for the U.S. military over the past years,” Islamic State said in a statement that claimed responsibility for Thursday’s bombing.

Sensitive to such accusations, the Taliban say they aren’t cooperating with American troops—and deny reports of a recent meeting between their leaders and CIA Director William J. Burns.

“Just because we have an agreement not to attack the Americans until they complete their pullout doesn’t mean that we have cooperation with them or provide security for them,” said Habibi Samangani, a member of the Taliban administration in Kabul. “In the airport, they are in one part, we are in another.”

U.S. troops stood guard behind barbed wire as Afghans sat on a roadside near the military part of the airport in Kabul on Aug. 20 hoping to flee Afghanistan after the Taliban's takeover.

Photo: wakil kohsar/Agence France-Presse/Getty Images

The American forces’ vulnerable position and reliance on Taliban goodwill meant the U.S. could not extend its Aug. 31 deadline for evacuations without Taliban assent, which wasn’t forthcoming. The Taliban could disrupt later flight operations with a few rocket strikes against the airport, and restoring those operations without their cooperation would require the deployment of a much larger U.S. force.

The Taliban established checkpoints around the airport the night of Aug.15, using force to disperse thousands of Afghans who crowded the civilian terminal, hoping for passage across the tarmac to the U.S.-controlled military side.

Those checkpoints mostly prevented incidents such as the one on Aug.16 when scores of Afghans held on to the landing gear of a U.S. C-17 military aircraft at takeoff, some falling to their deaths. Yet, this deployment also made it nearly impossible for Afghans at risk to reach evacuation flights, with local U.S. embassy staffers beaten and humiliated by Taliban guards.

“Following the traumatic experience, several of the local staff expressed distrust in the U.S. government and a sense of betrayal,” said a U.S. embassy cable written diplomatic cable written by an American diplomat in Kabul on Aug. 20.

“It would be better to die under the Taliban’s bullet than face the crowds again,” the cable said, citing an Afghan employee of the embassy. “‘We are human beings and considered as animals and abused…happy to die here but with dignity and pride.’ Local staff shared their observations reluctantly, citing fear that they would be punished or have their visas revoked.”

Behind the scenes, as the U.S.-led evacuation effort came to rely heavily on Taliban support, U.S. and other Western officials and activists pleaded with Taliban leaders to open secret cordons through Kabul so convoys of buses could carry Afghans to the airport. Taliban gunmen rode on many of these buses, expediting evacuations.

In the aftermath of Thursday’s deadly attack, efforts to get vulnerable Afghans and Westerners out of Kabul reached a fever pitch, just as the opportunities for leaving narrowed dramatically.

Hundreds of people gathered near an evacuation-control checkpoint on the perimeter of the airport in Kabul on Aug. 27.

Photo: Khwaja Tawfiq Sediqi/Associated Press

After the blast around 6 p.m. local time, U.S. embassy staffers were taken south of the airport runway to a compound where they hunkered down and awaited updates, according to a U.S. official. Word reached them that there were Marine casualties, but nobody knew how many. About two hours later, the U.S. chief of mission, Ross Wilson, shared the grim news.

Shortly before 10 p.m. local time, diplomats on the night shift left the compound and returned to the airport gates to resume evacuations, the official said. The mission would go on. By then, however, most of America’s allies had stopped their evacuation flights. Thousands of Afghans who had tried and failed to get out began to panic.

A convoy of 11 buses chartered by NATO, carrying some 400 local staff, had begun assembling on Kabul’s airport road just as an Islamic State suicide bomber detonated his vest roughly a mile away. After waiting through the night, the buses headed toward another entrance but were stopped by a new Taliban checkpoint, established after the attack.

The Taliban guards had orders: Only passengers with foreign passports could be allowed through. Two Afghans with dual citizenship were escorted to safety. Others dispersed, afraid to take chances.

One of the Afghan men aboard kept calling contacts inside the airport. He eventually received a special affidavit allowing entry and was directed to walk to a secondary gate in the back of the airport, where U.S. soldiers let him in. On Friday night, he was awaiting processing for his evacuation flight, likely one of the last before the U.S. mission in Afghanistan ends.

Najib, a 33-year-old Afghan man who has worked for the German military in the northern Afghan city of Mazar-e-Sharif since 2013, wasn’t so lucky. He said he was told by the German government to head to the Kabul airport on Aug.19, alongside 50 other families selected for evacuation. Together with their spouses and children, these staffers camped all night at the gate, but no German official contacted them, he said.

They were summoned again several times, gathering outside the airport’s Abbey Gate for the final time Wednesday. They carried German flags so German troops could recognize them.

“We spent all the night there, and they said they will come at 5 a.m. We were waiting. Then they said 7 a.m. We were waiting. Up until 10 a.m., we were waiting. But nobody came,” said Najib, who was at the site with his wife and three small children, one of whom got sick with fever and diarrhea that night. “We were very tired and left the area.” he said.

With only a small group remaining in the area, German troops came out and picked up a handful of Najib’s colleagues. Some were expelled from the airport later because the children didn’t have appropriate paperwork, he said.

On Thursday, hours before Islamic State’s bomb ripped through crowds at the airport’s Abbey Gate, the German call center coordinating the evacuation advised them to stay home and await another call, according to Najib. Since then, the call-center lines have gone dead, he said.

Berlin announced Thursday it has concluded the air evacuation mission even though, according to German officials, hundreds of citizens and as many as 10,000 cleared Afghans remain under Taliban rule. Chancellor Angela Merkel told parliament Wednesday that Germany will try to continue its efforts to get these people to safety after the end of the airlift.

“I’m not sure if they will ever be able to get us out,” Najib said. “When we were working with them, they were very close with us, they always said you are special, you are unique. But unfortunately they were just using us. When they finished the job, they just left us behind, in danger.”

—Dion Nissenbaum, Ehsanullah Amiri and Zamir Saar contributed to this article.

U.S. soldiers escorted evacuees to buses near Hamid Karzai airport in Kabul on Aug. 25.

Photo: us marines/Reuters

Write to Yaroslav Trofimov at yaroslav.trofimov@wsj.com and Vivian Salama at vivian.salama@wsj.com