As the Islamic State comes apart in Syria and Iraq, around 5,600 foreign fighters of the “caliphate” have already returned to their home countries, according to a report issued by U.S. think tank The Soufan Center.

Those who have returned may not all continue the violent jihad, but they will all pose “varying levels of risk” for law enforcement and security services around the world, the report says.

Over 40,000 foreigners traveled to Iraq and Syria from over 110 different countries to join the Islamic State. Hundreds have already returned home to places like Russia, Saudi Arabia, France, the U.K., and Germany. Only a few hundred out of the thousands came from North America, with the group estimating that fewer than 129 fighters have come from the United States.

The report did say that those who had left to join ISIS were proving to be more manageable than originally anticipated if they returned home. “They generally appear to have had a stronger desire to join something new rather than destroy something old. As a result, returnees have, so far, proved a more manageable problem than initially anticipated.”