The chaotic end to the war in Afghanistan shows that U.S. intelligence drastically underestimated how quickly Afghan forces would fold to the Taliban. But is this intelligence failure really that unexpected? The Afghanistan blunder represents the baseline performance of American intelligence, and this baseline has been failing since the end of the Cold War. While the jury is still out on how our foreign adversaries might fill the void in Afghanistan, a more immediate and widespread concern is that U.S. intelligence is sorely lacking and is losing to the competition, particularly China.
Core to America’s traditional model of intelligence is the primacy of secrecy—stealing and protecting secret information through clandestine or covert operations. This secrecy-driven intelligence model is behind the times. China is aware of this, and Washington will continue to cede ground to Beijing absent a fundamental reform of how intelligence serves policymakers.
Consider former CIA Director Stansfield Turner’s assessment after the Cold War that the agency’s greatest failure during his time was not foreseeing the rapid overthrow of Iran’s shah in 1979. The problem, according to Turner, wasn’t that the agency was ignorant of Ayatollah Ruhollah Khomeini’s secret scheming, but that it overlooked “the breadth and intensity of feeling against the shah inside Iran.” His proposal in retrospect was a pivot to “forecasting events driven by ground swells in public attitudes.”
You’d be right to recognize the similarity between Iran’s “10 days that shook the world” and the Taliban’s tremor today. American intelligence misjudged the “ground swells” in Afghan forces’ attitudes, even after training them for 20 years.
Multiple intelligence community leaders since Turner have advocated for more focus on public—or, open-source—information. The intelligence failure surrounding the September 11 attack and Iraq’s alleged weapons of mass destruction has also prompted attempts at such a reform in U.S. agencies. But open-source intelligence remains largely on the sidelines, even as technology makes useful information more readily available.
The myopic vision and inward focus of secrecy-driven intelligence is prone to miss what’s in plain sight in our information age. In a marketplace for knowledge that has been massively enlarged by the internet, not only are there fewer secrets left for good old spycraft, but invaluable intelligence can be uncovered relatively quickly by mining public information and engaging civilians.
Bellingcat, a citizen-journalist website, made its name by identifying Russia as the culprit behind the 2014 downing of a Malaysia Airlines flight using Google Earth. Satellite images helped the Australian Strategic Policy Institute locate Chinese factories that were using Uyghur forced labor. Thanks to amateur sleuths, law enforcement arrested January 6 Capitol attackers, the world learned that COVID-19 may have potentially originated from Wuhan’s virology lab, and criminal cold cases have been solved where police failed.
A foreign regime’s propaganda also contains mineable intelligence; their publicly available words, while clearly not secrets, are windows into their minds.
The WWII-era agency Foreign Broadcast Information Service, for example, successfully predicted the deployment of secret German weapons by analyzing Nazi propaganda. While similar government work has continued to this day, recent advances in computational methods have made this type of analysis more accessible to nongovernmental actors, too. My own open-source machine learning research project—the Policy Change Index—aims to predict the Chinese government’s actions by mulling over its domestic-facing gazette, and it’s one of many such examples. Chinese diplomats are also playing an impressive catch-up with spreading propaganda on social media, which opens another door for reading Beijing’s mind on foreign policy.
Traditional kinetic warfare is a straightforward military action that comes down to building and wielding armaments, where governments largely maintain a monopoly and secrecy can indeed be an advantage. Yet open-source intelligence reveals that the time when governments monopolized information “armaments” is over.
The insufficiency of secrecy-driven intelligence is pronounced in U.S.-China “information warfare.” Just think about the Chinese government’s repeated attempts to push a baseless claim that COVID-19 originated in the United States.
A recent report by Georgetown University’s Center for Security and Emerging Technology shows how open source is the “first resort” of Chinese intelligence, not an afterthought. Not only is China a surveillance state, but a host of civilian companies and social organizations are using public information to produce intelligence for the Chinese government, especially foreign technological advances.
Democratic governments don’t have the same control over public information that China’s does, but that doesn’t justify the rigidity of the U.S. intelligence model. Intensifying competition with China requires America’s intelligence apparatus to embrace open-source technologies and tap into the vibrant brain trust within our civil society. The State Department’s Global Engagement Center is an example of such public-private partnership, but efforts like that need to be the norm throughout government, not isolated projects.
Sherman Kent, whom the CIA considers the father of American intelligence analysis, defined intelligence as “the knowledge which our highly placed civilians and military men must have to safeguard the national welfare.” It’s past time for U.S. intelligence to return to its roots, embracing civilian insights and the wisdom of crowds, or we’ll continue to see blunders like what’s unfolding in Afghanistan.
Weifeng Zhong is a senior research fellow with the Mercatus Center at George Mason University and a core developer of the open-source Policy Change Index project, which uses machine-learning algorithms to predict authoritarian regimes’ major policy moves by “reading” their propaganda.
Please note that we at The Dispatch hold ourselves, our work, and our commenters to a higher standard than other places on the internet. We welcome comments that foster genuine debate or discussion—including comments critical of us or our work—but responses that include ad hominem attacks on fellow Dispatch members or are intended to stoke fear and anger may be moderated.
With your membership, you only have the ability to comment on The Morning Dispatch articles. Consider upgrading to join the conversation everywhere.