The latest talks in Baghdad over Iran's nuclear program have prompted the usual arguments. Iran says it has only peaceful intentions. Israeli leaders scoff at that claim. Other world powers are unsure of Iran's intentions and demand that it take steps to show that it is not pursuing nuclear weapons.
The CIA and other U.S. intelligence agencies, meanwhile, are sticking with the assessment they made in November 2007, when they reported that Iran "halted its nuclear weapons program" in 2003 and apparently had not restarted it.
The CIA faced criticism 10 years ago for reporting, incorrectly, that Saddam Hussein's Iraq had weapons of mass destruction (WMD), and the caution it has shown since its 2007 estimate on Iran reflects lessons learned from its Iraq experience.
"Because we're so focused on the issues about the shortfalls of the Iraq WMD, we need to be very open about the nature of the evidence [on Iran], how much evidence, [and] the role of our assumptions," says Peter Clement, the CIA's deputy director of intelligence for analytic programs.
"Those [issues] factored into that estimate, to say we just haven't got what it takes to make the call that, yes, [the Iranians] have made a decision to go ahead with the [nuclear weapons] program."
Clement and other CIA officials discussed Iran and other issues with NPR in a rare on-the-record session at the agency's headquarters in Langley, Va.
The CIA's guarded statements on Iran have drawn criticism from conservatives.
Former U.N. Ambassador John Bolton said the 2007 estimate was "famously distorted," and his former chief of staff, Fred Fleitz, a former CIA officer, said U.S. intelligence agencies are "unwilling to conduct a proper assessment of the Iran nuclear issue."
But erroneous intelligence judgments can have costly consequences, and CIA officials insist their cautionary approach is well advised.
Criticism Over Iraq
In 2005, then-CIA Director Porter Goss, prompted by concerns that the United States and its allies went to war in Iraq on the basis of bad intelligence, ordered an internal review of its prewar Iraq reporting.
The conclusion: There were good reasons to believe Saddam Hussein had weapons of mass destruction. He had them previously, and had used them against his own people. He had kicked U.N. weapons inspectors out of the country, and some CIA informants were reporting that he still possessed them.
CIA leaders, however, concluded that the agency had erred in not highlighting for policymakers the extent to which its conclusions were based on assumptions rather than hard evidence, and in not making clear when the agency had depended on informants whose reliability was untested.
"Even if we had a great source who was a senior official in Iraq," Clement says, "and he's telling us what he believes to be the truth, you could easily be misled."
Even intelligence reports from intercepted telephone conversations can be wrong.
"It's worthy of close analysis," Clement says. "Just because someone said something to a friend doesn't make it true."
'Obsessed About Sourcing'
The 2005 Iraq review prompted the CIA management to institute a series of changes in its approach to analysis and in the way it reported its findings.
"Writing on any controversial issue right now, I think we're very — 'obsessed' may be too strong a word, but not entirely," says Clement, a 35-year CIA veteran who has overseen much of the analytic reforms undertaken at the agency.
"I personally get very obsessed about sourcing. If we make a statement of fact, I want it to be clear where that fact came from. If we got it from a [human] source, how good is the source? Did they have firsthand access? Second-hand access? Are they someone we recently got on the payroll?" he says.
Clement believes if CIA analysts had included those observations in their reports on Iraq WMD concerns, it might have made a difference, even if the overall conclusion had been the same.
"If they had written it with all the appropriate caveats," he says, "perhaps the readers on Capitol Hill or down at the White House would have said, 'You know, reading this, I'm not getting a real strong feeling about how well-founded this estimate is.' And the answer to that would [have been], 'You're correct. It is not well-founded.' "
The post-Iraq changes at the CIA also involve new analytic techniques, highlighted in a "tradecraft primer" in use at the agency since 2009. The manual is now used at the Sherman Kent School, the agency's in-house training institute for new analysts. The manual opens with a section on the "mind-set" challenge.
"If you're only looking at [an issue] through one narrow view of the world, you're not looking at the whole picture," says John, who teaches at the Kent School. He and other CIA analysts were willing to be quoted only if their last names were not revealed.
"Your biases will get you things like a confirmation bias: 'I've seen it before, so it must be happening again.' Or an anchoring bias: 'We've come up with that conclusion, and I think it's true, and it's not going to change.'"
One exercise now in use at the CIA is called "Analysis of Competing Hypotheses." Analysts who may be inclined toward one explanation for some notable development are forced to consider alternative explanations and to tally up all the evidence that is inconsistent with their favored hypothesis.
"You're looking for the hypothesis with the least inconsistencies," says John, who's been at the CIA for 34 years. "We call it the Last Man Standing approach."
Dealing With Uncertainty
Such exercises are employed throughout the CIA's Directorate of Intelligence. Each office now includes a "tradecraft cell," staffed by specialists whose mission it is to make sure their colleagues are using all the latest analytic techniques and challenging their own judgments.
"We want to try to get analysts to be honest about the uncertainties they face or the gaps in their information," says Mark, a tradecraft specialist in the CIA's East Asia office.
With all the checks and balances now incorporated into CIA analysis, it is not surprising that the agency has been reluctant to go out on a limb with its judgment on Iran's nuclear designs.
"We have used many of these tools on the Iran program," says Clement.
The cautions, in fact, are reflected throughout the agency's intelligence work, with analysts more reluctant these days to "make a call" on an issue by offering a prediction or a sweeping judgment.
Suggesting Alternative Outcomes
In the aftermath of the Sept. 11 attacks, the agency established a "red cell" with the mission of suggesting alternative visions of the future.
"There definitely was an emphasis in years past to say, 'It is most likely going to go this way,' " says Maria, the current chief of the CIA red cell. "We still have to make those calls, but now we try to explain what factors would take it in a different direction. Could the future be 180 degrees in an opposite direction?"
In one recent example, the CIA red cell offered five separate visions of how the Middle East might be different in 18 months, ranging from a "mosaic" scenario characterized by new democracies and reformist governments, to a region-wide Sunni-Shiite conflict shaped by a Saudi-Iranian competition for influence.
The reluctance to predict one future above others is another shift in the CIA's traditional approach.
"When I came onboard as a young analyst in the '70s and '80s," says Clement, "I distinctly remember managers telling me, 'Peter, the reason they pay you the big bucks is to make an analytic call. You know, you're the expert. You've studied all this material. Make the call. Give me the answer."
The Risks Of 'Making A Call'
As history shows, however, intelligence agencies can get in trouble when they're forced to "make a call" on an important issue, like whether Saddam Hussein had WMD.
Some former CIA officials have argued that the Bush administration bears some responsibility for the mistaken conclusion on Iraq. White House officials, including Vice President Dick Cheney, they claim, were so determined to go to war in Iraq that they pressured the agency to find a reason for invading that country.
Peter Clement does not share that view. "We appreciate interaction with the policymakers," he says. "Vice President Cheney was a serious consumer."
The new CIA approach does carry some risks. One danger is that the new exercises could incorporate so much caution into CIA analysis that the agency will be paralyzed. Analysts could become so focused on questioning their conclusions that they can't come up with answers.
Robert Jervis of Columbia University, who has written widely on intelligence failures and advised the CIA on its performance, is undecided on whether the new analytic techniques will prove worthwhile.
"I could see them becoming a substitute for careful, informed thought," Jervis says, "a formula [that leads to analysts saying], 'Oh, you don't need to know anything about the country. You just plug these things in.' "
"Misapplied, they would make things worse," Jervis says. "Applied well, they will make things better. How much better? Hard to tell."