CIA’s Kent
School: A Step In the Right Direction
Stephen
Marrin
University
of Virginia
18
March 2002
International
Studies Association Conference
The
Central Intelligence Agency's (CIA's) Sherman Kent School for Intelligence
Analysis was created to increase the expertise of intelligence officers within
its Directorate of Intelligence (DI) responsible for the production of finished
intelligence analysis.[1]
In 1998 the Director of Central Intelligence (DCI) George Tenet proposed a
strategic plan to improve CIA’s capabilities. A task force responsible for
assessing ways to increase analytic depth and expertise recommended CIA’s
leadership promote training as a way to increase expertise, and with the
concurrence of the then-Deputy Director for Intelligence (DDI) John McLaughlin, in
2000 the Kent School was born.
The Kent School provides primarily analyst and
managerial training for the DI, although it also houses the Kent Center which
acquires and disseminates information regarding analytic ‘best practices.’ The
School is named after Sherman Kent who was “a leader in developing the
profession of intelligence analysis” and “did much to create the institutions
that evolved into the present day DI,” according to the CIA website.[2]
The School is aptly named, for according to DCI Tenet the Kent School “will
prepare generations of men and women for the…profession of intelligence
analysis in the 21st Century …(by teaching) the best of what we as an Agency
have learned about the craft of analysis.”[3]
The Kent School “enhance(s) the skills and expertise of analysts and managers
in the (DI)” by “offer(ing) four DI-core programs: the 26-week career analyst
training program in which all new analysts will be enrolled; the managing and
teaching analysis program; an intelligence managers seminar series; and an
academic outreach and intelligence analysis studies program,” according to a
CIA press release.[4] It also
provides training in CIA’s analytic disciplines to officers of all levels.
Evaluating the ability of the Kent School to achieve
its goals is necessary to determine what kinds of changes may be needed in the
future to improve CIA’s analytic capabilities. Assessing whether the School
provides the DI with greater expertise requires comparing the School’s programs
to those existing previously to determine whether the School is in fact an
improvement over past practices. In addition, assessing whether training
improves the quality of the analytic product requires comparing the specific content
of the training programs to the specific duties of an intelligence analyst or
manager. However, an evaluation of the Kent School’s impact on the DI’s output
is hindered by the School’s programmatic variety--analyst training, managerial
training, and the more intangible but no less important task of the Kent
Center—which all impact the DI’s expertise and the quality of analysis in
different ways. Therefore, to keep the project manageable this paper focuses
tightly on assessing the impact of the analytic training programs.
This assessment indicates that the Kent School has achieved its goal of increasing analytic expertise by noticeably improving the agency’s training for new analysts and providing sorely needed disciplinary training. The training also provides new analysts with the knowledge and skills that enable them to be more effective in the production of finished intelligence as well as overall job performance. It also provides new analysts with the means to produce more accurate analysis by teaching them the causes of--and means to avoid--intelligence failure, and cognitive tools to assist them in structuring their analysis more along the lines of the scientific method. In addition, the disciplinary training enables analysts to create more accurate analysis by providing them with greater grounding in political, economic, or military theory. In sum, the training provides analysts with the skills necessary to be more effective and produce higher quality and more accurate intelligence analysis.
However, whether CIA’s analytic output
actually improves in quality will depend on whether the analyst has the
opportunity to apply his or her newly acquired expertise when back on the job
after training. In the end it is the institutional assessments of analytic
production processes and not those of the Kent School’s programs alone that
will determine whether the analytic training programs allow the CIA to improve
its production of accurate, timely and tailored intelligence that fits the
needs of the CIA’s national security policymaking customers. In sum, the Kent
School is a step in the right direction but more steps may be necessary in
order to produce a measurable improvement in CIA’s analytic quality.
School
Created to Improve CIA’s Analysis
Just as every reform is intended to be
a solution to a particular problem, CIA’s leaders created the Kent School as a
way to address the agency’s analytic weaknesses. In May 1998, DCI Tenet “gave a
classified internal briefing on the CIA's problems and what he intend(ed) to do
about them,” according to an article in US News and World Report.[5]
In presenting this ‘Strategic Direction’ DCI Tenet “la(id) out his five-to
10-year strategic plan for U.S. intelligence agencies…(and) told his employees
he believed the agency’s espionage capabilities had eroded since the Cold War,”
according to Washington Post reporter Walter Pincus.[6] In
articulating his vision for the CIA, DCI Tenet emphasized the importance of
changing past practices to improve CIA’s contribution to national security
policymaking. As he noted in 2000:
“(When) I launched a Strategic Direction plan for
the CIA. ... I told our people that we had to take charge of our destiny. That
we would do all within our power to maintain our edge and our vibrancy. That we
had to streamline and realign ourselves and adopt business practices like the
best in the private sector. That we would think big and think different. That
we would work smarter and in new ways so that we would have the agility and the
resiliency to do what the President—this President or a future President—wants
and the American people expect.”[7]
One week after this speech,
CIA failed to warn American policymakers of India’s intentions to test nuclear
weapons. This failure highlighted the agency’s limitations and likely
accelerated the implementation of DCI Tenet’s Strategic Direction by providing
a renewed emphasis on better collection and analysis as well as giving a
political boost to the implementation of reforms.[8] Many factors
were cited in post hoc analyses as contributing to the failure, but of note was
the DI’s inability to add together all indications of a possible nuclear test
and warn top policymakers.[9]
This failure of analysis led to charges of “lack of critical thinking and
analytic rigor.”[10]
Admiral David Jeremiah--who headed the official investigation into the
failure—“recommended hiring more analysts, improving their training and
increasing contact with outside experts to challenge conventional wisdom,”
according to The Wall Street Journal’s Carla Robbins.[11] She also
reported that DCI Tenet said he would “mak(e) it my highest priority to
implement (Admiral Jeremiah’s recommendations) as quickly as possible.”[12]
DCI
Tenet subsequently commissioned a number of task forces to implement his
Strategic Direction. One of these task forces–entitled the ‘Analytic Depth and
Expertise Task Force’--was made up of approximately eight DI officers and given
free rein to investigate ways to “develop true world class experts” consistent
with the Strategic Direction’s goals, according to Kent School program director
Denis Stadther.[13] The officers met several times a week
through the latter half of 1998 brainstorming ideas to increase the expertise
of junior, mid-level, and senior analysts. The task force ruled out a training
course for mid-level officers because they believed that the broadening
experience provided by rotational assignments—especially overseas or to
policymaking institutions--would provide greatest value to the institution. The
task force also noted that the DI’s career paths provided greater advancement
possibilities for managers over senior analysts and as a result the more
ambitious analysts shifted to management rather than using their knowledge in
an analytic capacity. Therefore, rather than provide training to senior
analysts the task force proposed the creation of a position whereby senior
analysts could continue to use their expertise rather than shift into
management. Finally, they thought that junior officers would be best served
with a training program intended to ‘build a common foundation upon which
analytic expertise could be built,’ according to Denis Stadther.
The task force submitted two recommendations to
then-DDI McLaughlin who approved both and subsequently attributed their
implementation to DCI Tenet’s support and tenure that “lasted longer than the
‘task force phase,’” according to Vernon Loeb.[14] The program
for senior analysts took form as the Senior Analytic Service--“a career track
(enabling) analysts to rise to very senior rank without branching out into
management” thereby “reward(ing) world-class expertise in topics of key concern
to the US Government,” according to the CIA’s website.[15] The training
program for new analysts took shape as the Kent School whose impending creation
DCI Tenet announced at the annual symposium of the Association of Former
Intelligence Officers (AFIO)—the primary national level intelligence interest
group--in October 1999.[16]
In May 2000, DCI Tenet dedicated the Kent School and the first classes
began.
School’s
Goal: Increasing Analyst Expertise
The Kent School was created to improve CIA’s
analytic quality in part by increasing the expertise of its analysts. According
to Martin Petersen, director of CIA’s human resources department, the Kent
School’s creation “sends a very, very powerful message about what you value,
and the value here …is analytic expertise.”[17]
The Kent School improves CIA’s analytic production by providing analysts
with knowledge that they otherwise would acquire haphazardly or not at all in
on-the-job training.
Expertise—“the skill of an expert” according to
Merriam-Webster’s collegiate dictionary--is a crucial component for the
production of accurate intelligence analysis.[18]
“In the lexicon of US intelligence professionals, ‘analysis’ refers to the
interpretation by experts of unevaluated (‘raw’) information collected by the
intelligence community” according to intelligence scholar Loch Johnson.[19]
Therefore, increased analyst expertise should produce better interpretation
resulting in a higher quality product. DCI Tenet described the importance of an
analyst’s expertise at the Kent School’s dedication:
“In our (DI) it is not enough just to make the right
call. That takes luck. You have to make the right call for the right reasons.
That takes expertise. It is expertise—built up through study and
experience—that combines with relevance and rigor to produce something that is
very important: insight….(O)ur analysts blend a scholar’s mastery of detail
with a reporter’s sense of urgency and clarity. At its best, the result is
insight. And it is insight that wins the confidence of our customers and makes
them want to read our publications and listen to our briefings.”[20]
The DI’s exploitation of analytic expertise in its
production of finished intelligence is not simple, however. CIA uses a complex
web of analytic specialties to produce multi-disciplinary analysis. Not hired
by the CIA or even the DI, most analysts are hired by the individual DI
offices, assigned to ‘groups’ which cover specific geographic areas, and then
assigned a functional specialty—‘discipline’ or ‘occupation’ in DI
terminology—such as political, military, economic, leadership, or scientific,
technical, and weapons intelligence, according to CIA’s website.[21]
For the most part each CIA analyst possesses a very small area of direct
responsibility defined by a combination of their regional area and discipline,
work in country ‘teams’ with analysts of other disciplines, and interact with
other regional or disciplinary specialists as the need arises. Therefore, an
analyst’s expertise can vary depending on the relative possession of regional
knowledge, disciplinary theory, and intelligence methods in general:
CIA’s small analytic niches creates specialists, but
their specialties must be re-integrated in order to provide high-level
policymakers with a bigger picture that is more accurate and balanced than the
limited perspective or knowledge of the niche analyst. This process of re-integration
includes a procedure known as ‘coordination’ in DI parlance which allows
analysts of all kinds to weigh in with their niche expertise on pieces of
finished intelligence before they are disseminated. According to CIA analyst
Frank Watanabe: “We coordinate to ensure a corporate product and to bring the
substantive expertise of others to bear.”[22]
Accordingly, the bureaucratic norm is that an analyst will make every effort to
coordinate a draft with other analysts in related regional or disciplinary
accounts prior to submitting the draft to management for editing and
dissemination. As a result, while the expertise of the primary drafter of the
piece of intelligence analysis usually has primary influence on the accuracy of
the final piece, the coordination process exerts a strong influence as well.
Inaccuracies in the analytic product can occur as a
result of insufficient substantive and disciplinary expertise. Columbia
University Professor Robert Jervis points out that “a grav(e) danger lies in
not having sufficient expertise about an area of a problem to detect and
interpret important trends and developments. To make up for such deficiency,
analysts tend to impose on the information the concepts, models, and beliefs
that they have derived elsewhere.”[23]
In addition, in 1991 former DCI Stansfield Turner noted that: “Another reason
for many of the analytic shortcomings is that our analytical agencies do not
have an adequate grasp of the cultures of many countries with which we must
deal.” He goes on to suggest that analysts could use a “a better opportunity to
attend academic institutions, participate in professional conferences, travel
and live abroad, acquire language skills and thus become true experts in their
areas.”[24] Robert Jervis adds ‘adequate training’ and
‘first-hand exposure to the country’ as additional methods to increase
expertise.[25]
In
sum, analyst expertise is an important component of analytic quality, but the
Kent School’s analytic training—even if ideal—could never increase expertise to
the point that perfection would be attainable. In addition, the CIA’s
inter-dependent analytic structure implies that training can only increase
aggregate capabilities through the cumulative effects of multiple kinds of
targeted training. Even so, that these factors limit the Kent School’s
potential to improve analyst quality does not preclude improvement on the
margins. If training can improve analytic quality it does so by increasing the
knowledge and skills at the analyst’s disposal. This increase in knowledge is
relative, though, and evaluating whether the Kent School increases analytic
expertise requires comparing its programs to those that it replaced.
Overcoming
Existing Deficiencies
The Kent School’s training programs provide
intelligence analysts with knowledge that they were not acquiring sufficiently
prior to its creation. Its training mission—to increase expertise--corresponds
with the efforts of many other professions and even other intelligence
organizations that train employees as part and parcel of organizational
improvement programs. Training programs in general provide information specific
to the needs of the institution, and different institutions within the
governmental foreign policy process use training programs to bolster their
unique informational needs:
· State Department’s Foreign Service Institute provides area and language studies helpful to outgoing State Department Foreign Service Officers and other government officials.
· The Department of Defense’s (DOD’s) command colleges—including the Army War College, Naval War College, and National Defense University among others--provide rising military officers with a more advanced conceptual and informational context within which to make command decisions.
CIA’s intelligence analysts require an understanding of their role within the foreign policy process, tools that help them structure and analyze complicated issues, a theoretical framework to approach an issue from a specific disciplinary perspective such as political or economic analysis, and the presentational skills necessary for busy policymakers to incorporate into their decisionmaking processes. Yet prior to the Kent School’s creation new analysts were not receiving the information they needed to do their jobs effectively, and on the whole the institution was failing to provide analysts with a disciplinary foundation substantial enough to ground sophisticated analysis upon.
The CIA’s training for new analysts was insufficient
to provide them with the information they needed to do their jobs well prior to
the creation of the Kent School. One might think that CIA’s central role in providing
high-level policymakers with analysis of national security threats and
opportunities would require stringent training for new analysts prior to
conducting analysis and writing reports for senior policymakers. In fact, CIA
analysts have been hired and assigned a desk at CIA headquarters without any
analytic training whatsoever, with two or four weeks of analytic training
coming six months or so later.[31]
For example, in 1996 the introduction to CIA for new analysts consisted of one
week of pro forma briefings regarding security and other administrative issues
but nothing specific to analysis. Within weeks, and in some cases days, these
newly hired analysts were producing finished intelligence—albeit limited and
under the careful watch of more senior analysts—without any official training
in substance or methods whatsoever.
This minimal formal training appears to have
historical roots in the CIA. According to former DDI and current Deputy
Director of Central Intelligence (DDCI) John McLaughlin: “Dick Lehman, the
creator of what we now call the President's Daily Brief, once remarked that the
basic analytic training he got back in 1949 came down to a single piece of
advice from his boss: “Whatever you do, just remember one thing—the Soviet
Union is up to no good!” Simple, but that said it.”[32]
In addition, Mark Lowenthal—a former staff director of the House Permanent
Select Committee on Intelligence—said that CIA does not “do a lot of training.
… They say, ‘Congratulations, you’re the Mali analyst, have a nice day.’”[33]
The training process usually relied upon the analyst’s prior formal education
combined with an initial period of sink-or-swim adaptation to the DI. The
sink-or-swim analogy is used frequently inside the CIA to describe its junior
analyst acculturation. In May 2001, the Kent School’s former dean likened
previous DI training to being thrown into the deep end of a pool, and added
that if the training or mentoring ‘missed,’ the analyst ‘floundered.’[34]
In
1996 the CIA improved its training for junior analysts with the creation of a
month-long survey course for new analysts entitled ‘Fundamentals of DI
Tradecraft’ (FDIT). Some of FDIT’s content was based on a tradecraft course
developed under the auspices of former DDI Doug MacEachin and delivered to all
analysts and managers.[35]
Nonetheless, although FDIT was intended only as a limited survey course, it
received a fair amount of criticism internally in its first few runnings due to
the slow pace and inconsistent quality of instruction.[36]
It also demonstrated that one month of training was too short to familiarize
analysts with the range of information that an intelligence analyst should know
to do his or her job correctly. However these weaknesses became strengths as
the criticisms may have provided the Analytic Depth and Expertise task force
with indicators that there was greater potential inherent in the new analyst
training than was being actualized through FDIT. In the end, the task force was
able to use the lessons derived from FDIT to inform the structure of the Kent
School and the content of its new analyst training program.
Dissipating
Disciplinary Expertise
Disciplinary—or occupational--training compensates
for the loss of expertise resulting from a reorganization the structured
offices by region rather than discipline. From approximately the early 1960s to
1981, the DI was structured primarily by discipline with political, economic,
military, and leadership offices each subdivided by geography. In 1981 “(a)
(DI)-wide reorganization …shuffled most analysts and created geographically
based, or "regional" offices out of the previous organization.”[37]
According to former CIA officer Arthur Hulnick: “These (new) offices combined political, economic, military, and
other kinds of research under “one roof,” thus making more detailed analytic
research feasible.”[38]
This integration of analytic disciplines in regional offices provided a more
accurate interplay of the forces at work within a country but because the
organizational structure provided the basis for constant contact and
communication it also negatively affected the DI’s ability to maintain
disciplinary knowledge.[39]
The 1994 dismantling of the DI’s Office of Leadership Analysis (LDA)
provides a good case study into the loss of disciplinary knowledge and need for
disciplinary training to compensate.
Since at least 1963 the CIA has consolidated
biographic information on foreign leaders, and over time the value of
leadership analysis as a complement to political, economic, and military
analysis became evident.[40]
Analysis grounded in the theory of political psychology had the potential to
provide insight into a leader's behavior above and beyond the limited
biographic scope of a biographic profile.[41]
LDA was thus created to develop and consolidate the discipline of leadership
analysis and put it on a near-equal basis with the other established analytic
disciplines. However, in 1994, then-DDI Doug MacEachin reorganized LDA--the
only disciplinary office left--out of existence as part of “downsizing the DI.”[42]
The actual physical co-location of leadership
analysts with their regional counterparts did not occur for months--and in some
cases years--due to lack of space in CIA’s overcrowded headquarters building.
As a result of being ‘out of sight, out of mind,’ leadership analysts at times
were not informed of ongoing projects, briefings, and meetings, and as a result
the DI published products which at times contained factual errors such as the
wrong names or positions for foreign government officials. Once physical co-location
and integration into regional teams occurred, the regional analysts began to
incorporate leadership contribution into their products more easily than had
occurred during the existence of LDA.
However, LDA’s elimination had an impact on the expertise of the DI’s
leadership analysts. The distribution of specialized expertise of
analytic tools, product formats, and disciplinary theory throughout the DI and
lack of disciplinary training meant that new leadership analysts were not well
trained in their discipline and relied solely on the fast dissipating knowledge
of the handful of former-LDA officers also assigned to their team or issue.
Therefore, the elimination of a leadership analysis-based office resulted in
both increased incorporation of leadership analysis and insight into the
regional teams’ products, and decreased corporate knowledge and expertise in
leadership analysis as a discipline. The same occurred for the other
disciplines when their respective offices were eliminated in 1981.
By the mid-1990s DI analysts were realizing that
while multi-disciplinary analysis on country teams made for better integration
of disciplines it also led to the dissipation of disciplinary knowledge. Former
LDA analysts’ efforts to sustain their hold on disciplinary knowledge triggered
similar efforts by political, economic and military analysts to both sustain
and reconstruct occupational-specific knowledge. In 1997 the DI created the
‘senior-level’ Council of Intelligence Occupations (CIOC) with the initial intent
of disciplinary “workforce strategic planning…in the areas of recruitment,
assignments, and training” as well as “identify(ing) core skills and standards
for expertise in the (DI).”[43]
In practice, CIOC became a home for senior analysts interested in learning and
teaching their discipline’s methodologies. They “establish(ed) a professional
development program for all DI employees that provides explicit proficiency
criteria at each level so that everyone can see the knowledge, skills, and
experiences required for advancement within each occupation or across
occupations.”[44] All this
was done—according to the CIA website--“so that the DI has the expertise needed
to provide value-added all-source analysis to its customers.”[45]
CIOC emphasized disciplinary training as a method to achieve this goal. As the
CIA’s website notes, “(t)he Council views learning and skills development as an
essential part of work and the means whereby employees can respond to changes
in customer priorities and the external environment.”[46]
In 2000, CIOC was disbanded and its training
programs were incorporated into the Kent School, according to Kent School Dean
Dan Wagner.[47] As a
result, the Kent School was able to rely on three years of accumulated CIOC
expertise on the role of the disciplines within the DI and this enabled it to
focus on reconstructing the disciplinary expertise that had previously been
lost due to organizational change.
Before the Kent School was created the DI lacked an
effective way to distribute disciplinary expertise to analysts, and the
one-month survey course for new analysts proved to be insufficient for its
needs. The Kent School fixed these problems by “fundamentally chang(ing) the
way that (CIA) …train(s),” according to Martin Petersen, and these changes
improved CIA’s provision of analytic expertise.[48]
Structurally the School was built on lessons learned
from prior attempts to provide training to the DI. One member of the Analytic
Depth and Expertise task force had been intimately involved in FDIT’s
development as a DI representative to the CIA’s Office of Training and
Education (OTE), and he contributed the lessons he learned from this to the
task force.[49] Once the
Analytic Depth and Expertise task force decided to recommend that the DDI
implement training for junior analysts, it began to prepare the outlines of a
proposal. Prior DI training programs run through OTE—located in CIA’s
Directorate of Administration--had difficulty obtaining qualified instructors
from the DI because of the prevalent belief that non-analytic assignments in
OTE were not career-enhancing. In addition, OTE’s separate structure led to
difficulties providing training tailored to the DI’s needs. As a result of
these problems, the task force recommended that training be run through the DI
so that it could provide better integration of training with the DI’s
requirements. They also suggested a school structure for the training programs
be created to ensure continuity and a knowledge base for future training
efforts. Finally, they suggested the School be headed by a Senior Intelligence
Service officer with a presence on the DI’s corporate board in order to provide
greater “bureaucratic weight and
budgetary authority” for the School to meet its goals, according to
Denis Stadther.[50] Then-DDI
John McLaughlin implemented all the suggestions, and as a result the Kent
School’s location within the DI and bureaucratic structure provide it with a
solid base from which to deliver its informational product.
The School’s training program for new analysts—known
as its Career Analyst Program (CAP)--is CIA's “first comprehensive training
program for professional intelligence analysts” according to a CIA press
release.[51][52]
Although the CAP used lessons derived from FDIT, it “was a clean-sheet exercise
in developing new training that was founded in the belief that comprehensive
training for new analysts must go beyond skills development to include history,
mission, and values of the DI as well as orientation to specialized knowledge
and skills like operations, intelligence collection, and counterintelligence
issues,” according Denis Stadther.[53]
Comprehensive training could not be shoe-horned into a one-month course akin to
FDIT, however, and the issue of CAP’s length became a matter of some dispute.
According to the School’s former dean, DCI Tenet had “chided the DI for not
taking training as seriously as the (CIA’s Directorate of Operations or
DO)”--which has a year-long program for trainees—and “instructed the DI to
create a program as long as the DO’s.”[54]
As a result, the Analytic Depth and Expertise task force initially proposed a
ten-month long training program. However this was opposed by the DI offices who
were reluctant to lose their new analysts for that long due to tight personnel
resource constraints. To accommodate their concerns the program was reduced
from ten months to 25 weeks and yet retained ‘all (its) training components’ by
providing analysts with fewer interim assignments to other components,
according to Stadther.[55]
He also emphasized that the CAP’s curriculum undergoes constant revision
derived from analyst and manager feedback in a search to make the program more
effective.
Between 2000 and 2002 the CAP’s length was reduced
to 22 weeks as the curriculum was refined due to student feedback and a
comprehensive review of the curriculum by the CAP faculty, according to
Stadther. In February 2002, the initial week entails an introduction to
intelligence topics including the history, mission and values of the CIA, and a
unit on the history and literature of intelligence that is taught by the Center
for the Study of Intelligence’s (CSI’s) history staff.[56]
The following few weeks introduce analysts to a variety of DI skills including
analytic thinking, writing and self-editing, briefing, data analysis techniques
and a teamwork exercise akin to an ‘outward bound’ for analysts, according to
Stadther. Training in these kinds of basic skills are necessary even for CIA
analysts with advanced degrees because intelligence analysis focuses on the
informational needs of policymakers who ask different questions than are
encountered in academia, according to Stadther. Also, briefing skills are not
emphasized in academia but are vital for the effective distribution of
information within government. In addition, analysts are not accustomed to
writing in the specialized writing style of the DI which emphasizes bottom-line
judgments at the beginning of the piece while strategically using evidence to
bolster the main conclusions. The training is reinforced throughout the CAP as
analysts write an intelligence paper in which they self-consciously apply
lessons learned to a real-world analytic problem.[57]
The first five weeks of training culminate in a task force exercise that
provides analysts with the opportunity to take the skills they have learned and
practiced in a classroom setting and apply them in a more realistic one.
After
five weeks in the classroom the CAP students go on a four-week interim
assignment that helps them understand how the DI relates to other components of
CIA, the intelligence community, and the policymaking agencies they serve. The
analysts—in consultation with their managers—identify positions within the
bureaucracy that are related to their assignment and provide working knowledge
of units that the analyst will either work with or rely upon throughout his or
her career.[58] In
particular, seeing how intelligence analysis is viewed and used in other
components of the policymaking bureaucracy can help new analysts understand their
role and perform it better, according to Denis Stadther. These interim
assignments can entail a stint at a military command—such as the Pacific
Command headquartered at Pearl Harbor—which provides the analyst with insight
into working with a military customer. These assignments can have other
benefits as well. In early 2001 DDCI John McLaughlin spoke enthusiastically of
the insight that a new analyst can acquire from visiting Pearl Harbor and
seeing first-hand the reason CIA was created and how he or she fits into a
50-year tradition of working to prevent similar destruction.[59]
After this four week interim assignment the analysts
return to the classroom for four more weeks of training in more advanced topics
such as writing and editing longer papers and topical modules that address
issues such as denial and deception and indicators and warning.[60]
According to Stadther these are special kinds of analysis that require
application of more advanced and sophisticated analytic tradecraft techniques.
The analysts also broaden their knowledge of other positions within the
intelligence community by visiting other intelligence agencies and
participating in an ‘ops familiarization course’ which provides a brief
exposure to the intense training that a DO case officer receives, according to
the School’s former Dean.[61]
The analysts also are provided with greater information on tradecraft
topics—consisting of concepts, techniques, and methodologies--that form the
core of an intelligence analyst’s specialized skill set which they can use to
rigorously and self-consciously analyze raw intelligence data. This skill set
includes adapting the study of intelligence issues to the needs of the user,
presenting complicated issues in simple terms stripped of long-winded assumptions,
qualifications and background, and addressing issues related to controversial
foreign policies and policy goals without taking a policy position.[62]
Specifically, the CAP uses Sherman Kent’s ‘Principles for Intelligence
Analysis’ as guidelines for its students to follow as they learn the craft of
analysis.[63] These
guidelines emphasize the importance of intellectual rigor, the conscious effort
to avoid analytical biases, a willingness to consider other judgments, and the
systemic use of outside experts as a check on in-house blinders as well as the
importance of candidly admitting analytic shortcomings and learning from
mistakes. The CAP also teaches other tradecraft skills such as alternative
analysis techniques including but not limited to scenario building and
competitive ‘A/B Team’ approaches that entail periodic checks on analysis by
external experts.[64][65]
In addition the CAP uses Richards Heuer’s writings to provide analysts with
tips and techniques for overcoming flaws and limitations in the human thinking
process.[66] The CAP
also uses analytic failure as a teaching tool for what not to do. Since much of
the intelligence analysis tradecraft has been derived from the intelligence
failure, “the lessons of …high-profile CIA screw-ups form the core of the Kent
School curriculum,” according to Bob Drogin. He then quoted the School’s Dean
who said: “We spend a lot of time in this course studying mistakes.”[67]
The analysts then go for a second four-week interim
assignment, and come back for a final four-week classroom session that deals
with more advanced topics than the one prior. For example, a session on
politicization and professional ethics teaches analysts “how to walk the fine
line that separates effective intelligence support for policymakers from
prescribing policy,” according to Stadther. Another session entitled ‘Writing
for the President’ is taught by senior DI editors and provides insight and
lessons learned on writing for the CIA’s most senior and demanding
policymakers. The CAP also provides new analysts with a ‘roundtable on customer
relations’ in which experienced analysts provide their hard-earned lessons in
dealing with congress, the public and the press, and a ‘secrets of success
panel’ providing tips of what does and does not work from analysts with a few
years of experience.
Once again, the classroom session ends with a task
force exercise, although this time it entails a two-day terrorist crisis
simulation that takes place outside the classroom and requires total immersion
to handle the fifteen-hour days. The CAP instructors use this as an opportunity
to present analysts with situations they might see later in their careers and
spike the simulation with dilemmas to force the analysts to improvise and apply
the skills they learned throughout the course, according to Stadther. As Bob
Drogin notes, it is “deliberately designed to be intensive and stressful.”[68]
Finally, at the end of the program the CAP provides written evaluations of the
students to their home offices documenting their skills, abilities, and
performance during training, according to former dean Frans Bax.[69]
In sum, the CAP’s 22 weeks provide analysts with information that is unique
to the realm of intelligence analysis as well as “greater opportunity for
individual growth in understanding of the profession, and hopefully a greater
self-conscious rigor in the use of analytical tools.”[70]
The School also offers occupational training
programs that provide analysts with more extensive theoretical frameworks with
which they can approach intelligence issues. The School offers courses for the
political, economic, military, leadership and scientific, technical, and
weapons disciplines and may create a few multi-disciplinary courses to
demonstrate how the disciplines interact with each other to produce a balanced
picture of the particular issue under study, according to Kent School Dean Dan
Wagner.[71]
The existing programs are intended for three levels of experience: junior,
journeyman, and expert. Entry-level training provides familiarization with the
disciplinary base while journeyman training entails deepening this knowledge
and specializing in areas relevant to the most pertinent issues on their
accounts. For example, if a political analyst has an account assessing the
developments within a country undergoing civil strife, disciplinary training
may provide training in political risk and stability assessments. Finally,
senior level training—for both analysts and managers—entails broadening their
knowledge of individual disciplines to include other disciplines and their
interactions. According to Dan Wagner, senior analysts may need to address
issues beyond the scope of their specialty when briefing high-level
policymakers, and managers require broad knowledge of the disciplines to manage
a team of multi-disciplinary analysts. The disciplinary training can be as long
as three weeks, but is usually dispensed in seminars lasting less than a week
and require intensive and focused learning.[72]
As with CAP, through disciplinary training analysts acquire a theoretical
foundation to their work that is job-relevant but difficult if not impossible
to acquire through on-the-job training.
Both training programs also utilize interactive
teaching methods as ways to maximize analyst learning and acquisition of
expertise. The training is intended to improve analyst performance, and rather
than risk student disengagement from abstract lecture courses the programs use
varied and active teaching methods which are ‘very important’ to provide
multiple pathways for students to apply their unique learning styles, according
to Dan Wagner.[73] He also
said that as a result disciplinary training methods require student interaction
rather than passive listening and include seminar discussions, practical
exercises, role-playing, and required reading before class. The CAP also
employs the case study method to make the lessons more fungible to real-world
situations.[74] Former CIA
instructor Thomas Shreeve noted that “cases fit best in those parts of the CIA
curriculum in which the teaching objectives included helping students learn how
to analyze complex, ambiguous dilemmas and make decisions about how to resolve
them.”[75]
The Kent School’s analyst training programs appear
to be improvements on CIA’s past efforts to provide analysts with job-relevant
expertise, although measuring this improvement from outside the institution is
impossible. Hypothetically, evaluating the impact of the Kent School is possible
through a simple pretest-posttest control group research design.[77]
For example, to establish the efficacy of the School in increasing new analyst
expertise a test could be administered both before and after training to
measure any increases in analyst knowledge. A control group composed of new
analysts who remain on the job instead of receiving training could be tested as
well, and any changes in analyst knowledge between the two groups would be
attributable to the training alone.
In October 2001, the Kent School’s director
implemented a version of this design by hiring a contractor to interview CAP
alumni and their managers to assess the effect of training on their job
performance.[78] CAP program
director Denis Stadther provided an overview of its conclusions. According to
Stadther, all interviewees considered the CAP to have been beneficial. In
particular, line managers noted that new analysts who went through it appeared
to possess greater confidence and maturity in applying their skills on the job.
In addition, CAP analysts created lasting ties with their classmates and the
use of these contacts increased their ability to work the system in job-related
tasks such as obtaining information from a counterpart agency, coordinating
with experts elsewhere within the government, or finding a fellow analyst
within CIA who covers a particular issue. Managers appreciated the analysts’
greater institutional expertise because it meant that they required less
intense mentoring to perform well on the job, according to Stadther. He also
noted that contrary to the belief of early CAP students that their promotion
prospects might be adversely affected by being pulled off the job for five
months of training, anecdotal evidence suggests that their promotion rates have
been as good or better than those who did not attend the CAP. According to
Stadther, a good performance by analysts in the CAP appears to translate well
into good job performance.
However, Stadther also emphasized that not all CAP
student feedback was positive, and that this led to adjustments in the
curriculum. For example, when analysts had difficulty relating to a theoretical
unit intended to improve interpersonal skills, the CAP’s program manager
refocused it on practical applications of those skills. Nonetheless, even
though feedback indicates that some aspects of the CAP’s curriculum require
modification, in the aggregate the CAP improves a new analyst’s individual
effectiveness and this may improve the DI’s effectiveness on the margins.
Training also provides other benefits to analyst
effectiveness that Stadther did not mention was part of the study. For example,
CAP promotes a better fit between analyst and assignment by encouraging
analysts to pursue transfers to units where their skills or interests may be
more effectively applied.[79]
The CIA’s placement policies are cumbersome due to the six-month or greater
time-lag between when a vacancy opens up and a qualified person can be hired
because of the background checks and other matters necessary to obtain security
clearances. This cumbersome process can lead to initial assignments that are a
misfit for the analyst’s knowledge and abilities. In 2001, the School’s former
Dean noted that the CAP “provides the opportunity for officers to find a better
fit for themselves” and that “there is an infinite range of career
possibilities and if an analyst sees something better they should go for it.”[80]
He added that in the year CAP had been in operation a number of analysts had
switched offices and even directorates, and that this was an intentional
attempt to break out of the apprenticeship model that had formerly formed the
core human resource approach for the DI.[81]
Therefore the CAP provides CIA with the opportunity to shift personnel in a way
that allows for interest and skill mixes to be applied more effectively towards
institutional goals.
In sum, the CAP provides analysts with greater
knowledge of the institution and its practices that can make their job
performance more effective, and even side benefits like providing the
opportunity to refine placement policies with transfers can improve
institutional performance on the margins. However, neither addresses the core
reason the Kent School was created; to improve CIA’s analytic quality.
Training
Could Also Improve Accuracy
More fundamental to the mission of intelligence
analysis than bureaucratic savvy or presentational ability, however, is the
accuracy of the finished intelligence product. According to CIA’s website, its
mission includes ‘(p)roviding accurate, evidence-based, comprehensive, and
timely foreign intelligence related to national security.’[82]
Directly measuring the impact of the Kent School’s training programs on
analytic quality is impossible from outside the institution, however, due to
classification restrictions on the analysis. In addition, so far as I know
neither the CIA in general nor the School measures the accuracy of the analytic
product because accuracy in intelligence analysis is a very difficult concept
to operationalize. Since intelligence analysis can influence what American
national security policymakers decide to do, and what they do has the potential
to prompt or preclude actions of other international actors, a hit-or-miss
yardstick would not effectively capture the quality of the analysis.[83]
For example, if CIA predicts that a terrorist bombing is imminent and
policymakers implement security procedures to deter or prevent this incident
based on CIA warnings, then the intelligence prediction will be inaccurate even
though it helped prevent the bombing. This causal dynamic exists for all
intelligence issues—including political, economic, and scientific--due to the
nature of the intelligence mission. Therefore, post-hoc assessment of
intelligence accuracy may not provide a true sense of the quality of the
analysis. Instead, both the Kent School and CIA’s evaluations focus on the
soundness of the analytic process because it is implicitly a modified version
of the scientific method.
The CIA analyst uses a version of the scientific
method to create intelligence analysis. The traditional ‘intelligence cycle’
describes how an analyst integrates information collected by numerous entities
and disseminates this information to policymakers. As former DCI William Colby
noted, “at the center of the intelligence machine lies the analyst, and he is
the fellow to whom all the information goes so that he can review it and think
about it and determine what it means.”[84]
While the ‘intelligence cycle’ presents this process in sequential terms, more
accurately the analyst is engaged in never-ending conversations with collectors
and policymakers over the status of international events and their implications
for United States policy. In converting raw intelligence data into finished analysis,
analysts interpret the international environment through an information
processing methodology approximating the scientific method.[85]
As intelligence author Washington Platt noted in 1957: “The so-called
'scientific method' means different things to different people, but the basic
features are much the same. These features are: collection of data, formation
of hypotheses, testing the hypotheses, and so arriving at conclusions based on
the foregoing which can be used as reliable sources of prediction.”[86]
An analyst builds his or her operating hypothesis upon constructs found in the country’s history, culture, regional dynamics, governmental workings or whatever aspect may be relevant to determine how the country’s leaders might respond to foreign challenges. The Kent School bolsters this knowledge with both procedural and disciplinary expertise to provide a methodological and theoretical base for the production of intelligence analysis. For example, academia develops theories to explain outcomes in the realms of economics, politics, psychology, and international relations, and these theories form the basis for any good intelligence analysis. The analyst’s reigning conceptual framework is grounded in disciplinary theory modified by knowledge of regional or country-specific substance. Analysts simultaneously—and for the most part instinctively—use inductive reasoning to find patterns amongst the flood of data, and use deductive reasoning to provide meaning and context for the patterns they find. Incoming bits of intelligence data are filtered through this framework, and those that fit are imbued with meaning and context, while those that do not fit are set aside as potential modifiers of the concepts. As a result, intelligence analysis when done accurately is a model of social scientific inductive and deductive reasoning in action.
Yet despite the similarities to social science
research, intelligence analysis is more complex. Two methodologists note that
intelligence analysis has difficulties—including externally imposed time
constraints which necessitate analysis prior to acquisition of sufficient
information, an inability to control the variables under study, an unknown data
quality due to imperfections in the collection process and the possibility of
deception, an emphasis on prediction, and a focus on utility--that combined
make intelligence analysis more difficult that social scientific research in an
academic setting.[87]
Also, like many specialties in the study of international relations
intelligence analysts cannot test their hypotheses or replicate outcomes in the
real world, and instead must allow the progression of events to falsify them.
In addition, international relations writ large has a minimal empirical base
relative to other fields such as medicine or physics, meaning that the etiology
of certain kinds of activities such as war or revolution cannot be aggregated
into anything other than a very general—and uncertain—theory. Unlike academics,
though, the intelligence analyst does not create theory, but must interpret—at
times under tight deadlines--facts that are themselves subject to distortion
and deception.[88] This means
that high levels of uncertainty are implicit in most analytic judgments.
As a result of the high levels of uncertainty,
intelligence analysis rarely has certainty or proof and for the most part
conclusions are tentative. In what has become a cliché, “(e)vidence collected
by intelligence agencies is often ambiguous and can lead to differing
conclusions,” as an article in the Washington Post reported.[89] In addition, Yitzhak Katz and Ygal Vardi
cite an experiment demonstrating that the same data can lead to differing
conclusions depending on the conceptual structures applied.[90]
The result of such ambiguity is an environment of conflict and debate over the
interpretation and implications of intelligence data. Loch Johnson says that
within the institutional context “analysis is usually a messy affair, with
incomplete information and often heated debates over what the available facts
may really mean about the intentions of secretive adversaries in distant
capitals. ... Uncertainty, ambiguity, debate, and partial answers to tangled
questions will remain an existential condition of the analytical process.”[91]
Given this complex information environment, not
knowing—or negligence in applying—social science techniques can lead to
analytic inaccuracies. As Robert Jervis observed in 1986: “Good intelligence
demands three interrelated conditions. Analysts should present alternatives and
competing explanations for a given even(t), develop the evidence for each of
the alternatives, and present their arguments as fully as is necessary to do
justice to the subject matter. This is not to imply, of course, that if these
conditions are met the resulting analyses will always be excellent but only
that their omission will substantially reduce the probability that the product
will be of high quality.”[92]
At its most extreme outcome, analytic inaccuracies due to failure to apply
tradecraft can lead to intelligence failures. As intelligence scholar Robert
Folker notes, “the root cause of many critical intelligence failures has been
analytical failure.”[93]
Yitzhak Katz and Ygal Vardi argue that although Israeli intelligence possessed
“data pointing to the probability of war” Israel was surprised by the 1973 Yom
Kippur War because it “was incorrectly interpreted.”[94]
In addition, Georgetown Professor Walter Laqueur notes that sources of analytic
failure include “bias…(which is) an unwillingness…to accept evidence contrary
to their preconceived notions, … ignorance, lack of political sophistication
and judgment, (l)ack of training and experience, lack of imagination, and the
assumption that other people behave more or less as we do” which is known in
intelligence parlance as ‘mirror-imaging.’[95]
In sum, failure to apply appropriate social science standards of rigor and
procedure can lead to analytic inaccuracy. If so, then accuracy can be improved
on the margins by providing analysts with greater knowledge of fact, theory or
procedure and the application of methodologies to simplify and structure their
analysis.
The Kent School’s CAP and disciplinary training provide analysts with improved theoretical and methodological knowledge as well as analytical tools that should provide a more rigorous basis for their analysis. This greater rigor approximating social science methods should increase the chance that the analytic product will be more accurate than it otherwise would have been without training. Just as increases in the quality of research methodology allows a researcher to more effectively operationalize and measure the constructs of interests, better training in relevant theory or methodology would hypothetically provide an intelligence analyst with a better foundation from which to produce more accurate analysis. Therefore, if the theory holds then the Kent School will provide analysts with the capability to produce more accurate analysis so long as its programs provide information that enables analysts to apply the scientific method more effectively.
There are many possible methodological tools
analysts can use to organize and simplify the process of intelligence analysis.
As Washington Platt noted in 1957: “In intelligence production as yet we find
little study of methods as such. Yet a systematic study of methods in any given
field by an open-minded expert in that field nearly always leads to worthwhile
improvement.”[96] For
example, one JMIC publication—a distillation of a student's Masters
thesis—demonstrates that “analysts who apply a structured method (specifically)
hypothesis testing…to an intelligence problem, outperform those who rely on…the
intuitive approach.”[97]
Just as social science is structured upon a foundation of hypothesis
falsification and revision which requires that hypotheses are compared for
accuracy and relevance, intelligence analysts self-consciously compare
hypotheses in a process termed ‘competitive analysis.’ Competitive analysis is
a technique that can be applied in particular circumstances—such as when an
issue is of particular importance and yet imbued with partisan implications—to
get at underlying assumptions that might be biasing the analysis in one
direction or the other.[98]
Some have even advocated the structural incorporation of this method through an
institutionalized ‘Devil’s Advocate’ position or ‘Team B’ exercises. According
to Richards Heuer, other tools that help an analyst to structure the most
appropriate approach to a particular intelligence problem include “outlines,
tables, diagrams, trees, and matrices, with many sub-species of each. For
example, trees include decision trees and fault trees. Diagrams includes causal
diagrams, influence diagrams, flow charts, and cognitive maps. …(In addition, a
matrix can be used) to array evidence for and against competing hypotheses to
explain what is happening now or estimate what may happen in the future.”[99]
Finally, R.V. Jones--the ‘father of modern scientific and technical
intelligence’--suggests analysts apply Occam's Razor—or ‘the simplest
hypothesis that is consistent with the information’—as a way to simplify the
analysis of contradictory or insufficient information.[100]
The analytic tradecraft standards and techniques
taught in CAP were developed in the 1990s. According to former CIA officer Jack
Davis, former DDI Doug MacEachin articulated “corporate tradecraft standards for analysts..aimed…at ensuring
that sufficient attention would be paid to cognitive challenges in assessing
complex issues.”[101]
Davis added: “MacEachin advocated an approach to structured argumentation
called "linchpin analysis," to which he contributed muscular terms
designed to overcome many CIA professionals' distaste for academic
nomenclature. The standard academic term "key variables" became drivers.
"Hypotheses" concerning drivers became linchpins--assumptions
underlying the argument--and these had to be explicitly spelled out.….
MacEachin thus worked to put in place systematic and transparent standards for
determining whether analysts had met their responsibilities for critical
thinking.”[102] MacEachin
then formalized promulgation of methodologies to all its analysts. In 1996 and
1997, “(n)early all DI managers and analysts attended” a new course based on
the standards entitled ‘Tradecraft 2000,’ according to Jack Davis.[103]
In addition, in 1997 Jack Davis authored “a series of notes on analytical
tradecraft” that encapsulated the lessons from the new course.[104]
According to the CIA website—where portions of these notes can be found--the
notes “elaborate on some of the skills and methods used by DI intelligence
analysts” and have “become a standard reference within CIA for practitioners
and teachers of intelligence analysis.”[105]
These methodologies have become, in essence, analytic doctrine, and through
them CIA analysts have become familiarized with social science methodologies if
with a modified vocabulary. In 2001, the CIA website notes that: “Invariably,
analysts work on the basis of incomplete and conflicting information. DI
analysts are taught to clearly articulate what is known (the facts), how it is
known (the sources), what drives the judgements (linchpin assumptions), the
impact if these drivers change (alternative outcomes), and what remains
unknown.”[106] The
analytic standards were subsequently incorporated into the existing training
courses for analysts, and maintain their place in the Kent School’s curriculum
today.
In addition, the Kent School’s disciplinary training
provides analysts with a more comprehensive theoretical framework with which to
interpret raw intelligence data and insight into what kinds of information will
confirm or repudiate the hypothesis under consideration. As such, the
disciplinary training provides analysts with a more effective filter through
which to sift the data, and so long as the theory is accurate the training will
provide analysts with greater ability to produce more accurate intelligence
analysis.
In sum, the Kent School’s training programs provide
analysts with the expertise necessary to produce more accurate analysis. This
conclusion, however, is insufficient to address whether the Kent School’s
analytic training programs actually improve the quality of CIA’s analytic
output.
The Kent School’s analytic training programs provide the potential to improve overall analytic accuracy, but this potential can be stifled if the DI’s business practices prevent application of this hard-earned expertise. Intelligence analysis takes place within a specific organizational context, and the processes of this organization can promote or impede the application of lessons learned in training and thereby affect the ultimate accuracy of the intelligence product.
For example, the length of finished intelligence
product emphasized by the DI can impact expertise acquisition. The DI produces
many types of finished intelligence--some reportorial, some analytical, some
estimative--largely as a result of its attempt to meet policymakers’ varying
needs for information.[107]
The reportorial products—known as current intelligence--tend to be shorter but
more targeted to policymaker needs, while the more estimative and analytic
products can be longer and potentially less relevant for policymakers due to
the time of production and length of product. Over time the DI tends to
emphasize the benefits of one product length over another. In the 1980s the DI
emphasized the production of longer pieces for the knowledge that they create,
but after arguments surfaced that these longer papers were of decreased utility
for policymakers the DI promptly reversed course in the mid-1990s and limited
“most DI analytical papers … to 3 to 7 pages, including graphics and
illustrations.”[108]
Presumably this was done to increase the relevance of DI products for policymakers.
However, while the production of a single piece of current intelligence has
limited effect on expertise because it draws on the knowledge and tools that an
analyst has developed through training and prior analysis, expertise
acquisition and application problems become endemic if the institution
emphasizes current intelligence over longer products. If the analyst does not
have time to think, learn, or integrate new information with old to create new
understandings, knowledge of facts and events may increase but the ability to
interpret these events accurately decreases as the analyst does not have the
opportunity to reinforce knowledge through the more in-depth research and
drafting processes. In addition, it deprives analysts of the opportunity to
apply ‘best practices’ including the structured tradecraft methodologies and
benefits arising from disciplinary theory. By 1997, DI management was making
efforts to address the decreased expertise levels resulting from excessive
current intelligence production.[109]
In the end, the Kent School’s effectiveness in improving analytic accuracy may
depend in part on whether the CIA provides its analysts with assignments that
allow them to utilize the expertise acquired in training while still providing
analysis that is relevant at the same time.
In addition, the length of time that the DI
encourages an analyst spend on an account can also impact an analysts
expertise, and by extension accuracy. In October 1991, former CIA Inspector
General Fred Hitz argued that CIA’s analysts have “tended to be 18-month
wonders who hopscotch from subject to subject without developing any broad,
long-term expertise in any one area” and as a result “are ill-equipped to
grapple with the information that arrives on their desks.”[110]
An analyst cannot apply sophisticated methodological tools to intelligence data
until he or she becomes knowledgeable with the specific regional area or
discipline of the account, and frequent changes will prevent the analyst from
achieving the potential inherent in the analytic tools that the Kent School
provides new analysts. Therefore, for the Kent School to improve analytic
accuracy through the application of the analysts’ increased expertise,
“analysts should be expected to make a career of their functional or geographic
specialty so that the daily flows of information are added to an already solid
base of knowledge” as Fred Hitz suggests.
In sum, the DI must adapt its practices to provide
analysts with the greatest opportunity to apply the expertise they have
acquired both on-the-job and in training if the potential inherent in the Kent
School’s programs to improve the DI’s analytic accuracy is to be fully
realized. Yet despite the importance of institutional context in determining
CIA’s analytic quality, few have studied the interplay between analyst and
institution. As Robert Jervis has observed, “perhaps the intelligence community
has paid too (little attention) to how the community’s internal structure and
norms might be altered to enable intelligence to be worth listening to.”[111]
Further studies of analytic structures and processes must be done to determine
which reforms would have the greatest improvement on analytic quality, and
perhaps their implementation will take CIA a step closer to creating ‘intelligence
worth listening to.’
In 1988, Loch Johnson said “probably the most
important problem faced by any intelligence agency (is) how to improve the
quality of its product.”[112]
The Kent School is part of the CIA’s effort to do this, and it appears that the
School has achieved its goal of increasing analytic expertise by improving the
agency’s training for new analysts and providing disciplinary training to
analysts of all levels. This training improves the analysts’—and consequently
the DI’s—effectiveness, and should provide each analyst with a greater ability
to produce more accurate intelligence analysis. However, even if increases in
analyst expertise due to training can be documented, the analyst may not have
the time, inclination or opportunity to apply recently acquired expertise in
the production of finished intelligence analysis. Analytic training may be a
necessary component for improvement in analytic accuracy writ large, but is not
sufficient in and of itself. Therefore, complementary modifications may have to
be implemented to the DI’s business practices if the desired outcome—increased
analytic accuracy--is to occur. As such, the Kent School is a step in the right
direction, but further assessments are required to determine which changes in
business practices or organizational structure might magnify possible
improvements.
The Kent School’s training programs, however, only
skim the surface of the potential organizational benefits arising from an
educational unit protected from line-production pressures. Every organization
can have difficulty successfully adapting its business practices to changes in
environmental conditions, but successful change can be even more difficult to
achieve if the organization lacks an ability to capture and utilize knowledge
of its own processes. CIA has historically accomplished its reforms through
task forces or consultations with outside experts, but the knowledge gained
thereby is subsequently lost when the task force or consultancy is disbanded. A
staff unit with the ability to create, accumulate, and disseminate knowledge
could improve CIA’s ability to adapt to changing conditions and thereby improve
the quality of its overall output. In 2000, I wrote an article advocating the creation
of a ‘CIA University’ built around the Kent School in order to use
institutional synergies to capture knowledge of innovative practices and
experiment with new ways of analyzing or producing intelligence.[113]
The Kent Center provides some of these benefits, as does OTE, the Center for
the Study of Intelligence, and a few other units spread throughout the
bureaucracy. However, it seemed to me that just as a university is made up of
different schools that build on each other’s strengths, each innovative unit
within CIA would function more effectively if brought together formally into a
loose affiliation. Apparently senior officials at CIA agree, for in early 2002
they created a ‘CIA University,’ although its specific structure and functions
are as yet unclear. Further assessments of the Kent School’s functions vis a
vis the CIA University would provide a better sense of how training programs
fit into the institutional attempt to improve the quality of intelligence
analysis.
In addition, a broader study comparing and
contrasting CIA’s approach to training, education and innovation to that of the
American military could provide an even better sense of the Kent School’s
strengths, weaknesses, and potential to improve the quality of CIA’s analysis.
The DOD has a long tradition and cultural acceptance of training, education,
and innovation, and a comparison to CIA’s approach could provide insight into
the specialized requirements of intelligence production. In fact, some
synergies may already exist that could be exploited. DOD’s JMIC has done a
number of studies that could be helpful for CIA’s analytic recruitment and
training, and CIA’s disciplinary training has provided JMITC with a model for
its own training programs.[114]
Also, both the Kent Center and JMIC’s recently created “Center for Strategic
Intelligence Research” intend to centralize knowledge of intelligence practices
and improve the intelligence literature by taking experienced intelligence
officers offline and providing them with the opportunity to document their
experiences.[115] They also
intend to disseminate unclassified versions of these papers through some kind
of occasional paper series. While each program also provides unique services to
its home institution that the other does not duplicate, on the whole it appears
that since both the Kent School and JMIC are producing analogous programs
nearly simultaneously, a comparison of the approaches taken by each institution
could provide an interesting perspective on what works and does not work for
each organization.
Beyond the relatively limited scope of CIA’s
training and education programs, however, this paper also broaches issues that
have implications for American intelligence more broadly. The tragic events of
September 2001 have highlighted the limitations of institutionalized
intelligence—including the problems inherent in intelligence analysis--and
present policymakers with the opportunity to redefine the intelligence
community of the next century. This time “(w)e are trying to treat the
intelligence community more like a corporation that should have goals and a way
to measure success or failure against those goals,” according to an
intelligence expert advising President George Bush on ways to improve the
workings of the intelligence community.[116]
This paper uses increased analytic quality as its goal and assesses one how
small part of CIA’s current capabilities—the Kent School’s analytic training
programs—help CIA reach that goal. Yet in the broader context of national
security policymaking, improving the accuracy of CIA's analytic output on the
margins may not necessarily lead to improved policymaking. Intelligence
analysis is only one of many information streams that the policymaker listens
to, and may not be the most important. In addition, a policymaker's decisions
will be affected by policy agendas, tradeoffs, and other cognitive limitations
that create 'bounded' rationality. Therefore, if the researcher’s primary
concern is more effective national security policymaking, then it may be more
useful in this limited-resource environment to assess how intelligence
production can be modified to better fit policymaker needs. In the end the
utility of intelligence analysis may be just as important a criterion of
analytic quality as is accuracy.[117]
Nevertheless, understanding the options available to improve policymaking is
crucial, and improving CIA's analytic accuracy is one of those options. As
Robert Jervis said, “(w)e will never be able to do as well as we would like,
but this does not mean that we cannot do better than we are doing now.”[118]
Sherman Kent’s Principles for Intelligence Analysis
The Career Analyst Program teaches these principles.
1. Intellectual Rigor
Ø Judgments are supported by facts or credible reporting.
Ø All sources are reviewed and evaluated for consistency, credibility.
Ø Uncertainties or gaps in information are made explicit.
2. Conscious Effort to Avoid Analytical Biases
Ø State working assumptions and conclusions drawn from them explicitly.
Ø Subject assumptions and conclusions to structured challenge: what developments would indicate they would be wrong.
Ø If uncertainties or the stakes of being wrong are high, identify alternative outcomes and what it would take for each to occur.
3. Willingness to Consider Other Judgments
Ø Recognize the limits to your own expertise and avoid treating your account as yours alone.
Ø Seek out expertise that will complement your own as a product is being prepared.
Ø Strong differences of view should be made explicit.
4. Collective Responsibility for Judgment
Ø Seek out and allow time for formal coordination of your product.
Ø Represent and defend all Agency and DI views.
Ø Make it clear when you express individual views; do so only when asked.
5. Precision of Language
Ø Provide your most unique or new insight or fact quickly.
Ø Use active voice and short sentences; avoid excessive detail; minimize the use of technical terms. Follow DI writing guidelines.
Ø Shorter is always better.
6. Systematic Use of Outside Experts as a Check on In-House Blinders
Ø Seek out new external studies and experts relevant to your account and discipline on a continuing basis.
Ø Keep up with news media treatment of your account and consider whether their perspective offers unique insight.
Ø On key issues, indicate where outsiders agree or disagree with your judgments.
7. Candid Admission of Shortcomings and Learning from Mistakes
Ø Recognize that intelligence analysis will sometimes be wrong because it must focus on the tough questions or uncertainties.
Ø Review periodically past judgments or interpretations; what made them right or wrong; how could they have been better.
Ø Alert the policymaker if you determine that a previous line of analysis was wrong. Explain why and what it means.
8. Attentiveness To and Focus On Policymaker Concerns
Ø Deliver intelligence that is focused on and timed to the policymaker’s current agenda.
Ø Make clear the implications of your analysis for US policy.
Ø Provide “actionable” intelligence that can help the policymaker handle a threat, make a decision, or achieve an objective.
9. Never Pursue a Policy Agenda
Ø Personal policy preferences must not shape the information presented or the conclusions of intelligence analysis.
Ø Politely but clearly deflect policymaker requests for recommendations on policy.
Ø Intelligence helps the policymaker by reducing the range of uncertainty and risk, and by identifying opportunities for action. It does not make the choice for him.
Asker, James R. ‘Same Ol’, Same Ol’’ Aviation Week and Space Technology, 8 June 1998
Auster, Bruce B. ‘What's really gone wrong with the CIA,’ U.S. News and World Report, 1 June 1998,
Vol. 124, No. 21, 27.
Bax, Frans. Personal Interview. 14 May 2001
Bax, Frans. Presentation at an Association of Former Intelligence Officer’s Luncheon. Fort Myer, Virginia.
22 May 2001
Betts, Richard K. ‘Analysis, War and Decision: Why Intelligence Failures Are Inevitable,’ World Politics,
XXX1, 1 (October 1978): 61-89.
Campbell, Donald T. and Julian C. Stanley, ‘Experimental and Quasi-Experimental Designs for Research,’
Houghton Mifflin: Boston, 1963. 13-22.
Chan, Steve. ‘The Intelligence of Stupidity: Understanding Failures in Strategic Warning.’ The American
Political Science Review. Vol. 73. No. 1 (Mar. 1979). 171-180.
CIA Publication. Kent School Materials. Sherman Kent Principles of Intelligence Analysis. Acquired 14
May 2001.
CIA Publication. “What’s News at CIA.” Issue number 703. Unclassified article: ‘DDI Establishes
Sherman Kent School for Intelligence Analysis. Frans Bax Named Dean.’ Unknown Date.
CIA Website. Analytical Products of the DI http://www.odci.gov/cia/di/work/major.html
CIA Website. CIA Vision, Mission, and Values.
http://www.cia.gov/cia/information/mission.html
CIA Website. DI
Analytic Toolkit. http://www.odci.gov/cia/di/toolkit/index.html
CIA Website.
Directorate of Intelligence: Organizational Components
http://www.odci.gov/cia/di/mission/components.html (see: Council of Intelligence Occupations)
CIA Website. Intelligence Analysis in the DI: Frequently Asked Questions.
http://www.odci.gov/cia/di/work/analyst.html
CIA Website. Intelligence Disciplines. http://www.odci.gov/cia/di/work/disciplines.html
CIA Website. John E. McLaughlin: Deputy Director of Central Intelligence
http://www.cia.gov/cia/information/mclaughlin.html
CIA Website. Key Events In DI History http://www.odci.gov/cia/di/history/index.html
CIA Website. Remarks of the Deputy Director of Central Intelligence John E. McLaughlin at the
Conference on CIA's Analysis of the
Soviet Union, 1947-1991, Princeton University 9 March 2001.
http://www.cia.gov/cia/public_affairs/speeches/ddci_speech_03092001.html
CIA Website. Remarks of the Director of Central Intelligence George J. Tenet at the Dedication of the
Sherman Kent School, 4 May 2000. http://www.cia.gov/cia/public_affairs/speeches/archives/2000/dci_speech_05052000.html.
CIA Website. Tenet Dedicates New School for Intelligence Analysis. CIA Press Release. 4 May 2000.
http://www.cia.gov/cia/public_affairs/press_release/archives/2000/pr050400.html
CIA Website. Tenet Lauds Appointment of McLaughlin as Acting DDCI. Press Release. 29 June 2000.
http://www.cia.gov/cia/public_affairs/press_release/archives/2000/pr06292000.html
Clark, Robert M. “Intelligence Analysis: Estimation and Prediction.” American Literary Press Inc.
Baltimore, Maryland. 1996.
Clauser, Jerome K. and Sandra M. Weir. Intelligence Research Methodology. State College, PA; HRB
Singer, Inc. 1976.
Colby, William. 'Retooling the Intelligence Industry.'
Foreign Service Journal. Jan 1992: 21-25.
Davis, Jack. ‘Improving Intelligence Analysis at CIA: Dick Heuer’s Contribution to Intelligence Analysis,’
Psychology of Intelligence Analysis, Center for the Study of Intelligence, CIA, 1999. xvii-xix
DIA Website. About the JMIC. http://www.dia.mil/Jmic/about.html
Dobbs, Michael. ‘U.S., Russia At Odds on Iranian Deal.’ Washington Post, 15 June 2001, A01.
Drogin, Bob. ‘School for New Brand of Spooks.’ Los Angeles Times. 21 July 2000. A-1
Folker, Robert D. ‘Intelligence in Theater Joint Intelligence Centers: An Experiment in Applying
Structured Methods.’ Joint Military Intelligence College Occasional Paper Number Seven.
Washington, DC. January 2000.
Garst, Ronald D. ‘Intelligence Types: The Role of Personality in the Intelligence Profession.’ Joint Military
Intelligence College. Washington D.C. August 1993.
Garst, Ronald D. and Max L. Gross. ‘On Becoming an Intelligence Analyst.’ Defense Intelligence Journal.
6-2 (1997). 47-59.
Gentry, John. A Framework for Reform of the U.S. Intelligence Community. 6 June 1995
http://www.fas.org/irp/gentry/
Hammond, Thomas H., “Agenda Control, Organizational Structure, and Bureaucratic Politics” American
Journal of Political Science, v. 30, no. 2 (May 1986), 379-420.
Heuer, Richards J. Jr. Psychology of Intelligence Analysis, Center for the Study of Intelligence, CIA, 1999.
Hitz, Frederick P. “Not Just a Lack of Intelligence, a Lack of Skills.” The Washington Post. 21 October
2001, B3.
Hulnick, Arthur S. ‘Managing Intelligence Analysis: Strategies for Playing the End Game,’ International
Journal of Intelligence and Counterintelligence, v. 2, no. 3 (1988): 321-343.
Jervis, Robert. 'What's Wrong with the Intelligence Process?' International Journal of Intelligence and
Counterintelligence, vol. 1, no. 1 (1986): 28-41.
Johnson, Loch K. ‘Analysis For a New Age.’ Intelligence and National Security. Vol. 11, No. 4 (Oct 96)
657-671.
Johnson, Loch K. A Season of Inquiry. Dorsey Press, Chicago, 1988.
Johnson, Loch K. ‘Making the Intelligence “Cycle” Work,’ International Journal of Intelligence and
Counterintelligence, v. 1, no. 4, (1986): 1-23.
Jones, R.V. Reflections on Intelligence. Heineman: London. 1989.
Katz, Yitzhak and Ygal Vardi. ‘Strategies for Data Gathering and Evaluation in the Intelligence
Community’ International Journal of Intelligence and Counterintelligence. Vol. 5, No. 3, 313-328.
Laqueur, Walter. ‘The Future of Intelligence.’ Society, Jan-Feb 1998 v35 n2 p301(11).
Loeb, Vernon. ‘CIA Goes Deep Into Analysis; Agency Opens School, Elevates Analysts,’ Washington
Post, 4 May 2000, A23.
Loeb, Vernon. ‘Tenet, Krongard Alter CIA Power Structure.’ The Washington Post. 1 May 2001, A21.
MacEachin, Douglas J. et al. ‘The Tradecraft of Analysis: Challenge and Change in the CIA,” Consortium
for the Study of Intelligence, Washington DC, 1994
McLaughlin, John E. 'New Challenges and Priorities for Analysis,' Defense Intelligence Journal, vol. 6, no.
2 (1997): 11-21. Also on the CIA website at
http://www.cia.gov/cia/di/speeches/428149298.html
McLaughlin, John E. Personal Conversation. 9 March 2001.
Marrin, Stephen. “The CIA's Kent School: A Step in the Right Direction.” The Intelligencer: Journal of
U.S. Intelligence Studies. V. 11, No. 2. (Winter 2000). 55-57.
Marrin, Stephen. “Complexity is in the Eye of the Beholder.” DI Discussion Database. 6 May 1997.
Merriam-Webster Website: http://www.m-w.com
Olson, Ken. Personal Interview. 16 May 2001
Omestad, Thomas. 'Psychology and the CIA: Leaders on the Couch.' Foreign Policy. Summer 1994. 105-
122.
Pincus, Walter. “CIA Chief Cited Loss of Agency’s Capabilities; Remarks Preceded Indian Bomb Tests.”
The Washington Post. 25 May 1998. A4.
Pincus, Walter. “Intelligence Shakeup Would Boost CIA.” The Washington Post. 8 November 2001. A1.
Platt, Washington. Strategic Intelligence Production: Basic Principles. Frederick A. Praeger: New York.
1957.
Reich, Robert C. ‘Re-Examining the Team A-Team B Exercise.’ International Journal of Intelligence and
Counterintelligence. vol. 3, no. 3 (Fall 1989): 387-403.
“Research: Design and Methods.” Joint Military Intelligence College. Washington D.C. September 2000.
Robbins, Carla Anne. ‘Failure to Predict India’s Tests Is Tied to Systemwide Intelligence Breakdown.’ The
Wall Street Journal, 3 June 1998. A8.
'Say It Ain't So, Jim': Impending Reorganization of CIA Looks Like Suppression, Politicizing of
Intelligence' Publications of the Center for Security Policy, No. 94-D 74, 15 July 1994,
http://www.security-policy.org/papers/1994/94-D74.html
Schum, David. “Evidence and Inference for the Intelligence Analyst.” University Press of America,
Lanham, Maryland. 1987.
Shreeve, Thomas W. and James J. Dowd, Jr. ‘Building A Learning Organization: Teaching with Cases at
CIA. International Journal of Intelligence and Counterintelligence. Vol. 10, No. 1, 97-107.
Stack, Kevin P. “A Negative View of Competitive Analysis.” International Journal of Intelligence and
Counterintelligence. vol. 10, no. 4. 456-464.
Stadther, Denis. Personal Correspondence. 5 September 2001.
Stadther, Denis. Personal interview. 14 May 2001.
Stadther, Denis. Personal interview. 26 February 2002.
Stringer, Kenneth T. Presentation at an Association of Former Intelligence Officer’s Luncheon. Fort Myer Virginia. 22 May 2001.
Swenson, Russell. Personal interview. 8 March 2002.
Tenet, George. Keynote address to the 1999 Association of Former Intelligence Officers Annual
Symposium. 22 October 1999.
Tenet, George J. “The CIA and the Security Challenges of the New Century.” International Journal of
Intelligence and Counterintelligence. Vol. 13. No. 2. (April 2000): 133 - 143.
Turner, Stansfield. 'Intelligence for A New World Order.' Foreign Affairs, Fall 1991, 150-166.
Wagner, Dan. Personal interview. 23 August 2001.
Watanabe, Frank. 'Fifteen Axioms for Intelligence Analysts,' Studies in Intelligence; unclassified edition.
1997. http://www.odci.gov/csi/studies/97unclass/axioms.html.
Weiner, Tim. “Naivete at the CIA: Every Nation’s Just Another U.S.” New York Times. 7 June 1998.
Weiner, Tim. “U.S. Intelligence Under Fire in Wake of India’s Nuclear Test.” New York Times. 13 May
1998.
Young, Jay T. ‘US Intelligence Assessment in a Changing World: The Need for Reform.’
Intelligence and National Security, v. 8, no. 2 (Apr 1993), 125-139.
[1] This paper is adapted from a Master’s thesis prepared for the University of Virginia under the tutelage of Professor Kenneth Thompson and Ambassador Marshall Brement. Its content was also shaped by an earlier seminar paper written for former CIA inspector general Fred Hitz. Parts of the argument may also soon appear in a book being compiled by Professor Russell Swenson of the Joint Military Intelligence College. Also, the paper’s information on CIA’s analytic process was acquired through personal experience as a leadership analyst for CIA’s Office of Near East, South Asia, and Africa in 1996 and 1997, a project assistant for a group in the Office of Transnational Issues in 1998, and a team leader for a contractor supplying analytical support to the Office of Transnational Issues in 1999 and 2000. CIA’s Publication Review Board has reviewed the Master’s thesis upon which this article was based, and had no security objection to its dissemination. Their review, however, does not confirm the accuracy of the information nor does it endorse the author's views.
[2] CIA Website. “Tenet Dedicates New School for Intelligence Analysis.” CIA Press Release. 4 May 2000.
[3] CIA Website. “Remarks of the Director of Central Intelligence George J. Tenet at the Dedication of the Sherman Kent School, 4 May 2000.”
[4] CIA Website. “Tenet Dedicates New School for Intelligence Analysis.” CIA Press Release. 4 May 2000.
[5] Auster, Bruce B. “What's really gone wrong with the CIA.” U.S. News and World Report, 1 June 1998, Vol. 124, No. 21, 27.
[6] Pincus, Walter. “CIA Chief Cited Loss of Agency’s Capabilities; Remarks Preceded Indian Bomb Tests.” The Washington Post. 25 May 1998. A4.
[7] Tenet, George J. “The CIA and the Security Challenges of the New Century.” International Journal of Intelligence and Counterintelligence. Vol. 13. No. 2. (April 2000): 140-141.
[8] Various newspaper articles including: Drogin, Bob. “School for New Brand of Spooks.” Los Angeles Times. 21 July 2000. A-1.; Weiner, Tim. “U.S. Intelligence Under Fire in Wake of India’s Nuclear Test.” New York Times, 13 May 1998.; Pincus. “CIA Chief Cited Loss of Agency’s Capabilities; Remarks Preceded Indian Bomb Tests.”
[9] For a good presentation of the specific indicators, see: Weiner. “U.S. Intelligence Under Fire in Wake of India’s Nuclear Test.”
[10] Asker, James R. ‘Same Ol’, Same Ol’’ Aviation Week and Space Technology, 8 June 1998
[11] Robbins, Carla Anne. “Failure to Predict India’s Tests Is Tied to Systemwide Intelligence Breakdown.” The Wall Street Journal, 3 June 1998. A8.
[12] Ibid.
[13] Denis Stadther was a member of this task force and provided all the information on the Kent School’s creation during an interview on 14 May 2001 except where otherwise indicated.
[14] Loeb, Vernon. ‘CIA Goes Deep Into Analysis; Agency Opens School, Elevates Analysts,’ Washington Post, 4 May 2000, A23.
[15] CIA Website. “Tenet Lauds Appointment of McLaughlin as Acting DDCI.” Press Release. 29 June 2000. Also see: CIA Website: John E. McLaughlin: Deputy Director of Central Intelligence
[16] Tenet, George. Keynote address to the 1999 AFIO Annual Symposium, 22 October 1999.
Tenet’s public statement preceded the internal CIA announcement of the Kent School’s creation by two months. Although the annual symposium is considered "For Background Use Only," both AFIO and DCI Tenet--through CIA Public Affairs contact with an executive assistant--have approved of this reference to DCI Tenet's statement at that forum.
[17] Loeb, Vernon. ‘CIA Goes Deep Into Analysis; Agency Opens School, Elevates Analysts.’ For more on Martin Petersen, see: Loeb, Vernon. ‘Tenet, Krongard Alter CIA Power Structure.’ The Washington Post. 1 May 2001, A21.
[18] Merriam-Webster Website: http://www.m-w.com
[19] Johnson, Loch K. “Analysis For a New Age.” Intelligence and National Security. V11, N.4 (Oct 96) 657.
[20] CIA Website. “Remarks of the Director of Central Intelligence George J. Tenet at the Dedication of the Sherman Kent School, 4 May 2000.”
[21] CIA Website. Directorate of Intelligence: Organizational Components. See also: CIA Website. Intelligence Disciplines.
[22] Watanabe, Frank. “Fifteen Axioms for Intelligence Analysts.” Studies in Intelligence; unclassified edition; 1997.
[23] Jervis, Robert. “What's Wrong with the Intelligence Process?” International Journal of Intelligence and Counterintelligence, vol. 1, no. 1 (1986). 31-32.
[24] Turner, Stansfield. “Intelligence for A New World Order.” Foreign Affairs, Fall 1991, 164.
[25] Jervis. “What's Wrong with the Intelligence Process?” 33.
[26] Jervis. “What's Wrong with the Intelligence Process?” 31-32.
[27] Jervis. “What's Wrong with the Intelligence Process?” 28, 30.
[28] Marrin,
Stephen. “Complexity is in the Eye of the Beholder.” DI Discussion Database.
6 May 1997.
[29] The inevitability of intelligence failure appears to be a consensus position in the intelligence failure literature and stems from Richard Betts’ 1978 article: Betts, Richard K. ‘Analysis, War and Decision: Why Intelligence Failures Are Inevitable,’ World Politics, XXX1, 1 (October 1978): 61-89.
[30] Olson, Ken. Personal Interview. 16 May 2001; DIA Website: About the JMIC
[31] Author’s personal experience circa 1996.
[32] CIA Website. “Remarks of the Deputy Director of Central Intelligence John E. McLaughlin at the Conference on CIA's Analysis of the Soviet Union, 1947-1991, Princeton University 9 March 2001.”
[33] Weiner, Tim. “Naivete at the CIA: Every Nation’s Just Another U.S.” New York Times. 7 June 1998.
[34] Bax, Frans. Presentation at an Association of Former Intelligence Officer (AFIO) Luncheon. Fort Myer, Virginia. 22 May 2001.
[35] Davis, Jack. “Improving Intelligence Analysis at CIA: Dick Heuer’s Contribution to Intelligence Analysis,” Psychology of Intelligence Analysis, Center for the Study of Intelligence, CIA, 1999. xix.
[36] Author’s experience in one of FDIT’s earlier runnings in the fall of 1996.
[37] Gentry, John. “A Framework for Reform of the U.S. Intelligence Community.” 6 June 1995.
[38] Hulnick,
Arthur S. ‘Managing Intelligence Analysis: Strategies for Playing the End
Game,’ International Journal of Intelligence and Counterintelligence, v. 2, no.
3 (1988): 323.
[39] Thomas Hammond argues that organizational structure impacts analytic output by using informational flow models to demonstrate that given the same information one group of intelligence analysts organized by discipline would produce different analytic output than one group organized by region. He also concludes that it is impossible to design a structure that does not impact output. See: Hammond, Thomas H., “Agenda Control, Organizational Structure, and Bureaucratic Politics” American Journal of Political Science, v. 30, no. 2 (May 1986), 379-420.
[40] CIA Website. Key Events In DI History
[41] See: Omestad, Thomas. 'Psychology and the CIA: Leaders on the Couch.' Foreign Policy, Summer 1994, 105-122.
[42] CIA Website. Key Events In DI History. However, political factors may also have played a part in this decision. The Clinton Administration’s Haiti policy was greatly complicated by partisan opposition in the wake of a leaked leadership profile produced by LDA questioning the psychological stability of Jean Bertrand Aristide. “Say It Ain't So, Jim”: Impending Reorganization of CIA Looks Like Suppression, Politicizing of Intelligence' Publications of the Center for Security Policy, No. 94-D 74, 15 July 1994.
[43] McLaughlin, John E. “New Challenges and Priorities for Analysis.” Defense Intelligence Journal, vol. 6, no. 2 (1997): 18. Also on the CIA website at http://www.cia.gov/cia/di/speeches/428149298.html
[44] CIA Website: Directorate of Intelligence: Organizational Components, Council of Intelligence Occupations
[45] Ibid.
[46] Ibid.
[47] Wagner, Dan. Personal interview. 23 August 2001.
[48] Loeb. ‘CIA Goes Deep Into Analysis; Agency Opens School, Elevates Analysts.’
[49] Stadther, Denis. Personal interview. 14 May 2001.
[50] That the Kent School dean resides on the DI corporate is from CIA’s internal ‘newspaper’ ‘What’s News at CIA’ Issue number 703. Unclassified article: ‘DDI Establishes Sherman Kent School for Intelligence Analysis. Frans Bax Named Dean.’ Unknown Date.
[51] CIA Website. Tenet Lauds Appointment of McLaughlin as Acting DDCI. Press Release. 29 June 2000.
[52] Most of the information on CAP was provided by its program director Denis Stadther during an interview on 14 May 2001 and a second interview on 26 Feb 2002. All direct information on CAP stems from these two interviews except where otherwise indicated.
[53] Stadther, Denis. Personal correspondence. 6 Sept 2001.
[54] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[55] Stadther, Denis. Personal correspondence. 6 Sept 2001.
[56] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[57] Drogin. “School for New Brand of Spooks.”; Stadther, Denis. Personal interview. 14 May 2001.
[58] Stadther, Denis. Personal interview. 14 May 2001. Re-emphasized on 26 Feb 02.
[59] McLaughlin, John. Personal Conversation. 9 March 2001.
[60] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[61] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[62] For greater information on intelligence tradecraft, see: MacEachin, Douglas J. et al. “The Tradecraft of Analysis: Challenge and Change in the CIA,” Consortium for the Study of Intelligence, Washington DC, 1994. Also see the DI Analytic Toolkit which contains “excerpt(s) from Notes on Analytic Tradecraft, published between 1995 and 1997, which elaborate on some of the skills and methods used by DI intelligence analysts. These notes become a standard reference within CIA for practitioners and teachers of intelligence analysis.” It can be found at: http://www.odci.gov/cia/di/toolkit/index.html
[63] Sherman Kent’s ‘Principles for Intelligence Analysis’ acquired from informational materials provided by Kent School officers 14 May 2001. See appendix for the full text of the document.
[64] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[65] The ‘A/B Team’ approach stems from 1976 when former DCI George Bush commissioned a ‘team’ of outside experts to review the same information that had led to a controversial National Intelligence Estimate on Soviet military spending. This outside ‘B Team’ based its analysis on more pessimistic assumptions of Soviet intentions, interpreted intelligence of Soviet military spending accordingly, and came to a judgment more skeptical than the IC’s that was consistent with reigning conservative views. See: Reich, Robert C. ‘Re-Examining the Team A-Team B Exercise.’ International Journal of Intelligence and Counterintelligence. v. 3, n. 3 (Fall 1989): 387-388. See also: Stack, Kevin P. “A Negative View of Competitive Analysis.” International Journal of Intelligence and Counterintelligence. v. 10, n. 4. 456-464.
[66] Heuer, Richards J. Jr. Psychology of Intelligence Analysis, CSI, CIA, 1999.
[67] Drogin. “School for New Brand of Spooks.”
[68] Ibid.
[69] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[70] Marrin, Stephen. “The CIA's Kent School: A Step in the Right Direction.” The Intelligencer: Journal of U.S. Intelligence Studies. V. 11, No. 2. (Winter 2000). 55-57.
[71] Dan Wagner was a member of CIOC prior to becoming Dean of the Kent School, and provided the information on disciplinary training during an interview on 23 August 2001.
[72] Wagner, Dan. Personal interview, 23 August 2001; Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[73] Wagner, Dan. Personal interview, 23 August 2001.
[74] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001; Stadther, Denis. Personal interview. 14 May 2001.
[75] Shreeve, Thomas W. and James J. Dowd, Jr. ‘Building A Learning Organization: Teaching with Cases at CIA. International Journal of Intelligence and Counterintelligence. Vol. 10, No. 1. 104.
[76] Bax, Frans. Presentation at an AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[77] Campbell, Donald T. and Julian C. Stanley, ‘Experimental and Quasi-Experimental Designs for Research,’ Houghton Mifflin: Boston, 1963. 13-22.
[78] Stadther, Denis. Personal interview. 26 February 2002.
[79] Bax, Frans. Personal interview. 14 May 2001.
[80] Bax, Frans. Presentation at AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[81] Bax, Frans. Presentation at AFIO Luncheon. Fort Myer, Virginia. 22 May 2001.
[82] CIA Website: CIA Vision, Mission, and Values
[83] For more on the limitations of using accuracy to measure intelligence see the strategic surprise and intelligence estimation literature. Of particular note, see: Chan, Steve. ‘The Intelligence of Stupidity: Understanding Failures in Strategic Warning.’ The American Political Science Review. Vol. 73. No. 1 (Mar. 1979). 171-180.
[84] Colby, William. “Retooling the Intelligence Industry.” Foreign Service Journal. Jan 1992: 21.
[85] For more information on the processes of intelligence analysis, see: Clark, Robert M. “Intelligence Analysis: Estimation and Prediction.” American Literary Press Inc. Baltimore, Maryland. 1996.; “Research: Design and Methods.” Joint Military Intelligence College. Washington D.C. September 2000.; Schum, David. “Evidence and Inference for the Intelligence Analyst.” University Press of America, Lanham, Maryland. 1987.
[86] Platt,
Washington. Strategic Intelligence
Production: Basic Principles. Frederick
A. Praeger: New York. 1957. 75.
[87] Clauser, Jerome K. and Sandra M. Weir. Intelligence Research Methodology. State College, PA; HRB Singer, Inc. 1976. 37-46.
[88] Jervis. “What's Wrong with the Intelligence Process?” 29.
[89] Dobbs, Michael. “U.S., Russia At Odds on Iranian Deal.” Washington Post, 15 June 2001, A01.
[90] Katz, Yitzhak and Ygal Vardi. “Strategies for Data Gathering and Evaluation in the Intelligence Community,” International Journal of Intelligence and Counterintelligence. Vol. 5, No. 3, appendix.
[91] Johnson. “Analysis For a New Age.” 661.
[92] Jervis. “What's Wrong with the Intelligence Process?”: 33.
[93] Folker, Robert D. ‘Intelligence in Theater Joint Intelligence Centers: An Experiment in Applying Structured Methods.’ Joint Military Intelligence College Occasional Paper Number Seven. Washington, DC. January 2000. 3.
[94] Katz and Vardi. “Strategies for Data Gathering and Evaluation in the Intelligence Community” 313.
[95] Laqueur, Walter. “The Future of Intelligence.” Society, v. 35, no. 2, (Jan-Feb 1998), 301-.
[96] Platt, Washington. Strategic Intelligence Production: Basic Principles. Frederick A. Praeger: New York. 1957. 151.
[97] Folker. ‘Intelligence in Theater Joint Intelligence Centers.’ 2.
[98] Richards Heuer devotes an entire chapter to this approach entitled “Analysis of Competing Hypotheses.” Heuer. Psychology of Intelligence Analysis. 95-109.
[99] Ibid. 89.
[100] Jones, R.V. Reflections on Intelligence. Heineman: London. 1989. 87-88. The characterization of R.V. Jones is attributed to former DCI James Woolsey at: http://www.odci.gov/cia/public_affairs/press_release/archives/1997/pr122997.html
[101] Davis. ‘Improving Intelligence Analysis at CIA.’ xvii.
[102] MacEachin. “The Tradecraft of Analysis: Challenge and Change in the CIA.” 1.; Davis. “Improving Intelligence Analysis at CIA.” xvii-xix.
[103] Davis. “Improving Intelligence Analysis at CIA.” xix.
[104] The reference to Davis authoring the tradecraft notes is not on the CIA’s website. Instead, the citation was in the foreword of the hardcopy version, which can be found at:
http://intellit.muskingum.edu/intellsite/analysis_folder/di_catn_Folder/foreword.htm Davis. “Improving Intelligence Analysis at CIA.” xix.
[105]
CIA Website. DI Analytic Toolkit.
[106] CIA Website. Intelligence Analysis in the DI: Frequently Asked Questions.
[107] CIA Website. Analytical Products of the DI
[108] For the emphasis in the 1980s on longer papers, see: Johnson, Loch K. “Making the Intelligence “Cycle” Work.” International Journal of Intelligence and Counterintelligence, v. 1, no. 4, (1986): 6-7. For the benefits of longer papers, see: Hulnick, Arthur S. “Managing Intelligence Analysis: Strategies for Playing the End Game.” 322-323; and: Johnson. “Analysis For a New Age.” 665. For the backlash against longer papers, see: Young, Jay T. “US Intelligence Assessment in a Changing World: The Need for Reform,” Intelligence and National Security, v. 8, no. 2 (Apr 1993), 129, 134. For the DI’s change in the 1990s, see: “Say It Ain't So, Jim”: Impending Reorganization of CIA Looks Like Suppression, Politicizing of Intelligence' Publications of the Center for Security Policy, No. 94-D 74, 15 July 1994. For the backlash against shorter papers, see: Gentry. “A Framework for Reform of the U.S. Intelligence Community.”
[109] McLaughlin. “New Challenges and Priorities for Analysis.” 16-17.
[110] Hitz, Frederick P. “Not Just a Lack of Intelligence, a Lack of Skills.” Washington Post. 21 Oct.2001.B3.
[111] Jervis. “What's Wrong with the Intelligence Process?”: 41.
[112] Johnson, Loch K. A Season of Inquiry. Dorsey Press, Chicago, 1988. 197.
[113] Marrin, Stephen. “The CIA's Kent School: A Step in the Right Direction.”
[114] During an interview on 16 May 2001, Ken Olson—Chief of JMITC’s General Intelligence Training Branch—stated that he had used a certification mechanism developed by a part of the Kent School as the concept underlying a proposal for increased training efforts. For examples of JMIC’s publications, see: Garst, Ronald D. ‘Intelligence Types: The Role of Personality in the Intelligence Profession.’ Joint Military Intelligence College. Washington D.C. August 1993; and Garst, Ronald D. and Max L. Gross. ‘On Becoming an Intelligence Analyst.’ Defense Intelligence Journal; 6-2 (1997). 47-59.
[115] Stringer, Kenneth T. Presentation at AFIO Luncheon. Fort Myer, Virginia. 22 May 2001; Swenson, Russell. Personal interview. 8 March 2002.
[116] Pincus, Walter. “Intelligence Shakeup Would Boost CIA.” Washington Post. 8 Nov. 2001. A1.
[117] In fact, in 2001 the Kent School’s former dean noted his preference for a utility measure over an accuracy one. Bax, Frans. Personal interview. 14 May 2001.
[118] Jervis. “What's Wrong with the Intelligence Process?” 30.