US Army, Martin Seligman "CSF Research Fails the Test"

Last week, psychologists Stephen Soldz and Roy Eidelson published their analysis of the Army's Comprehensive Soldier Fitness Program. It's a professional look at the much-hyped program that finds it seriously wanting. A press release by Coalition for an Ethical Psychology details their findings, and I've reposted it below.

Their report, "Does Comprehensive Soldier Fitness Work? CSF Research Fails the Test" (PDF), seriously dismantles the structure and implementation of the Army's program, while understanding the rationale -- to lower rates of mental illness, suicide, and PTSD among enlisted personnel -- is both meaningful and important.

The conclusions of the report were foreshadowed in a March 2011 Psychology Today article by Eidelson, Soldz and Mark Pilisuk, "The Dark Side of 'Comprehensive Soldier Fitness'". The PT article stressed ethical concerns with the CSF program, particularly the fact that it constituted a research program, but that soldiers were not given informed consent regarding their participation, and no Institutional Review Board had reviewed the program.

In January 2011, Jason Leopold at Truthout published an investigation describing criticism of CSF from those who found its emphasis on "spiritual fitness":

CSF is comprised of the Soldier Fitness Tracker and Global Assessment Tool, which measures soldiers’ “resilience” in five core areas: emotional, physical, family, social and spiritual. Soldiers fill out an online survey made up of more than 100 questions, and if the results fall into a red area, they are required to participate in remedial courses in a classroom or online setting to strengthen their resilience in the disciplines in which they received low scores. The test is administered every two years. More than 800,000 Army soldiers have taken it thus far and more than 100,000 soldiers have participated in the remedial training.

But for the thousands of “Foxhole Atheists” like 27-year-old Sgt. Justin Griffith, the spiritual component of the test contains questions written predominantly for soldiers who believe in God or another deity, meaning nonbelievers are guaranteed to score poorly and will be forced to participate in exercises that use religious imagery to “train” soldiers up to a satisfactory level of spirituality.

Griffith, who is based at Fort Bragg, North Carolina, took the test last month and scored well on the emotional, family and social components. But after completing the spiritual portion of the exam, which required him to respond to statements such as, “I am a spiritual person, my life has lasting meaning, I believe that in some way my life is closely connected to all humanity and all the world, ” he was found to be spiritually unfit because he responded by choosing the “not like me at all” box.

The Soldz/Eidelson paper does not focus on this "spirituality" critique, but on more technical matters of quantification of effectiveness, and the self-promotion aspect of program. Their work includes a "Technical Appendix" for those who wish to follow the statistical and methodological arguments.

The critique of self-promotion includes a look at the role of psychologists Martin Seligman and the American Psychological Association in promoting this shoddy program, dressing it up with the language of science, and reaping millions of dollars for those who are contracting with the government to implement the program. In October 2010, Mark Benjamin at Salon revealed Seligman's Positive Psychology Center at the University of Pennsylvannia was the ultimate recipient of a $31-million no-bid contract for the Army's resiliency program.

Seligman, of course, is best known as the primary theorist of "learned helplessness," a theory of the total psychological break-down of animals or humans due to uncontrollable stress. "Learned helplessness" (LH) became one of the primary theories behind the use of certain torture techniques used by DoD/CIA psychologists after 9/11. LH was taught to incoming members of the Behavioral Consultant Science Teams (BSCT) used by DoD to assist interrogations of "war on terror" detainees at Guantanamo and elsewhere. Behavioral Science Consultants are psychologists or psychiatrists, and they are still used in interrogations to this day. A former trainer for BSCT teams is today the Chief Clinical Officer of the Washington DC Department of Mental Health.

Finally, there is the question of just what this program is actually trying to do. Reducing PTSD rates is one thing, but producing "indomitable" soldiers who can fight brutal wars without psychic damage to them is another, for it presents an unrealistic view of what war actually is, and hides the fact it seriously damages the mental health and psychic coherence of those who engage in it. (Of course, war outright kills untold tens of millions, both soldier and civilian, and physically damages tens of millions of individuals more.)

In a report released today by the Coalition for an Ethical Psychology (http://www.ethicalpsychology.org/Eidelson-&-Soldz-CSF_Research_Fails_the_Test.pdf), two psychologists call upon the Army to retract or publicly correct a recent research report that claims the Army’s $140 million Comprehensive Soldier Fitness (CSF) resilience program “works.” The psychologists Roy Eidelson and Stephen Soldz argue that the study design is flawed and that the results do not justify the researchers’ favorable conclusions.

Report coauthor Roy Eidelson stated: “The over-hyping of CSF’s effectiveness should be of concern to everyone, including taxpayers who have paid over $100 million for the program, and especially the one million soldiers who are forced to participate in this massive experiment, whether they want to do so or not.”

Without pilot testing, the CSF program was launched in 2009. It trains soldiers in thinking skills that purportedly diminish the likelihood of suffering post-traumatic stress disorder (PTSD), depression, suicide, and other combat-related psychological problems. CSF is based upon the “positive psychology” framework of University of Pennsylvania psychologist Martin Seligman.

In their new report Eidelson and Soldz identify five areas of serious concern with the Army’s CSF evaluation: (1) the researchers’ failure to measure the important outcomes of PTSD, depression, or other psychological disorders despite the availability of validated measures for doing so, (2) a flawed research design that fails to control for important confounding variables, (3) significant problems with the method of data analysis, (4) the researchers’ failure to acknowledge plausible risks of harm from the CSF intervention, and (5) miscellaneous related issues of concern. Individually these concerns raise troubling questions regarding the CSF study. Taken together, they severely undercut the CSF researchers’ assertion that “There is now sound scientific evidence that Comprehensive Soldier Fitness improves the resilience and psychological health of Soldiers.”

Stephen Soldz, Professor at the Boston Graduate School of Psychoanalysis and report coauthor, noted that CSF has been the subject of a wide range of criticism since it was rolled out in 2009: “The problems identified with CSF are legion. It is time for the Army to step back from uncritically promoting this untested program. A careful, independent, evaluation is urgently called for.”

Concerns raised by critics in the past span a wide range of significant issues, including indications that CSF is actually a research study involuntarily imposed upon troops without mandated protections such as independent ethical review by an institutional review board (IRB) and informed consent; the possibility that CSF may serve as a distraction from the documented adverse effects of multiple and lengthy deployments and high levels of combat exposure; potential negative effects of CSF, common in prevention programs, that have not been carefully considered or monitored; and the insufficient examination of ethical questions posed by efforts to build “indomitable” soldiers.

This new Coalition report follows a detailed critique last year of CSF by Eidelson, Soldz, and their colleague Marc Pilisuk, The Dark Side of Comprehensive Soldier Fitness, which led to Congressional inquiries regarding the program. CSF has also been criticized in a series of comments in the October 2011 issue of the American Psychologist and by experts interviewed by the PBS NewsHour and other press.

About the research weaknesses they identify, Eidelson and Soldz conclude in their report:

“These scientific shortcomings are all the more troubling given the obvious importance of what is at stake here: soldiers’ welfare. It may be comforting to some to assume that, at worst, CSF is merely ineffective. However, we should not settle for such wishful thinking. It is not outlandish to suggest that CSF may negatively impact some soldiers, and unjustified enthusiasm about the program can prove costly in terms of directing attention and funding away from the consideration and development of alternatives that may be far more beneficial for our troops.

“It is not hard for us to imagine the tremendous pressures faced by those responsible for addressing and protecting the psychological health of the men and women who serve in our military. We recognize and admire the dedicated work of so many toward this goal. But in the search for answers, nobody benefits from research that, inadvertently or not, misrepresents the current state of knowledge and accomplishment in this arena. For this reason, we believe it is essential that the Comprehensive Soldier Fitness leadership correct the record in regard to their Research Report #3.”

For more information go to: http://www.ethicalpsychology.org

Both Eidelson and Soldz are past presidents of Psychologists for Social Responsibility. Dr. Eidelson is also the former executive director of the Solomon Asch Center at the University of Pennsylvania, and today is the president of Eidelson Consulting. Dr. Soldz is Director of the Center for Research, Evaluation, and Program Development at the Boston Graduate School of Psychoanalysis. He was also a contributing writer on the Physicians for Human Rights report, Experiments in Torture: Human Subject Research and Experimentation in the “Enhanced” Interrogation Program.