A new study finds that previous studies of mental illness using brain scans may be too small to trust the results.
Andrew Brooks/Getty Images/Image Source
hide caption
toggle caption
Andrew Brooks/Getty Images/Image Source
A new study finds that previous studies of mental illness using brain scans may be too small to trust the results.
Andrew Brooks/Getty Images/Image Source
MRI scans have given researchers a peek inside the human brain. The technology is also great at revealing stroke damage and areas that glow when you look at your face.
However, brain scan studies have yet to provide much insight into traits such as intelligence and the underpinnings of mental health conditions such as anxiety and depression.
The main reason is that these studies need to include scans of thousands of brains, rather than the dozens typically used, the team reported in the March 16 issue of the journal. Nature.
“You need a very large sample, and the bigger the sample, the better,” says Nico Dosenbach, M.D., an associate professor of neurology at Washington University in St. Louis and the study’s author.
That’s a lesson the field of genetics is already learning, says Paul Thompson, a neuroscientist at the University of Southern California.
“Twenty years ago, I heard that someone had found a gene for criminality, a psychosis, a gene for autism, but another group didn’t find the same one, or they found another gene, and they didn’t. I would have scratched my head.”
Geneticists have finally solved the problem by expanding their studies from dozens to hundreds to millions, Thompson says. Neuroscientists now appear to be in a similar position and should reconsider the results of many small studies.
A quest to illuminate the intellect
A new paper on brain scan research stems from a 2018 effort to understand how children develop cognitive abilities, also known as intelligence.
The team, which includes Scott Marek, a researcher in Dosenbach’s lab at the University of Washington, planned to use data from a federal study that scanned the brains of thousands of adolescents.
“What we wanted to do was use this huge sample to ask the question: How are cognitive abilities represented in the brain?”
Previous studies have found that intelligence is strongly related to the thickness of the outermost layer of the brain and the strength of connections between specific brain regions.
So Marek’s team analyzed about 1,000 brain scans from federal studies. We then checked our work using 1,000 different scans.
“What we realized was that you can’t replicate everything,” says Marek. “I couldn’t see very well.”
Areas or connections that appear important in one set of scans may appear unimportant in another set of scans. It was only when the sample size was increased to thousands of brains that the results became more reliable.
The team wondered if this was true for other studies that searched the whole brain for differences related to complex problems such as anxiety, depression and ADHD.
So they took brain scan data from about 50,000 people and used computers to conduct a number of large and small simulation studies. Again, the team found that thousands of scans were required to get reliable results.
This has been troubling because, for years, much smaller samples have been used to produce a series of scientific papers on mental illness and behavioral disorders.
So far, the research “has not really been tangible for patients,” Dosenbach said.
Risk of small samples
One problem with small studies is that they only find brain features that have a relatively large impact on mood, behavior, or mental performance. For example, in Alzheimer’s disease, it is easy to show that hippocampal atrophy is accompanied by dramatic memory loss.
Brain differences associated with mental illness tend to be far less obvious and far more controversial. I know However, the strength of the correlation varies greatly between studies. Also, there is no way to know how a person is feeling by looking at their frontal lobe activity.
Another problem with small studies is what is called publication bias.
“If multiple groups are doing similar studies with small samples, chances are that one or more of them will have important results,” says Dosenbach. “And that is what is reported.”
When these studies are well published, misleading findings can become conventional wisdom. However, this does not mean that small studies are necessarily wrong.
“Even a little research can hold the truth,” he says. “The chances of that happening are just much, much, much smaller than in a very large study.”
Therefore, the public should be wary of headings that extrapolate results from small MRI studies to the general population.
Research using “aftershocks”
Many brain scientists are trying to make sense of the news that human behavioral studies may require thousands of scans.
“It’s a bit like the Los Angeles earthquake,” says UCLA’s Thompson. “It sent some aftershocks through the neuroscience community.”
But Thompson says the solution is clear and achievable. He combines scans from many small studies into one or more large databases and verifies the results.
The ENIGMA Consortium, which Thompson helped create, is one effort to facilitate this. The group maintains a database of over 50,000 MRI scans. Scientists are already using it to identify brain differences associated with schizophrenia.
“In schizophrenia, there are big differences across the brain,” says Thompson. “The auditory center involved in hallucinations is abnormal. There are changes in the memory system, the visual system.”
However, the differences are much more subtle, so larger studies may be needed to find brain regions and associations associated with psychiatric disorders such as depression and bipolar disorder.
Some of those studies are already underway.
For example, a National Institutes of Health study of adolescent brain development enrolled more than 11,000 adolescents and regularly scanned their brains to track changes.
Terry Jernigan, a brain scientist at the University of California, San Diego, and one of the study’s principal investigators, says the study’s large size is part of an effort to address problems found in smaller studies. It is said that the department
But it’s not enough to include thousands of people in brain-scanning studies, Jernigan says, and studies must be more diverse than usual.
“We want to know to what extent your observations generalize to all groups in our society,” she says.
