The Docsplainer

As I reported Monday , the National Institutes of Health (NIH) cancelled a major, cutting edge study into Alzheimer’s disease among African Americans, on the putative grounds that it relied on self-report data, which director Jay Bhattacharya has characterized as unreliable (Glenza, 2026). I said then we’d talk about this more today, and so here we are.

Aside from the fact that NIH has historically — and happily — funded a gazillion studies in a number of fields, including work outside the social sciences, that gathered self-report data, setting aside the fact that much of the government’s own data-gathering is self-report (Census, anyone?), it is not unreliable per se.

Self-report (i.e., questionnaires you fill out, paper-and-pencil personality tests, surveys, and the like) definitely have their faults. But they are known faults and good researchers have empirically-validated means of compensating for them. And for various reasons, much of psychological research is based on self-report. So claiming that you can’t use tests that themselves have been validated independently of the studies in which they are used is just, pardon me, bullshit (Wood, 2026).

So let’s talk truth.

What is a self-report study?

A self-report study is the only way to get data directly from people. It is a type of observational study, which is used when it would be unethical to run an actual experiment — a point this administration either does not understand or chooses to ignore.

What is a self-report measure?

Questionnaires, surveys, structured interviews, and paper-and-pencil tests are the only way to conduct a self-report study that results in statistical data. You can’t observe the information from the outside.

Social scientists use them because they are cheap and easy to administer so that you can poll large numbers of people and thus hope to get more representative samples. They are not easy to design or interpret, however, so–again–the administration slamming this as sloppy science is unfair.

Sometimes (think Kinsey) self-report studies are combined with more experimental procedures and measures, as they were in the Presh study: The experiment involved comparing neighborhoods with different levels of access to food, the measure being outcomes like Alzheimer’s rates in the two samples. And then there were the interviews and questionnaires (Glenza, 2026).

At other times, they may precede a true experiment, as when researchers are doing an exploratory sort of study before refining their theories and hypotheses and designing their experiments.

What are the risks with self-report studies/measures?

As is often the case with such political maneuvers, Bhattacharya is a little bit right. Self-report has known flaws.

1. Unlike an experiment, it cannot reveal cause and effect. (This would not be problem, only a shortcoming, if people didn’t consistently misinterpret correlational relationships as causal relationships.)
2. Self-report bias includes people generally wishing to cast themselves in a good light, not wanting to reveal illegal behavior even if they know the data is anonymized, or, conversely, people with self-esteem issues consistently casting themselves in an unrealistically negative light.
3. There are also known tendencies that people may have such as
   1. consistently choosing the extreme response on questions that give you a range from, for example, “Completely Agree” to “Completely Disagree”. Some people consistently gravitate toward the negative, others toward the positive.
   2. Still other folks refuse to commit, and consistently choose the center option, as if to say, don’t know, don’t care, whatever…
4. People with mental problems or reading difficulties can answer inconsistently, invalidating the whole thing.

How do researchers mitigate those risks?

Briefly, just a couple of things you can do include:

1. Eliminate the middle option altogether (forced choice).
2. Swap ends with the extreme negative/extreme positive choices from question to question, so there’s that split second the brain has to take to think about the response.
3. ‘Salt’ your test or questionnaire with questions that, if not answered honestly, enable you to adjust scores for self-report bias.
4. Ask the same thing twice in different ways near the beginning and end of your test or survey to flag inconsistent responders.
5. Pilot-test your questions to be sure the phrasing isn’t pulling for certain answers, that the questions are confusing, the options incomplete, etc.

Big, long tests like the Minnesota Multiphasic Personality Inventory (MMPI) have entire scales of questions that cover the first four options, and it has been extensively pilot-tested and continues to be tested (Mitigation No. 5). Each scale contributes to detecting one sort of bias or another, and there are mathematical formulae for adjusting final scores accordingly, so that tests that would otherwise be deemed invalid (or worse, go undetected as invalid) can be ‘corrected’ and still be useful.

Researchers have gone into depth on the issue in their grad school studies. Heck, there are scientists who study nothing else but how to design studies, how to design self-report measures to get the most accurate, objective results possible, and how to interpret results (statistical procedures). (For a thorough review of the field, see Science Direct’s web pages on the topic.) Peer reviewers consider it with every research study they critique for major scientific journals. The NIH certainly considers it when they agree to fund a study in the first place, and to continue funding for long-term studies like Presh year over year. For heaven’s sake, first-year college students cover this in Intro to Psych and other gen ed courses in the social sciences. I taught it every semester (Grison et al., 2026).

We all rely on self-report every day of the week to do our jobs and obtain services, even to know how to vote:

• Large organizations survey politicians and publish comparisons of where who stands on what. That’s not all self-report, sometimes it’s their voting records, but.
• Colleges and universities survey every student at the end of every semester, in every class, asking questions about the professor, the course, and the text, surveys that help determine what will be taught and by whom, who will not be retained, and who will get promoted. Most large corporations do the same.
• When you sit down with your doctor to discuss your current health status, at the very least your self-report will guide her in her selection of labs and other tests to try to narrow down the problem. Your lawyer needs you to self-report everything you know about your case before they can do the first thing to help you.
• Pollsters rely on self-report to tell us what the political temperature of the nation is at any given time.
• The Census is one giant self-report study that we have used since before the Civil War to determine all sorts of program and policy decisions. We use it to outline voting districts.

In other words, you can’t just throw out self-report studies! We live, eat, and breathe the darn things.

So if the administration does not understand the basics I’ve outlined here, it is only because of its appalling habit of appointing people with zero training or experience in the departments they are supposed to be running, from defense to education to foreign policy, health and welfare and the list goes on. This is why we teach it, why colleges and universities demand that students acquire a general education, including social science courses, before they begin to concentrate on their majors. It’s absolute rock-bottom minimum knowledge required to develop informed voters and it empowers people to make rational decisions in other areas of their lives, from child-rearing to nutrition to work.

Or hey. Maybe these guys do know.

If this was helpful in any way, please… buy me a coffee! It helps keep the African Grey Parrot in kibble.

And if there’s any part of this that you would like to read more about in the coming weeks, let me know in the Comments.

---

References

Experimental Design—An overview | ScienceDirect Topics. (n.d.). Retrieved September 23, 2026, from https://www.sciencedirect.com/topics/social-sciences/experimental-design

Glenza, J. (2026, September 18). ‘Clear retribution’: Dismay as key Alzheimer’s study cancelled amid Trump attacks on science. The Guardian. https://www.theguardian.com/us-news/2026/sep/18/pittsburgh-trump-funding-cuts-nih-alzheimers

Wood, V. S. (2026, September 21). Another important study cancelled. The Docsplainer. https://docwood.wordpress.com/2026/09/21/another-important-study-cancelled/