Below are the results of the skepticism quiz that I posted over Memorial Day weekend. I collected 71 responses, with scores ranging from 10 to 34. A histogram of the scores is shown below. If you liked the quiz and would like more in the future, let me know here or by subscribing to the website.

The quiz taught me three things:

• Limit the number of questions, especially to make it easier to discuss them later (see below)
• Questions about people tend to be more difficult
• Double check all of the questions and answers!

---

Skepticism Quiz Questions and Answers

I consider the question “easy” if the number of correct responses was 64 or above, “moderate” if the number of correct responses was 50 or above, “difficult” if the number of correct responses was 38 or larger, and “very difficult” for questions with the number of correct responses below 37. Note that this is not a scientific poll, it’s just a quiz for fun.

1. A common piece of “evidence” offered for Bigfoot’s existence is:

• Casts of footprints
• Genetic sequences recovered from hair samples
• Fossilized bones in museums
• Direct skeletal remains

This question was pretty easy, with most people knowing that casts of footprints are commonly offered as evidence for the existence of Bigfoot.

2. Which logical fallacy is committed when someone attacks the person rather than their argument?

• Ad hominem
• Ignoratio elenchi
• Non sequitur
• Petitio principii

This question was also easy.

3. The logical fallacy known as the “straw man” involves which of the following?

• Misrepresenting an opponent’s argument to make it easier to attack
• Distracting from the actual argument by focusing on tone or delivery
• Responding to criticism with personal attacks
• All of the above

This question was moderate difficulty, with “all of the above” catching people. Responding to criticism with personal attacks is the ad hominem fallacy. Focusing on the tone of an argument instead of its content is known as tone policing. It’s not always classified as a fallacy, but tone policing is similar to ad hominem because it diverts attention from the argument to the speaker’s manner of delivery.

4. The “appeal to authority” fallacy is problematic because:

• It relies on the status of a person rather than the strength of the evidence.
• Experts in one field are not necessarily authorities in another.
• Experts often disagree, so it can be hard to know who to trust.
• It’s not a fallacy at all—trusting expert consensus is the thing to do when we are not experts ourselves.

This question was also moderate difficulty. While it is true that experts in one field are not necessarily authorities in another (the second most popular choice), that’s not the appeal to authority fallacy.

5. Which of the following is the clearest example of a “false dilemma” fallacy?

• “We can either support stricter airport security, or risk another terrorist attack.”
• “My laptop crashed right after I installed that update, so the update must be to blame.”
• “Allowing kids to play video games will make them lazy and antisocial.”
• “Nine out of ten people in my office love pepperoni on pizza, so clearly it’s the best topping.”

This question turned out to be on the difficult end of moderate. The update crash answer is a post hoc ergo propter hoc fallacy, also known as false cause or post hoc fallacy. The video game answer is more of a slippery slope or hasty generalization fallacy.

6. Which of the following best describes the concept of “confirmation bias”?

• The tendency to seek out or prefer evidence that supports one’s existing beliefs.
• The tendency to start noticing something everywhere after first learning about it.
• A bias against agreeing with (confirming) any statement whether true or false.
• A pseudoscientific “bias” that Scientologists “measure” using an E-meter.

This one was also easy. The tendency to start noticing something everywhere after first learning about it is called the Baader-Meinhof phenomenon, or the frequency illusion. It is related to confirmation bias.

7. Why is peer review important in science?

• It helps evaluate and improve research quality before publication.
• It catches errors in research prior to publication, ensuring that published research is correct.
• It helps the public better understand scientific concepts.
• It is no longer necessary now that researchers can share findings online.

This was a difficult question. The problem is that while peer review may catch errors in research prior to publication, it doesn’t necessarily ensure that the research is correct.

8. The role of science is to…

• Develop testable explanations and revise them as new evidence emerges.
• Determine facts and prove hypotheses.
• Support established authorities and protect mainstream consensus.
• All of the above.

This question was of moderate difficulty, with “all of the above” catching a few people. “Proving hypothesis” is too strong. All knowledge is subject to revision with more data. And the role of science is definitely not to support established authorities and protect mainstream consensus. Science goes where the evidence leads.

9. Anecdotal evidence…

• Relies on individual stories instead of systematic, controlled data.
• Is only relevant in toxicology or poisoning cases.
• Is based on complex statistical methods that are hard to verify.
• Reflects lived experience, often by marginalized groups whose positions must be respected.

This was one of the easiest questions.

10. A p-value of 0.03 indicates that:

• There is a 3% chance of observing data at least as extreme as the sample, assuming the null hypothesis is true.
• The null hypothesis is “statistically significant.”
• There is a 3% chance that the study will be replicated successfully if the study is conducted again in the future.
• The alternative hypothesis has a 97% probability of being correct.

This question was very difficult. While p ≤ 0.05 is the convention for “statistical significance” in many fields, what the p-value actually indicates is the probability of observing the data being analyzed—or data that are even more extreme—assuming that the null hypothesis is true.

11. In the context of scientific publication, “p-hacking” refers to:

• Exploring many analyses or data combinations and selectively reporting those that produce statistically significant results
• Fabricating or modifying experimental data to achieve statistical significance
• Using Bayesian methods to update p-values post-hoc
• Using blinding to improve the p value

This question was also very difficult. The term “p-hacking” describes conducting multiple analyses of the same data, in a search for a “statistically significant” result. The problem is that the method for calculating “p” assumes a single, pre-specified analysis. Reanalyzing the data in different ways dramatically increases the odds of seeing something that appears to be “statistically significant” but is actually just the product of chance. A nice discussion of p-hacking is here.

12. An experiment’s statistical power is increased by:

• Increasing sample size
• Lowering the α (alpha) level
• Reducing the effect size
• Using one-tailed tests post hoc

This question was easy for almost everyone.

13. Sharing raw data and analysis code benefits scientific research by:

• Allowing independent verification and reanalysis of results
• Increasing visibility and potentially boosting citation counts through broader data availability
• Sharing data does not benefit scientific research, but it can lead to misappropriation
• Replacing the need for peer review

This question was also very easy.

14. Pre-registration of study protocols is primarily intended to:

• Prevent researchers from altering hypotheses after examining the data
• Expedite the peer-review process by streamlining documentation
• Ensure publication in top-tier journals regardless of study outcomes
• Avoid delays caused by Institutional Review Board (IRB) requirements

It’s interesting that this question was pretty easy despite the fact that the p-hacking question (#11) was very difficult. Pre-registration of study protocols is one of the primary techniques for preventing p-hacking. Specifying in advance what hypotheses will be tested, how the data will be collected, and how the analysis will be conducted, reduces the flexibility that enables p-hacking to occur.

15. Which of the following is a key aspect of the scientific method that distinguishes it from pseudoscience?

• Prioritizing empirical evidence and testable, falsifiable hypotheses
• Welcoming claims that cannot be tested or potentially disproven
• Giving weight to personal anecdotes and subjective testimonials as primary evidence
• Avoiding peer review in order to preserve novel or unconventional perspectives

This question was very easy.

16. Who is best known for leading the “Guerrilla Skepticism on Wikipedia” project, which recruits and trains volunteers to improve Wikipedia’s coverage of science and pseudoscience?

• Susan Gerbic
• Daniel Loxton
• Pamela Gay
• Melanie Trecek King

This question was of moderate difficulty. Check out Susan Gerbic’s website, Daniel Loxton’s website, Pamela Gay’s website, and Melanie Trecek King’s website.

17. This investigator has participated in Scientology courses, energy healing, and other fringe practices—all in the name of firsthand research for a long-running podcast on unusual beliefs. Who is he?

• Ross Blocher
• Brian Dunning
• Mick West
• Rob Palmer

This question was difficult. The correct answer is Ross Blocher from the podcast “Oh No, Ross and Carrie (ONRC)” (which ended after 13 years). Brian Dunning’s podcast “Skeptoid” is nearly 20 years old. The key distinction is that Ross and Carrie participated in the activities (so you don’t have to). Also check out Mick West’s work at Metabunk and YouTube and see all the work Rob “The Well-Known Skeptic” Palmer has been doing via Linktree.

18. This science communicator helped demote Pluto, leads the Hayden Planetarium, and frequently critiques pseudoscientific claims in the media. Who is he?

• Neil deGrasse Tyson
• Brian Dunning
• Daniel Loxton
• Rob Palmer

This question was easy. Neil deGrasse Tyson is a science rock star.

19. Who created the Thinking Is Power framework, which teaches students how to evaluate claims and think critically about misinformation?

• Melanie Trecek King
• Brian Dunning
• Pamela Gay
• Susan Gerbic

This question was difficult. I should have included a few new names.

20. Who is known for analyzing and debunking conspiracy theories—particularly those related to UFOs and chemtrails—using logic, evidence, and clear communication?

• Mick West
• Brian Dunning
• Daniel Loxton
• Rob Palmer

This was also a difficult question that should have included different names.

21. Which of the following is a primary goal of the Behrens Science Museum?

• To establish a museum of scientific skepticism.
• To promote scientific inquiry and critical investigation of extraordinary claims.
• To belittle proponents of the paranormal.
• To encourage people to get their vaccinations.

I thought at least one question should promote the Behrens Science Museum. This question was very difficult, with most people choosing the wrong answer. The second answer is essentially the mission statement of the Committee for Skeptical Inquiry (CSI). People should get their vaccinations, but that’s not our purpose, and we are not interested in belittling anyone.

22. Which organization publishes Skeptical Inquirer magazine?

• Committee for Skeptical Inquiry
• James Randi Educational Foundation
• The Skeptics Society
• Behrens Science Museum

CSI publishes Skeptical Inquirer magazine. The Skeptics Society publishes Skeptic magazine. The James Randi Educational Foundation published the online magazine SWIFT. The Behrens Science Museum only has this website.

23. The term “cargo cult science” was coined by:

• Richard Feynman
• Paul Kurtz
• Carl Sagan
• Isaac Asimov

This question was very difficult. You can read about Richard Feynman, Carl Sagan, Paul Kurtz, and Isaac Asimov on Wikipedia (and many other places).

24. Who founded the “Great Moon Hoax” parody website to critique pseudoscience?

• Phil Plait
• Robert Sheaffer
• John Rennie
• Carrie Poppy

This question was impossible because none of the answers are correct! I used generative AI to help create the questions and I failed to fact check this one. The “Great Moon Hoax” was actually a series of six articles published in The Sun newspaper in 1835! Phil Plait has a website which includes a section debunking the Apollo Moon Hoax conspiracy theory, but it’s not a parody. If you answered “Phil Plait” you got credit for a correct answer. John Rennie is a science writer, editor and lecturer who does not have a moon hoax parody website. Carrie Poppy is the “C” in ONRC, discussed above. Sorry about this one!

25. Which of the following best describes a core principle of homeopathy?

• Homeopathy involves giving diluted substances that, in larger amounts, are believed to cause symptoms similar to the illness.
• Homeopathic remedies work by stimulating the immune system with natural plant extracts.
• Products prepared according to the principles of homeopathy contain measurable amounts of active ingredients matched to specific diseases.
• Homeopathy was developed based on ancient Eastern philosophies of energy balance.

This question was medium difficulty, but just barely.

26. Which of the following treatments is not centered on the concept of a “life force”?

• Homeopathy
• Chiropractic
• Reiki
• Acupuncture

This question was difficult, with many people choosing chiropractic instead of homeopathy. The question could have been clearer. Some chiropractic is not centered on “innate intelligence” (a life-force like concept), but it is a foundational concept in chiropractic teachings.

27. Clever Hans teaches us:

• The need for blinding in experiments
• The need for control groups in experiments
• The value of training and positive reinforcement
• That some animals possess extraordinary mathematical abilities

This question was also difficult. The horse Clever Hans responded to cues in the body language of its human trainer, even though the trainer was unaware that he was providing such cues. Blinding ensuring that experimenters or observers do not inadvertently influence experimental results. Clever Hans could not answer questions that his trainer didn’t know the answer to, even if the answer had been given to the horse.

28. The origin of the term “flying saucer” is most closely associated with whom?

• Kenneth Arnold
• Stanton Friedman
• John Mack
• Whitley Strieber

This question was very difficult. Kenneth Arnold is known for reporting the first widely publicized modern (1947) sighting of an unidentified flying object (UFO) in the United States. This sighting led to the coining of the term “flying saucer”. Stanton Freedman was a nuclear physicist turned “flying saucer physicist”. John Mack studied people who claimed to have been abducted by extraterrestrials while he was head of the department of psychiatry at Harvard Medical School. Whitley Strieber wrote the book “Communion” about his alleged alien abduction experience, see below.

29. The book “Communion” was written by…

• Whitley Strieber
• L. Ron Hubbard
• Travis Walton
• Betty Hill

This question was difficult. The book “Communion” was written by Whitley Strieber. It was published in 1987 and featured the iconic image of a space alien on the cover. L. Ron Hubbard created Scientology and other science fiction. In September 1961, Betty and Barney Hill claimed to have been abducted by extraterrestrials. In November 1975, Travis Walton made a similar claim.

30. What was the primary purpose of Project Mogul?

• To detect Soviet nuclear tests
• To develop electromagnetic camouflage techniques for Navy ships
• To calibrate early-warning radar systems
• To test high-altitude parachute systems for military personnel

Project Mogul was a top-secret project to detect Soviet nuclear weapons tests using microphones lifted by high-altitude balloons. A Project Mogul balloon was recovered near Roswell, New Mexico in July 1947.

31. Who wrote the Gospels of the New Testament Bible?

• Anonymous authors
• The apostles Matthew, Mark, Luke, and John
• Jesus
• Paul the Apostle

This question was moderately difficult, with most common incorrect answer being “the apostles Matthew, Mark, Luke, and John”. The consensus of Biblical scholars is that the Gospels were written by anonymous authors, and that the names of the apostles were associated with them later. See for example, NT Pod 83 by Mark Goodacre and the book Forged by Bart Ehrman.

32. Which book posits that ancient aliens visited Atlantis?

• Chariots of the Gods?
• Mission Earth
• The Invaders
• Earth in Upheaval

The book “Chariots of the Gods?” was written by Erich von Däniken and published in 1968. “Mission Earth” was written by L. Ron Hubbard. It did not focus on Atlantis or ancient aliens. “The Invaders” is a 1967 TV series about aliens. “Earth in Upheaval” is a book by Immanuel Velikovsky that is not about aliens or Atlantis.

33. The 1781 painting “The Nightmare” by Henry Fuseli is now often interpreted by skeptics as an early depiction of:

• Sleep paralysis
• A manic-depressive episode
• A hypnagogic hallucination ritual
• Alien abduction

The correct answer is sleep paralysis. Hypnagogic hallucinations are related, but the word “ritual” makes that answer wrong.

34. The creature in the movie Species (1995) fits the original description of which creature:

• Chupacabra
• Skinwalker
• Slenderman
• Mothman

This question was very difficult. The book “Tracking the Chupacabra” theorizes that the original chupacabra sighting was inspired by the creature “Sil” in the movie “Species“.

35. Which group predicted a mass landing of 33 spaceships in El Cajon, California, to usher in a new era of enlightenment?

• The Unarians
• The Raëlians
• Heaven’s Gate
• The Aetherius Society

This was the most difficult question, with only six people correctly choosing the Unarians.