how to determine if a controversial statement is scientifically true

8/13/2019 How to Determine if a Controversial Statement is Scientifically True

1/9

How to Determine If A Controversial

Statement Is Scientifically True

Every day, we're confronted with claims that others present as fact. Some are easily debunked, someare clearly true, and some are particularly difficult to get to the bottom of. So how do you determine ifa controversial statement is scientifically true? It can be tricky, but it's not too difficult to get to thetruth.

Every internet user has developed a healthy dose of skepticism that keeps us from being duped bythings that don't pass thesmell test,but it's not enough to just thinksomething might not be true.What if you think the statement might be true and you want to learn more? What if you want torespond to the assertion or engage in conversation but you don't know enough to do so?

We sat down with experts Dr. Phil Plait, akaThe Bad Astronomer,and David McRaney ofYou Are NotSo Smartto figure out a working approach to discovering the truth of any statement, from obvioushoaxes (thinkNigerian prince emails)to more difficult topics (thinkvaccine "controversies".)
http://lifehacker.com/5919830/how-to-determine-if-a-controversial-statement-is-scientifically-truehttp://lifehacker.com/5919830/how-to-determine-if-a-controversial-statement-is-scientifically-truehttp://lifehacker.com/5919830/how-to-determine-if-a-controversial-statement-is-scientifically-truehttp://en.wiktionary.org/wiki/smell_testhttp://en.wiktionary.org/wiki/smell_testhttp://en.wiktionary.org/wiki/smell_testhttp://blogs.discovermagazine.com/badastronomyhttp://blogs.discovermagazine.com/badastronomyhttp://blogs.discovermagazine.com/badastronomyhttp://youarenotsosmart.com/http://youarenotsosmart.com/http://youarenotsosmart.com/http://youarenotsosmart.com/http://www.snopes.com/fraud/advancefee/nigeria.asphttp://www.snopes.com/fraud/advancefee/nigeria.asphttp://www.snopes.com/fraud/advancefee/nigeria.asphttp://en.wikipedia.org/wiki/Vaccine_controversieshttp://en.wikipedia.org/wiki/Vaccine_controversieshttp://en.wikipedia.org/wiki/Vaccine_controversieshttp://en.wikipedia.org/wiki/Vaccine_controversieshttp://www.snopes.com/fraud/advancefee/nigeria.asphttp://youarenotsosmart.com/http://youarenotsosmart.com/http://blogs.discovermagazine.com/badastronomyhttp://en.wiktionary.org/wiki/smell_testhttp://lifehacker.com/5919830/how-to-determine-if-a-controversial-statement-is-scientifically-truehttp://lifehacker.com/5919830/how-to-determine-if-a-controversial-statement-is-scientifically-true


2/9

SEXPAND

First, Learn to Avoid Confirmation Bias

Before we get into what you should do when confronted with a statement you're curious about, thefirst thing you have to strip yourself of isconfirmation bias.Says Dr. Plait:

The biggest problem is one of confirmation bias: finding an answer you already believe. If someonehas a question about a belief or opinionsay, that vaccines are dangerousthen when they look itup online they'll tend to be biased toward sites that have information they already agree with! Thisis a well-known effect, and is one reason some things, like anti-vaccination beliefs, are strong evenin well-educated communities. The people are smart enough to look up and understand what theyread, but perhaps not experienced enough in critical thinking to evaluate what they're readingwithout bias.

So how do you beat back confirmation bias? "Even with experience, it's incredibly tough to do," Dr.Plait explained. First, be aware that confirmation bias exists, shake yourself of your natural tendencyto draw a conclusion before you've researched a topic, and be open to information that falls on eitherside of a statement. Don't just demand someone elsepresent studies that support their assertiongolooking for them yourself.This must-read by David McRaney tackles the topic of confirmation bias indetail,but in short: Keep an open mind, seek evidence to the contrary for every opinion (especiallyones you believe), and don't treat your research like a crusade.Photo byYi Chen.
http://en.wikipedia.org/wiki/Confirmation_biashttp://en.wikipedia.org/wiki/Confirmation_biashttp://en.wikipedia.org/wiki/Confirmation_biashttp://youarenotsosmart.com/2010/06/23/confirmation-bias/http://youarenotsosmart.com/2010/06/23/confirmation-bias/http://youarenotsosmart.com/2010/06/23/confirmation-bias/http://youarenotsosmart.com/2010/06/23/confirmation-bias/http://www.flickr.com/photos/yiie/4865201576/http://www.flickr.com/photos/yiie/4865201576/http://www.flickr.com/photos/yiie/4865201576/http://www.flickr.com/photos/yiie/4865201576/http://youarenotsosmart.com/2010/06/23/confirmation-bias/http://youarenotsosmart.com/2010/06/23/confirmation-bias/http://en.wikipedia.org/wiki/Confirmation_bias


3/9

Your First Line of Defense: Search Google, Snopes, and

Other Popular Web SitesYour first instinct when confronted with a statement that seems controversial is probably to hit Googleand start looking around for more information on it. That's good: it's the first thing that Phil Plait andDavid McRaney both suggest you do as well. In fact, you can prove and disprove many statements andresearch other contentious journal articles just by doing a little legwork and seeing what the web hasto say.

"If you are skeptical of a claim or a factoid or a quote or whatever, try some Google-fu that includesthe word "skeptic" or "hoax" or "bogus" or "rumor" or "urban legend" with your search term. If you arelucky, you might find existing discussions in the forums of skeptical societies or science groups,"McRaney suggested. Dr. Plait suggested the same, but noted, "This is becoming less useful, though, aspeople on the, um, wrong side of reality have taken up the mantle of skeptics.' That's irritating. And

you can't always trust experts, since sometimes their credentials are inflated, or they're simply wrong(I can name several Nobel laureates who have said probably wrong things about their own beliefs)."

So if Google doesn't seal the deal, and you're still awash in more opinions than facts, what do you do?Here are a few good places to visit if you're on the hunt for good, unbiased information:


4/9

Snopes:Snopes.com has been the rumor debunker of the internet for years, and despite the occassional

claim against its own impartiality, it's an excellent resource. "The first place I go when I'm in a hurry isSnopes," McRaney said. "I'm often surprised at how quickly they cover a topic that is bouncing around inthe echo chamber." He also noted that Snopes lists its sources, so you can always jump off and do morereading if you want more data.

Wikipedia:It's not perfect, but Wikipedia does have a wealth of information, and in most cases, thatinformation is sourcedwhich means that it's less useful for its own articles, and more useful for thesourced links to any evidence you find.

Science Daily:While more news and less rumor-debunking, Science Daily has a huge database ofarticles to search, most of which link off to the source journal articles and studies that may have startedthe rumor in the first place.

Phys.Org:Again, Phys.Org is primarily a news site, but they've dedicated plenty of articles to debunkingpopular rumors or addressing trending topics in science and medicine. Plus, they also link to theirsources if they can.

Finally, when you're reading news articles and sites, even if they link to their supporting studies, trytopry apart the article from the studies.Speaking as someone who's spent a lot of time in the scientificand media communities, both camps almost always mean well when trying to share news or studieswith the publicit's just incredibly difficult to get those stories out in a digestible, commonlyunderstood way without losing the science in the message. If you're skeptical, don't dismiss the storyor the studylook deeper and draw your own conclusions.
http://snopes.com/http://snopes.com/http://www.wikipedia.org/http://www.wikipedia.org/http://www.sciencedaily.com/http://www.sciencedaily.com/http://phys.org/http://phys.org/http://blogs.scientificamerican.com/guest-blog/2012/03/18/a-fun-diy-science-goodie-proof-yourself-against-sensationalized-stats/http://blogs.scientificamerican.com/guest-blog/2012/03/18/a-fun-diy-science-goodie-proof-yourself-against-sensationalized-stats/http://blogs.scientificamerican.com/guest-blog/2012/03/18/a-fun-diy-science-goodie-proof-yourself-against-sensationalized-stats/http://blogs.scientificamerican.com/guest-blog/2012/03/18/a-fun-diy-science-goodie-proof-yourself-against-sensationalized-stats/http://phys.org/http://www.sciencedaily.com/http://www.wikipedia.org/http://snopes.com/


5/9

The Big Guns: Search Public Journals and Contact Science

Advocates

So you've done some Google searching, looked at a few skeptical sites, but you want more information.That's great! It's time to go to the source: peer-reviewed journals, and people you can trust about

science. One thing I like to do before I write about or share any news article that's based on a study ofsome kind is to see if I can get to the study itself. Is it mentioned in an article, or did someone say "astudy said X?" Fire Google back up and include the journal name and publication date, if you can."Include "research" or "evidence" or "study" in your search terms. With enough digging, you will oftenfind several scientific papers related to your topic," McRaney suggests. Read the full-text if it'savailable, or at least the abstract. It'll help you get a picture for what the study reallyconcludes.

Of course, you'll probably run into many studies that are locked up behind paywalls. McRaneysuggests usingGoogle Scholar,a search engine just for peer-reviewed journals. "Just copy the fullname of the paper you want and paste it into Google Scholar's search bar. Look for .pdf versions, asthey will most likely be the full paper." If you still can't find it, just email one of the study's authorsand ask for a copy. It may sound crazy, but "they are usually more than happy to oblige, and you'llprobably put a spring in their step for a full 24 hours."

Finally, you can always ask the experts you look up to for their thoughts on a topic. Dr. Plaitexplained: "I get questions often about topics I'm not all that familiar with. I actually have a long list ofpeople I know who are experts in various topics, and whose opinions I trust, so I will sometimes emailthem and ask. Not everyone has a stable of experts in their contacts list though!" He went on toexplain that there aresome people you can trust: researchers, medical professionals, and so on. Seekthem out, ask your questionsoften they won't be able to resist weighing in on a hot topic, even ifyou're asking out of the blue. McRaney pointed out that theAsk Science subreddit at Redditcovers a

lot of popular, contentious topics from multiple angles. See if your statement has been debated. If not,bring it up yourself.
http://scholar.google.com/http://scholar.google.com/http://scholar.google.com/http://www.reddit.com/r/asksciencehttp://www.reddit.com/r/asksciencehttp://www.reddit.com/r/asksciencehttp://www.reddit.com/r/asksciencehttp://scholar.google.com/


6/9

Extra Credit: Visit Your Local Library and Consult

Librarians and Reference Materials

You've done your homework, looked up some studies, read both sides of an argument, and you're stillnot sure what to believe, or if there's enough information to believe anything. That's greatyou're still

hungry for information, and there's one place left to get it: your local library.Photo byManchester CityLibrary.

If you catch yourself unable to download a specific study, or the study is so old (or too new!) that it'snot available, or you just want help getting to the bottom of an issue, visit your library's referencedesk. Often, public librariesand especially university librarieshave free access to scholarly journalsand their archives so you can download, print, and read full-text articles you wouldn't be able to get athome. Even many university libraries only require student ID if you're going to check something out,so they're a great resource for everyone.

"Most university librarians will happily provide you a copy of a paper if you or someone you know isenrolled in the university," McRaney adds. If you are going to chat up your local reference librarian,see what they think of the topic, and if they can do some digging on your behalf. Most often, they can

do some research for you and present you with findings to read through, or they can at least helpguide you to authoritative sources on the topic.
http://www.flickr.com/photos/manchesterlibrary/4690304527/http://www.flickr.com/photos/manchesterlibrary/4690304527/http://www.flickr.com/photos/manchesterlibrary/4690304527/http://www.flickr.com/photos/manchesterlibrary/4690304527/http://www.flickr.com/photos/manchesterlibrary/4690304527/http://www.flickr.com/photos/manchesterlibrary/4690304527/


7/9

A Last Word: Research Is Great, but Don't Forget to Think

Critically

All of the research in the world won't help educate you on a controversial issue or statement if you

don't look at the evidence and try to draw your own conclusions from it. "What I can say is that if youwant to know what's what, the best thing to do is approach the question honestly and openly," Dr.Plait explains. "Read up on opinions for and against. Do the basic arguments make sense? Are thearguments simply ad hominems (attacks against the other side) or is there actual evidence backing upthe claims? And what does the other side have to say about those claims?" Image byImage bySMBC.

This isn't perfect though, and both McRaney and Dr. Plait warned that there are a few things to watchout for when reading journals and articles that reference journal articles, or when you're talking toscience advocates:

Watch out for anecdotal evidence. The problem with anecdotal evidence iswell documented,and whenpresented with it, you should make a concerted effort to ignore it, or at least take it into lesser accountthan rigorous research. It's difficult: our minds are wired to exaggerate the value of anecdotal experience,

but if you'rethinking and reading critically,it's important to discount them. Watch out for scientists who are anything but. Most scientists know their field well, but you still have to

think critically about what you hear. Someone claiming to be knowledgeable may not be at all. "Theproblem there again is that there are people who claim to be in this group and aren't," Dr. Plait warned."Or worse, ones who actively promote conspiracy theories against scientistsand those are legion.You've no doubt seen thesepeople who say scientists are lying to us, or that Big Pharma is paying them,or whatever. I have no easy solution for things like that."

Be careful which science advocates you trust. The best science advocates have backgrounds in sciencethemselves, are passionate about science, and are willing to engage openly and non-defensively about agiven topic. The worst tend to behave like conversations about research are political or religious ones,and roll out the ad hominem attacks at the first sign of dissention.
http://www.smbc-comics.com/index.php?db=comics&id=2088#comichttp://www.smbc-comics.com/index.php?db=comics&id=2088#comichttp://www.smbc-comics.com/index.php?db=comics&id=2088#comichttp://thobbs.github.com/blog/2012/06/17/you-should-downvote-anecdotes/http://thobbs.github.com/blog/2012/06/17/you-should-downvote-anecdotes/http://thobbs.github.com/blog/2012/06/17/you-should-downvote-anecdotes/http://lifehacker.com/5872944/how-to-boost-your-reading-comprehension-by-reading-smarter-and-more-conscientiouslyhttp://lifehacker.com/5872944/how-to-boost-your-reading-comprehension-by-reading-smarter-and-more-conscientiouslyhttp://lifehacker.com/5872944/how-to-boost-your-reading-comprehension-by-reading-smarter-and-more-conscientiouslyhttp://lifehacker.com/5872944/how-to-boost-your-reading-comprehension-by-reading-smarter-and-more-conscientiouslyhttp://thobbs.github.com/blog/2012/06/17/you-should-downvote-anecdotes/http://www.smbc-comics.com/index.php?db=comics&id=2088#comic


8/9

If may sound cliche, but Dr. Plait and Dave McRaney agree: If most scientists fall on one side of anargument, it's a safe bet that's where the evidence lies. Controversy does not disprove fact, andongoing research doesn't diminish the research already done. Itmay require unlearning things you

already believe,but doing your own research is worth it. "There's a reason lots and lots of scientistsagree on something: usually because it's right," Dr. Plait said. "Sure, science can change its mind, but
http://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.htmlhttp://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.htmlhttp://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.htmlhttp://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.htmlhttp://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.htmlhttp://www.newyorker.com/online/blogs/frontal-cortex/2012/06/brain-experiments-why-we-dont-believe-science.html


9/9

going with it is the way to bet, because there is usually vast research, evidence, and experience behindit. Go that way and you'll be right the vast majority of the time." Image byXKCD.

McRaney sums it up nicely:

We invented the scientific method because we are naturally terrible at explaining our own

experiences. Without the scientific method, there is no way to know what causes simple, everydaythings like thunder. Every explanation is as good as another, and if an explanation becomesculturally bound and passed down, that becomes the official explanation for millennia. Our naturaltendency is to confirm our assumptions, but science tries to disconfirm our assumptions one by oneuntil the outline of the truth begins to form. Once we realized that approach generates results, wewent from horses and tobacco enemas to mapping DNA and walking on the moon in a fewgenerations.

What do you think? How do you avoid confirmation bias when you're researching a new study orarticle? Which sources do you trust? Share your research tips in the comments below.
http://xkcd.com/154/http://xkcd.com/154/http://xkcd.com/154/http://xkcd.com/154/

how to determine if a controversial statement is scientifically true

Documents