Last week, Chris Bullin did a lovely post on the tweets at the Science Online 2013 Conference (#SCIO13). I hope it intrigued some of you enough for you to go look at more of the tweets.
Science Online 2013: http://scienceonline.com/scienceonline2013/
As I was tracking the Twitter stream for the conference, I noticed several hashtags being used for specific sessions. They were all wonderful, but I thought #medskep was perhaps the most important one for information professionals, those curating & sharing science information, science journalists, science communication experts, and those trying to persuade and engage science professionals in social media.
Science Online 2013: Session 5E: How to make sure you’re being appropriately skeptical when covering scientific and medical studies (#MedSkep): http://scio13.wikispaces.com/Session+5E
Reading studies with skepticism RT @aetiology: Anyone at session 7b with myself and @ivanoransky, hashtag will be #medskep #scio13
— Ivan Oransky (@ivanoransky) February 1, 2013
#medskep= how to make sure you’re being appropriately skeptical … Evaluating scientific and medical studies. #scio13
— Catharine Zivkovic (@ccziv) February 1, 2013
Faculty often are engaged in trying to teach critical appraisal within their domain, to encourage students to select and cite more authoritative research in their academic products (ie. homework and research). Librarians share in this effort, teaching strategies for accessing resources of high quality, tips for how to identify high quality, and general skills for critical appraisal and critical thinking. This is a frequent topic of conversations among the profession at large, as well as at our own departmental meetings. We talk about how involved should the librarian be in teaching these skills, how to partner with faculty, how to improve our effectiveness, how to improve both our own skills and our credibility in this area among the students and faculty, and much more. Research is done and articles are written, all about how to help students and journalists and the public better understand the strengths and limitations of science research. Librarians are developing tools and resources to help teach and understand these skills, often in partnership with scientists, and at SCIO13 several scientists and faculty were sharing these, and then other scientists and librarians resharing them!
Sometimes, there is no substitute for taking the words from the most source. This is why the #MedSkep conversation was so powerful. Real scientists connecting with experts in science and critical appraisal (Ivan Oransky and Tara C. Smith) talking freely and honestly in plain language about why scepticism is essential in science literacy, and tips for communicating with special audiences (hint, hint, journalists?) about science.
So here, I want to share some of my favorite tweets from the conversation. I’ve organized them into three sections: Thoughts, Tools & Resources, and Debates. FYI, the debates were almost entirely about medicine and healthcare research, problems with peer review, there were some pointed comments about systematic reviews, and overall, they were … intense.
THOUGHTS
Journos should take “Trust me, I’m a scientist!” as cue to ratchet up their skepticism. #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
Covering something via press release without reading the paper is journalistic malpractice.@ivanoransky at #scio13 / #medskep
— John Timmer (@j_timmer) February 1, 2013
“I hope this shocks you that I would have to put this in a slide presentation”: Always read the study! @ivanoransky #scio13 #medskep
— Kelly April Tyrrell (@kellyperil) February 1, 2013
@ivanoransky says lots of studies DO talk about limitations in Discussion section; journos often don’t read that far. #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
It is essential for #scijournalists to understand #GlamourMag is not a term of respect.#scio13 #medskep
— Drug Monkey (@drugmonkeyblog) February 1, 2013
Being appropriately skeptical =/= labeling someone as a fraud. (Cf. Paul Bookes, ScienceFraud.org ) #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
who, who, how, how, how, what, how. (7 q’s to ask while reading medical research.) newsworks.org/index.php/loca… #medskep #scio13
— shan (@shanpalus) February 1, 2013
“sheer danger in looking at one study at a time” #medskep #scio13
— shan (@shanpalus) February 1, 2013
“scientists use systematic reviews to pee on trees, mark territory.” look at more than one @scicurious #medskep #scio13
— shan (@shanpalus) February 1, 2013
Focus first on the epidemiology and study design. If poorly designed, statistics can’t save it. #scio13 #medskep #internalvalidity
— Eli Perencevich (@eliowa) February 1, 2013
@eliowa Journalists also need to KNOW study design to talk critically about research & evaluate it.
— Rebecca Kreston (@thebodyhorrors) February 1, 2013
@thebodyhorrors Agree. Or collaborate/talk with an epidemiologist. The default shouldn’t be statisticians.
— Eli Perencevich (@eliowa) February 1, 2013
@eliowa Definitely. Especially if biostats & maths are perceived as boogie monsters, as I was picking up in the session. #scarypvalues
— Rebecca Kreston (@thebodyhorrors) February 1, 2013
“disclosure in and of itself does not remove the conflict of interest” “how deep do you go, how far do you dig?” #medskep #scio13
— shan (@shanpalus) February 1, 2013
Are results generalizable? Is what we’re measuring well-defined? Is study well-designed to measure what we think it does? #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
@ivanoransky “Be mindful of whether you’re looking for a real, meaningful clinical endpt.” Be wary of loose correlations. #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
Secondary endpoints might not be supported by good expt’l design #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
scio13 #medskep @ivanoransky “little bit pregnant ?” what to make of P values?Ask what was primary endpoint of study? Beware ofP >.05
— UNC med/sci Journ (@UNCmedsciJourn) February 1, 2013
Especially important to call out bad stories if they’re getting lots of (credulous) attention elsewhere! #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
TOOLS & RESOURCES
At Reuters, @ivanoransky drills the following criteria into his staff: healthnewsreview.org/about-us/revie… #scio13 #medskep
— John Timmer (@j_timmer) February 1, 2013
This is so important, I’m going to BRIEFLY distill the key points, but do please go read the original, in full.
#1 Identify costs, both economic & social or personal.
#2 Identify benefits.
#3 Identify the harms.
#4 How good is the evidence?
#5 Avoid “disease-mongering.”
#6 Use independent sources, and identify conflicts of interest.
#7 Compare the new way to what is existing best (or standard) practice.
#8 Is this new way actually available to the public? How available?
#9 Is this actually novel? As in unique & innovative.
#10 Don’t just crib from a press release.
shoutout to @medlineplus by @ivanoransky at #scio #medskep I *heart* MedlinePlus for reliable med / health info.
— CogSci Librarian (@CogSciLibrarian) February 1, 2013
shoutout to lit reviews, meta analyses when covering medical stories, by @ivanoransky at #scio13 #medskep How to find ’em? Ask a librarian!
— StephanieWillenBrown (@JoMCParkLib) February 1, 2013
Its very good MT @cogscilibrarian: .@tomlinden recs News & Numbers by Victor Cohn et al. worldcat.org/title/news-num… #scio13 #medskep
— Melissa J Bodeau (@MJBwrites) February 1, 2013
Free online course on critically appraising medical studies from Cochrane/John Hopkins Uni #medskep #scio13 us.cochrane.org/understanding-…
— Hilda Bastian (@hildabast) February 1, 2013
@helenbranswell @docfreeride Thanks! You can find out & keep up with @nihmedmedia here – and the website: prevention.nih.gov/medmediacourse/
— Hilda Bastian (@hildabast) February 1, 2013
We’re gathering free resources on understanding/interpreting clinical research here1.usa.gov/VFFLql#medskep #scio13
— PubMed Health (@PubMedHealth) February 1, 2013
stats resources for sci writers #scio13 #scibreak #medskep bit.ly/YrQsvx; 1.usa.gov/IriFNh; bit.ly/14vgwJH
— Robin Lloyd (@robinlloyd99) February 2, 2013
DEBATES
1) Is Medical Research Really Science?
scio13 #medskep @boraz says med sci is unique subculture of science. More skeptical of med science than herpetology, eg
— Tom Linden (@tomlinden) February 1, 2013
And half the room seems to disagree with @boraz on this. 🙂 #medskep #scio13
— Rachael Ludwick (@r343l) February 1, 2013
@boraz expects more honesty, objectivity, less COI from herpetologists than biomed researchers. #OhNoHeDidNot #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
“fun science” scientists easier to trust, says @boraz. “when i get to [ppl who study humans] all my red flags go off.” #medskep #scio13
— shan (@shanpalus) February 1, 2013
.@boraz: biologists do research for the love of it, won’t fudge results. @j_timmer disagrees, notes trivial press releases #medskep #scio13
— Robin Lloyd (@robinlloyd99) February 1, 2013
@robinlloyd99 @boraz It’s less fudging results as stretching them to interpretations that aren’t supported by the data. #scio13
— John Timmer (@j_timmer) February 1, 2013
@robinlloyd99 @boraz @j_timmer Many fields haven’t faced as much demand for replication, scrutiny of publication bias #medskep #scio13
— Hilda Bastian (@hildabast) February 1, 2013
@robinlloyd99 @boraz @j_timmer Press releases aren’t the scientists’ work though (perhaps I’m missing the point?) #medskep #scio13
— Grant Jacobs (@BioinfoTools) February 1, 2013
Perception w/ health studies: the science is inherently messy (studying humans, or much simpler model systems) #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
2) Problems with Peer Review
scio13 #medskep audience member senses distrust of peer review process, but says peer reviewers ARE skeptical
— Tom Linden (@tomlinden) February 1, 2013
Scientists have variety of reasons 2 publish where they do; journos can’t reliably read tea-leaves from journal reputations #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
former peer-reviewer: “you let things slide at those lower tier journals” #medskep #scio13
— shan (@shanpalus) February 1, 2013
#scio13#medskep audience member suggests lower-tier journals may have FEWER problems since they’re already been reviewed by top-tier journ
— Tom Linden (@tomlinden) February 1, 2013
V smart RT @eliowa: “Manuscript provenance”- why lower impact journals may have higher-level review. bit.ly/14vgwJH #medskep #scio13
— Ivan Oransky (@ivanoransky) February 1, 2013
@aetiology : scientists know that peer review is where the critical conversation STARTS, not where it ends. #scio13 #medskep
— Janet D. Stemwedel (@docfreeride) February 1, 2013
WANT MORE?
Here is a blogpost on the session from one of the organizers.
Aetiology: Skeptical science and medical reporting (#Scio13 wrap-up) http://scienceblogs.com/aetiology/2013/02/04/skeptical-science-and-medical-reporting-scio13-wrap-up/
And a lovely Storify collecting many more of the tweets. Well worth reading through in its entirely.
#medskep session at #Scio13 http://storify.com/aetiology/medskep-session-at-scio13
Emerging Technologies Librarian 