The NICE Transgender Evidence Report (and How to Read It)


"Enough’s enough. In the last 24 hours 
@hannahsbee
 & I have been called dishonest journalists, unethical, accused of killing children, liars...All for highlighting the lack of good evidence for an intervention used on kids & reporting on whistleblowing concerns at an NHS service" Deb Cohen
https://twitter.com/deb_cohen/status/1378099014340579331

Last week Twitter was blowing up over a new UK evidence review (using systematic review methods) on the topic of puberty blocking or delaying treatments for children and youth who identify as transgender. The largest pushback I witnessed was being directed not at the authors of the evidence review (link will force a PDF download), but the authors of the BBC news report that described the findings.


Now, the BBC news report did a fairly good job of reporting a synopsis of what the NICE review found. Minus one critical piece of context, which I would like to fill in here for people. I am speaking here as someone who previously has had a not insignificant side-gig as a consultant on systematic review methodologies. At the time, there weren’t so many of us medical librarians who were doing this, and now there are a LOT. And most of them will tell you what I’m about to say.

THE FIRST REVIEW ON A NEW QUESTION USUALLY FINDS INSUFFICIENT EVIDENCE. THIS IS NORMAL. I repeat, this is NORMAL!!

This is how it is supposed to work. Someone asks a question, and frankly, someone has to be the first. It starts somewhere. That first review finds insufficient evidence specifically because it is a new question or topic. Often researchers exploring emerging topics haven’t yet agreed on how to study it, how to define the methods and limitations, what are the standards, how to report data, how to define the study population, and so forth and so on. The first review is the first step toward doing this. The first review looks for emerging patterns and trends, picks out those that look most important and most likely to lead to consensus later, and then makes recommendations for future research in this area to use. This is the first really major step toward agreeing on those definitions, standards, measures, methods, etc.

Now comes the hard part. You have to wait FIVE WHOLE YEARS. During that time, people who care passionately will do new research, BETTER research, research that is reported better and meets the new emerging standards and guidelines. After five years, or ten years, now you can redo the study. If the new research on the topic is well done, it will either point to a new finding, or will confirm the original finding. Either way, you will know more. Sometimes it takes 10 years to achieve enough of an evidence base to show that something works. If there is not enough research funding for the topic, it may take longer.

My point is that this is not the end of the question, this is the BEGINNING. It will take a lot of money to fund more research to really answer this question. If people don’t understand how systematic review methods typically work, then politics can kill off the funding, and we will never really know the answer. With a topic as politicized and sensitive as this one, there are chances that policymakers may misread or misinterpret the significance of the findings in this report. That makes it really important to make this very clear.

The only thing this study tells us is that we don’t know the answer YET, according the the criteria specified in this report. If you want to actually know the answer, it is critical to not make a definitive decision, and to keep studying this for a while longer, and in a humane way. It might also be good to look at other reports on this from other authors, and which define the question differently. Consider the role of bias in each. I find it interesting that the similar US report had opposite findings, and am asking myself about the role of cultural context in defining how the questions were approached.

Resources

Evidence for puberty blockers use very low, says NICE https://www.bbc.com/news/health-56601386

NICE, 2021. Evidence review: Gonadotrophin releasing hormone analogues for children and adolescents with gender dysphoria. Download the full report from https://arms.nice.org.uk/resources/hub/1070905/attachment.

Transgender Identity and Experiences of Violence Victimization, Substance Use, Suicide Risk, and Sexual Risk Behaviors Among High School Students — 19 States and Large Urban School Districts, 2017 https://www.cdc.gov/mmwr/volumes/68/wr/mm6803a3.htm

2 responses to “The NICE Transgender Evidence Report (and How to Read It)

  1. “Nicely” said

    Having read the review (and many of the base articles 1) there is no validation of the measurement scales used so one can question whether those measurements are valid in this age 2) off puberty from one’s peers is catastrophic – whether too early or late – one more instance of being off step

    OTOH- we (by law) rarely emancipate minors prior to age 16. Legal consent is 18. Most states do not even allow such minor independent decisions such as vaccination or birth control prior to 16. If you want ugly, go look up child marriage laws and age of consent (which can be made by the parent by and over child wishes in several states).

    My point is that this is a serious medical/emotional concern and nothing is without potential serious long term cost. Reviews are important. Care needs to be in centers where the needs of the individual are paramount (and not the wishes, dreams, or pathology of the family).

    Sent from my iPhone

    >

    Like

  2. Thank you for this insightful comment. I especially appreciate your closing line: “Care needs to be in centers where the needs of the individual are paramount (and not the wishes, dreams, or pathology of the family).” Bravo. My post is just looking at a tiny portion of the reporting, which (IMHO) left out critical content and is leading to misunderstandings of the findings of the report. There have been other analyses and commentary I’ve found, both in the news and social media, and there’s a lot of vitriol and misunderstandings and bias on both sides. What you describe about need validation and consensus around the measurement scales is exactly the sort of process that usually happens after an initial review, and this is why those initial reviews are so critical — to trigger the process of consensus building and move it forward. Cost/risk/benefit analysis is absolutely the best way to support personal decision making, especially during these early stages in an emerging field or treatment modality. This is another aspect of evidence-based decision making that is sadly often overlooked — decisions need to be based on the best available evidence. Best AVAILABLE evidence. That doesn’t mean that evidence is necessarily good or strong evidence, but the lack of strong evidence doesn’t mean the decision can wait until we have better evidence. We’re talking healthcare, medicine, mental health. There is an urgency to these questions, and decisions have to be made while there is an opportunity to make a difference, hopefully a positive difference, whatever that turns out to be.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s