This post reviews the paper ‘Pubertal Suppression for Transgender Youth and Risk of Suicidal Ideation’, one of the primary scientific papers supporting the use of drugs to inhibit the progress of puberty in children. This research is winner of the American Academy of Pediatrics ‘Paper of the Year’ award, 2020, and is cited 104 times; thereby, lending large credence to its conclusions. The review focuses on the paper’s quality of data, arguments made by the authors, and statistical results used to support the arguments. Despite the many plaudits, the paper greatly exagerates the definitiveness of its conclusions. It appears to suffer from large bias in the dataset and completely disregards statistical power. This is similar to many other papers that drive the ‘replication crisis’ across scientific disciplines. While the data is not publicly available, a research team with strong statistical skills could deliver much needed clarity on the topic by a thorough review of similar papers.
Introduction
I was perusing NPR when I came upon the article: ‘Most teens who start puberty suppression continue gender-affirming care, study finds’. Most of the article seems to flow reasonably enough. But, as a parent, I was particularly taken aback by the paragraph:
the American Academy of Child and Adolescent Psychiatry has stated that it “supports the use of current evidence-based clinical care with minors. … Blocking access to timely care has been shown to increase youths’ risk for suicidal ideation and other negative mental health outcomes.”
This and accompanying paragraphs seems like quite the ‘call for action’ - children with such symptoms are ticking time-bombs if they do not get gender-affirming care. I’m only a layperson on the subject, but gender-affirming care appears to be a loose sequence of increasingly invasive actions applied to children by healthcare providers: i) introduction of preferred pronoun, ii) change of clothes and appearance, iii) treatment of puberty-blocking hormones, and iv) cosmetic surgery (such as double mastectomy). The article references similar posts, such as in Scientific American, which themselves list a number of articles, including The Evidence for Trans Youth Gender-Affirming Medical Care which lists a number of scientific papers. The largest and probably most well-publicisized study supporting gender-affirming care is the subject of this post, and involves a purported 30,000 individuals. This study won American Academy of Pediatrics’s “Paper of the Year” for 2020. So, it should be exceptionally strong in its methodology and analysis.
Data Available
The data used includes the large group of participants from the 2015 US Transgender Survey (USTS), which focused on 27,715 US transgender adults. The data can be found, here, but is not publicly available. While seemingly strong, comments to the paper concerning issues with how the survey was conducted (Patrick H Clarke, University of Adelaide) are disconcerting. They include conclusions from the UK High Court that found USTS ’employed a non representative, biased convenience sample’ and ‘Over 70% of the USTS respondents demonstrably did not know what puberty blockers were, claiming to have commenced treatment after age 18.’ Other issues exist, but we can give the survey the benefit of the doubt.
There is a large amount of conditions placed on the data. While these seem to be reasonable, they do add to the number of assumptions and the complexity of the study. Because pubertal suppression was only available to adults age 17 or younger in 1998, older participants are excluded. In addition, the data is filtered to those who selected “puberty blocking hormones (usually used by youth ages 9–16)” in response to the question “Have you ever wanted any of the health care listed below for your gender identity or gender transition? (Mark all that apply).” This leaves 3,494 individuals between the ages of 18 and 36 who ever wanted pubertal suppression as part of their gender-affirming medical care. Finally, participants who reported beginning treatment at age 17, the final stage of puberty, would be removed because hormone blockers would have no effect on them; however, there were none. This is a large subtraction from the abstract-touted number, which seems to be a little disingenuous.
The exposure group is made from those who have ever had puberty blockers, selected from: ’the question “Have you ever had any of the health care listed below for your gender identity or gender transition? (Mark all that apply).”’ The responses include ‘“counseling/therapy,” “hormone treatment/HRT,” “puberty blocking hormones (usually used by youth ages 9–16),” or “none of the above.”’ Four responses exist; however, the three non-“puberty blocking hormones" are rolled into one ‘No’ response for analysis. This muddies the waters quite a bit because of possible conflation among the levels, especially hormone treatment and puberty-blocking hormones. Let’s move on, anyhow.
Of those who wanted pubertal suppression, only 89 (2.5%) had received this treatment. This is a big statistical red flag! Only 2.5% of the sample will be the treatment group, versus the other 3,405 participants. Any analysis used will have to address this discrepancy because of challenges in some statistical methods associated with unbalanced data. Even more concerning, using over one thousand subjects can lead to over-powered tests. The survey also has quite a few independent variables, 14, many being factors with multiple levels. So, the small, 89 person, treatment group with large number of control variables will need to be used carefully to ensure a reasonable number of degrees of freedom in any statistical tests.
In addition to the original comments that the survey is not a random sample. This leads to the major flaw in this dataset that it only includes ’transgender adults’. It does not include adults who may have had gender-dysphoric traits when younger, but did not have these symptoms, later. This is important because a hundred years of data show that about 90% of youth with such characteristics lose them shortly after teenage years. It also completely removes from the analysis medical consequences associated with children treated with gender-affirming care, then not identifying as transgender as adults. This bias is very hard to overlook.
Analysis Conducted
The ultimate research question can feel a bit convoluted because of all the conditioning in the data, and the wording of the survey, so I will try to simplify it. I also align the question using the Pearson (1900) chi-square framework for contingency table analysis.
Is there a difference in suicidal tendencies based on puberty-blocking treatment? It is important to keep in-mind:
- the scenario has many conditions described in the section above
- the treatment is determined by the survey question: ‘Have you ever had puberty blocker treatment: Yes, or No’
- the response is suicidal ideation (health outcomes): Suicidality (past 12 mo), Suicidality (lifetime), or Mental health and substance use
The paper states: ‘There is a significant inverse association between treatment with pubertal suppression during adolescence and lifetime suicidal ideation among transgender adults who ever wanted this treatment.’ This strong statement is not tempered by statistical results.
Looking at ‘TABLE 3 Raw Frequencies of Outcome Variables’, we see the treatment group has suicide ideation 75.3%, versus 90.2% for the control. These raw proportions come without using the logisitic regression model to account for external factors. But, intuitively, this is not a large difference considering all the before-mentioned weaknesses in the data. What should be noted is that both of these groups suffer from severe emotional conditions. Yet, there is no documentation of therapy - only social conditioning factors.
Continuing with this argument, the 0.3 odds ratio between the outcomes is rather trivial. We can convert the odds ratio to Cohen’s d in order to get a relative idea of the magnitude. This uses the equation 𝑑=LogOR × √3/𝜋 from Effect-Size Indices for Dichotomized Outcomes
in Meta-Analysis.
d = log(0.3)*(sqrt(3)/pi)
round( d, 2)
-0.66
This number does not even register as small using Cohen’s t-shirt size of 2 as a cut-off value. Even more misleading, we can interpret the statement as the odds of suicidal ideation are 30% higher for transgender adults who ever wanted the treatment, but didn’t receive it. This is not a useful statement because the odds references the alternative group, not the total number of participants.
Typically, we want the relative risk, the ratio of risk of an event in one group (given puberty blocker treatment) versus the risk of the event in the other group (not given puberty blocker treatment). In fact, the odds ratio is often misleading in that it does not approximate well to the relative risk when the initial risk (the prevalence of suicidal ideation) is high. Worse still, this odds ratio is only compared to transgender who did not receive the desired treatment, which wholly omits the possibility of non-transgender who wanted the treatment, but later decided they were not transgender.
Remarkably, the most damning mistake is using over-powered tests that are so sensitive as to always result in a significant p-value. With a treatment group of 89 and a control group of 3,405 subjects, statistical tests cannot be expected to perform reasonably. This scenario is often called ‘p-value hacking’ in order to get positive results for publication. The subjects denominator will ensure that any difference will be significant. Unfortunately we don’t have the data. But taking random samples of 100 subjects, and performing the similar tests would provide a distribution of the p-value which could be used.
Beyond these indescretions, this is really a complex problem, with many control variables, performed on survey data about events that may have happened many years in the past. No medical records are used to confirm any of this! This revelation alone should make readers skeptical. Add the fact that only 89 participants fit the treatment group, in a purported 30,000 subject survey. Finally, the severe emotional issues being a conflated variable between both groups means that very few results the study can be considered useful.
Additional Context
I’m skeptical of research that claims to ‘prove’ that gender-dysphoria in older adolescents is actually occurring for several reasons. First, the primary researchers of this disorder, Kenneth Zucker and Pat Blanchard, who wrote the definition in the Diagnostics and Statistical Manual (DSM), are vocal in the lack of research and politicalization of this issue. Dr Zucker became one of the first victims of the LGBT activist movement when his clinic was closed by government activists who felt his therapy practice was not ‘in step with the latest thinking’ by not being aggressive with ’transition’ to a new gender. The accusers were forced to pay half a million dollars in restitution and provide an apology after a lawsuit found wrongful accusations.
Second, a large amount of marketing is now spent targeting children for LGBT influence. Personally, I’ve seen multiple ads on YouTube questioning, ‘How gay are you? Take our test!’ as well as similar approaches on Facebook and TicToc. The Heritage Foundation took note of many campaigns aimed at children across multiple ages. Regardless of the content, gender-dysphoria is an important diagnosis that should be discussed with a psychiatrist - not social media.
Third, the increase in advertising has coincided with the increase in a new phenomenon, loosely named ‘rapid onset gender dysphoria’. This name characterizes the fact that one subset of the developed world population, middle-class teenage white females, has seen a 4,400% increase in the incidence of gender dysphoria, in the last ten years. From an epidemiological perspective, such an incidence would be considered a major health crisis. Certainly, the rise of endocrine disrupters in our environment may be a major contributor to these new gender phenomena.
Fourth, there is now a large financial interest in generating diversity across multiple industries. Newly minted chief diversity officers need to produce results. This can be quite difficult in some communities that may take decades to reach parity in skills with others. This is playing-out in the lack of results despite major resources pushing this aggressive agenda. To demonstrate how aggressive it is, colleges now require faith-based statements committing new faculty to diversity pledges similar to religious-backed programs. Unfortuantely, faith is usurping science. One of the arguments used by proponents of transition is that it is reversible. However, there are countless personal interviews of people who transitioned with hormones, then attempted to de-transition, who are quite open that many of the effects are permanent. With medical centers now offering ‘Top Surgery’ a cute sobriquet for double mastectomey, medical malpractice, one of the primary culprits behind the opioid epidemic, has found a new revenue source. With so much hype, money is sure to follow.
Lastly, the scientific industry has had a history of experimenting with gender reassignment. The book ‘As Nature Made Him’ is one outcome from the standard practice by medical industry called ’normalizing surgery’ attempting to correct children born with abnormal genitals. That particular story does not end well, and the paper, ‘What Are We Doing to These Children? Response to Drescher, Clayton, and Balon Commentaries on Levine et al., 2022’, should give any parent cause for questioning the decisions of doctors in these types of situations.
Conclusion
The paper ‘Pubertal Suppression for Transgender Youth and Risk of Suicidal Ideation’ is one of the most influential and popular in the transgender movement. By winning a ‘paper of the year’ award and touting the results of a massive survey of thirty thousand applicants, a superficial glance gives the appearance of grave importance that is now being celebrated by mainstream news media. However, this cursory review of the paper’s methodology and analysis shows the lack of substance in its conclusions. That is unfortunate not only for the unwitting readers of publications such as NPR, but it also shows the degradation of scientific journals and the research institutions that contribute to them.