The problems with using population-level data to estimate prostate screening benefits

Almost any debate about the effectiveness (or lack thereof) of PSA-based screening for prostate cancer in the U.S. will usually involve whether the results of the two largest randomized screening trials or national mortality statistics more accurately represent the effects of intensive screening from the early 1990s to the late 2000s. Putting aside the conflicting results of the U.S. Prostate, Lung, Colorectal, and Ovarian Cancer Screening (PLCO) trial and the European Randomized Study of Screening for Prostate Cancer (ERSPC) - for which there are many plausible explanations - even the most optimistic statistical interpretation of these trials suggests that PSA screening reduces prostate cancer mortality by 25-30% at best, which does not fully account for the observed 40% decline in prostate cancer mortality from 1991 to 2008. Since the effectiveness of standard prostate cancer therapy did not change significantly during this time frame, PSA screening advocates have suggested that the discrepancy is probably due to flaws in the trials, rather than issues with "real-world evidence" derived from population-level mortality data.

However, in a thoughtful commentary recently published in Mayo Clinic Proceedings, Drs. Joaquin Chapa, Alyson Haslam  and Vinay Prasad provide lots of good reasons to question the validity of prostate cancer mortality trends. First, as any clinician who has filled out a death certificate knows, determining the underlying cause of death can be difficult in a patient with several serious health conditions. Patients with metastatic prostate cancer may die with incurable cancer, but not of it. Then, the algorithm used by the Mortality Medical Data System may introduce error, noise, and bias because prostate cancer is accepted as an underlying cause of death for many conditions (e.g., cirrhosis, bacterial endocarditis) that could be related to the cancer but could also simply co-exist.

In addition, studies show that patterns in attribution of causes of death often change over time due to factors other than actual changes in underlying causes. Changes in population composition (e.g., increases in the Hispanic and Asian proportion of the population relative to whites and African Americans) can also result in different overall prostate cancer mortality rates by increasing the percentages of populations who have lower cancer mortality.

In contrast, the methods used to determine causes of death in PLCO and ERSPC were much more rigorous; the cause listed on the death certificate was double-checked by 1 to 3 independent, blinded reviewers. These processes demonstrated that assigning a cause of death is potentially fraught with error and subject to human bias. As Chapa and colleagues observe:

Even with more rigorous processes for determining COD in the PLCO and ERSPC trials, COD determination remains difficult and is subject to uncertainty. Of all deaths in the PLCO study, 28% required additional human review because of discordance between the death certificate and the initial human reviewer. Of reviewed cases, 3% required a conference call to resolve discordance among 3 reviewers.

I've always thought that crediting PSA screening for the historical decline in U.S. prostate cancer mortality made little sense; for one thing, one wouldn't expect a mortality difference to be visible for at least 7-8 years after screening became common in clinical practice, the earliest point in the ERSPC trial when the survival curves separate. That would have been 1997 or 1998 at the earliest, not 1991. Other studies have observed that prostate cancer mortality also began falling in the U.K. in the 1990s, even though PSA screening was uncommon. This new analysis provides even more reason to doubt that there is a straightforward cause-and-effect relationship - if, indeed, there is any relationship at all.

By the way, I'd like to give a shout-out to the terrific medical podcast Plenary Session, hosted by Dr. Prasad. An interview with Dr. Chapa in a recent episode was the reason I knew about his paper in the first place. Plenary Session is too new to have made my most recent list of favorite podcasts, but you can bet that it will be on the next one.