April #UROJC: The Surgeon Scorecard – Merits of Publicly Reported Surgical Outcomes
The April 2016 International Urology Journal Club on Twitter (#urojc) hosted a discussion on our paper, “Comparing Publicly Reported Surgical Outcomes with Quality Measures from a Statewide Improvement Collaborative”. Published in JAMA Surgery on March 16, 2016, the paper was authored by Gregory Auffenberg MD, David Miller MD, Khurshid Ghani, Zaojun Ye, Apoorv Dhir, Yoquing Gao. I contributed as a member of MUSIC.
It was an honor to have the paper selected for a #urojc discussion, and the authors would like to thank JAMA Surgery for providing open access during the discussion period. This post serves as an overview, and the entire #urojc transcript is available for reading courtesy of Symplur
For those not familiar, the #urojc Twitter chat is a 48-hour asynchronous conversation amongst urologists around the world on Twitter on a selected journal paper, taking place on the first Sunday/Monday of every month.
The ProPublica Surgeon Scorecard
The subject of our research centered on the online U.S. surgeon ratings compiled for ProPublica’s Surgeon Scorecard. ProPublica is an investigative journalism organization that was given exclusive access to U.S. Medicare data for the years 2009 to 2013.
“Reporters Olga Pierce and Marshall Allen studied almost 75 million hospital visits billed to Medicare looking for eight common, elective surgeries. They then looked to see whether the same person returned to the hospital for what appeared to be complications from the surgery. Their full methodology is spelled out here.”
- John Keefe, WNYC Data News Team
The Michigan Urological Surgery Improvement Collective
Specifically, our research paper looked at ProPublica’s ratings for only one procedure – results on radical prostatectomy (RP) for prostate cancer – and correlation to reporting by MUSIC, the Michigan Urological Surgery Improvement Collaborative. MUSIC is a state-specific quality initiative in the U.S. in which I am a participating surgeon. Participation in MUSIC is voluntary, over 85 percent of urologists in the State of Michigan participate in the collaborative.
@MattBultitude @MUSICUrology participation is not mandated by govt or any other entity, no penalty for not participating #urojc
— Greg Auffenberg (@gregauff) April 3, 2016
@Surgeo having 85% sample of a large geographic area of urologists hopefully increases generalizability but bias always possible #urojc
— Greg Auffenberg (@gregauff) April 3, 2016
April #UROJC
As our paper states, the recent release of the Surgeon Scorecard accelerated debate around the merits of publicly reporting surgical outcomes. Surgical outcomes assessment is not a new concept, even dating back to 1860 as this tweet by @mattbultitude surfaced.
I rather like this quote from the footnote on @BAUSurology surgical outcomes page. Very relevant !! @iurojc#urojcpic.twitter.com/q0ETmj2WQa
— Matthew Bultitude (@MattBultitude) April 4, 2016
What does our community of urologists think about public reporting? Does greater transparency correlate with better outcomes? What are the benefits of a collaborative method like MUSIC? What methods are used in other parts of the world?
The #urojc discussion found that many urologists outside the U.S. were not familiar with the ProPublica ratings or debate. Some were not surprised that we did not find a correlation between our MUSIC outcomes data and the ProPublica data, thereby validating the need for quality outcomes data.
@wandering_gu @gregauff @iurojc highlights need for validated quality measures; @MUSICUrology valuable for checking surrogate endpts #urojc
— Parth K. Modi (@marthpodi) April 4, 2016
So get meaningful surgeon specific data. Can we really deny that some surgeons are better than others? https://t.co/UKm27yTqPl
— Barry Kogan (@bak_pedurol_mad) April 5, 2016
@marthpodi @bak_pedurol_mad physician outcomes may not always equal patient outcomes. Priorities may be different #qol #pcsm
— Sabin Motwani (@sabinbmotwanimd) April 5, 2016
.@wandering_gu #urojc Right-also population of interest way off base. >65 y.o. least likely to benefit from surgery pic.twitter.com/0ueHOIdxwQ
— Alexander Kutikov MD (@uretericbud) April 5, 2016
@iurojc@allaf_mo@marthpodi@wandering_gu@gregauff@MUSICUrology Long-term trifecta more impt than most short-term periop metrics #urojc
— Stacy Loeb, MD (@LoebStacy) April 4, 2016
If the Surgeon Scorecard is flawed, what needs to be done to create an acceptable public reporting system?
@iurojc also is public individual physician-level reporting needed? Perhaps MUSIC-like collaboration is better #urojc
— Parth K. Modi (@marthpodi) April 4, 2016
@marthpodi @iurojc Outcomes reporting req *very* good risk adjustment. We’re not there yet. But I think publishing volumes reasonable today
— Matt Cooperberg, MD (@dr_coops) April 5, 2016
@iurojc large scale registry similar to @STS_CTsurgery is a start, will make sure measures impt to us/pts captured.Move away from admin data
— Rich Matulewicz (@RichMDUrology) April 4, 2016
Is public reporting of surgical outcomes taking place in Australia, UK, Canada & elsewhere?
Validation clearly vital. In UK we self-report on several procedures but no checks (yet). #urojc https://t.co/438DqKzA1E
— Matthew Bultitude (@MattBultitude) April 3, 2016
@StorkBrian #urojc if surgeons lead reporting ie @MUSICUrology more transparency, tackle difficulties of creating soundbite that public want
— simon van rij (@sivanrij) April 4, 2016
@sivanrij
Indeed, seems like a real opportunity for #urology to lead the world wide surgery community @MUSICUrology#urojc— Dr. Brian Stork (@StorkBrian) April 4, 2016
How are ‘outliers’ identified by this study handled by MUSIC?
@iurojc@MUSICUrology goal: make the entire group better through measured outcomes and feedback #urojc
— Greg Auffenberg (@gregauff) April 4, 2016
Do ratings lead to cherry-picking of patients?
@iurojc someone can find the cardiac data (they were the first with public reporting) some evidence of decrease in high risk cases #urojc
— daviesbj (@daviesbj) April 3, 2016
@daviesbj@iurojc see “unintended consequences” section of this very thorough recent review on CV public reporting, (PMID:25918041) #urojc
— Greg Auffenberg (@gregauff) April 3, 2016
According to New York cardiologist, Sandeep Jauhar, MD via Medscape, 63 percent of cardiac surgeons acknowledged accepting only relatively healthy patients for heart bypass surgery owing to report cards in New York State.
Thanks to all who participated in the April #urojc. Again special thanks to authors @gregauff@StorkBrian & @JAMASurgery for article OA
— Urology JC #urojc (@iurojc) April 5, 2016
Moving Surgical Outcomes Forward
On behalf of the authors of the paper and the entire MUSIC collaborative, I would like to thank our #urojc colleagues around the world for their thoughts, insights, criticisms and questions about the paper.
The ProPublica Surgeon Scorecard has generated significant and serious discussion in the U.S. about the challenges and merits of the public reporting of surgical outcomes. In an increasingly connected world, it’s difficult to imagine how this can remain simply an American debate.
Urologists by their very nature are leaders. Personally, I see this debate as yet another opportunity for us to develop and implement systems and strategies that reassure the public and advance patient care.
