Individualised reporting of surgical outcomes in the UK
Its happening and its happening fast. What the cardiothoracic surgeons in the UK have lived with for 8 years is coming to a specialty near you very soon. Individual urological surgeons results to be published openly, in the press in the UK from summer 2013.
It’s a massive change to the way we work and to the way surgical data is presented. No longer will the interesting elements of the urological literature be studies of the results of a few surgeons in a few centres of world renown but rather the performance of outliers. Reports will start to reflect what is achieved throughout the nation not in one or two centres in that nation. Warts and all if you like.
There are risks. Big risks. For surgeons and for patients.
Will surgeons who operate on high risk cases be smeared because they appear to be underperforming?
Will patients who are high risk be denied operations because surgeons subliminally start to make recommendations that are good for the surgeon but not for the patient?
Will surgeons continue to train junior surgeons or will juniors simply cut the stitches?
Why should surgeons carry the can for the performance of the whole team?
Will sample sizes be big enough to ensure that results could not be due to chance?
What outcomes should we be measuring in order to judge the quality of many operations?
Who should collect the data – the surgeons or independent reviewers?
Are surgeons going to be tabloid fodder?
Is it simply time to head for the hills with a cigar and a bottle of red…..
In cardiac surgery all these were live questions and they have worked it out. Standards have been shown to be astonishingly high. Will urology be the same? The specialities have been challenged to come up with a plan. Why not contribute to that process and add to the blog….
Tim O’Brien is a Consultant Urological Surgeon at Guy’s and St Thomas’s NHS Hospital, London. His views are his own.
Comments on this blog are now closed.
Being discussed at BAUS council today.
Can’t believe my colleague O’Brien is in fact Bruce Groebellar and has been masquerading for 10 years in my department!!
Tim O’Brien as Chair of BAUS Oncology Committee has made a statement which largely reflects the views of many urologists in the country. Reaction to such news is expected to be mixed. Those who have been disciplined enough to maintain records of their performance and contributed to the national audit on voluntary basis would welcome the initiative. Others may have lot of catching up to do and may resent the idea.
The concerns Tim has raised regarding outcomes being perceived as poor of surgeons operating on high risk patients are genuine. Therefore it is vital that performance of the surgeon is taken in the context of patients’ co-morbidities, and complexity of the pathology while judging the outcomes. Concerns about patients being declined operations because of these factors may be real but one has to learn from experience of cardiac surgeons whether such a trend was observed and improved outcomes are due to careful case selection or real improvement in care.
Since audit has been declared mandatory for certain procedures it is only a matter of time until that spectrum would be widened in near future. Therefore irrespective of our specialist area of practice, let us make it a routine to record accurately what we do. It would be ideal to have independent data collection but at present this may simply be a wishful thinking.
brilliant article in this weeks BMJ from Ben bridgewater and James roxburgh et al on the lessons learnt from the cardiothoracic society’s reporting of outcomes .BMJ 9th March 2013; vol 346 pp 19-21. every surgeon in the uk should read it.
really interesting… . . clearly a journey. not perfect at the start but a start. improve the data collection every year; consolidate, be realistic and honest
we also need to define the endpoints that matter and are easily measurable.
declan cahill would only collect length of stay and positive margins in T2 for radical prostatectomy. length of stay measures complications, positive margins in T2 measures surgical skill. what would you collect ?
The performance status of the patient is one of the variables being recorded. So we can only hope that case mix and co-morbidities will be accounted for.
Great blog Tim. Rather than feel threatened by the inevitable move to transparent and public reporting of key outcome measures in surgery, specialty societies need to engage with the process and provide good leadership to ensure reasonable metrics are being used.
As we also see the “Urban Spoon-ification” of healthcare with consumers rating and commenting on their experience on various websites, it is also very important to present the public with appropriate factual information to help them assess the quality of service being provided. Nothing wrong with that
With respect to radical bladder cancer surgery % of surgical margins, extent of lymphadenectomy and 90 day mortality may be considered more realistic.
Vital that we embrace this rather than feel threatened by it. Think the key is providing a few key metrics for each operation that provide a fair reflection of performance without producing “information overload” for the public. Less is more.
Protecting training is also important. Vital to monitor proportion of cases made available for training to ensure this does not suffer.
Great blog Tim
For partial nephrectomy: Clavien Grade III/IV compllications, positive margins, warm ischaemic times. clearly depends hugely on case mix and whether elective or imperative situations.
Data collection must be independent as bad surgeons could deliberately under-report problems and this would be hard to prove.
The BMJ article is here
Ben
I agree that independent data collection would be the ideal but I don’t see it happening anytime soon. In the meanwhile, I think that this is at least a step in the right direction. I share your concern about the validity of the data submitted (or not), though. This might be another role for MDT co-ordinators: to verify that all data fileds are completed, especially complications.
Chris. You are an excellent surgeon. Consider this scenario:
Because you are an excellent surgeon you get sent the more difficult cases other people don’t want to do. As a result when the Sunday Times publish their league tables of prostatectomists you appear half way in the league table. You know that the people above you are the people who have turned down the difficult cases.
But how do the public interpret that ?
Matt
PS I am in favour of this by the way but does need to be done properly
As surgeons we should be counselling patients using personal outcome figures, and it follows that these figures should be available publicly. Personally I strongly support the BAUS database, however, as it currently stands it does not collect the nuances of data that will distinguish good and bad outcomes. This is a personal view, but also seems to be held widely. It seems wrong to loose this dataset to the public domain, before we are happy with the validity (meaning how and what data is collected, who collects and returns it) of its contents, and worse to do it at several months notice.
Where is the data going to come from? Does the individual surgeon record complications that occur with every case? Does the individual surgeon then record whether it was a teaching case? Does the individual surgeon keep a record of all his operation then?
What I have gathered from discussing with those involved is that the data is not going to be self-reported but will be collated from the HES data (unless there is a national registry). What is worrying about HES data is that it is riddled with mistakes. When discussing about this blog with one senior orthopaedic surgeon in theatre today, he cautioned that ‘Mail Online’ (which seems to be the most popular among Urologists!) would publish a shocking story about you in the front page and when it finds out that the data was wrong, there will be a small apology in the 7th page somewhere between the classifieds.
Yes, I agree that the principle of league tables may be sound, but until robust system is in place for accurate data collection, the half-baked information will be utilised for creating headlines rather than improving patient care!
As a DGH urologist I think its worth bearing in mind that since the IOG guidance there are many DGH urologists who don’t perform any of the operations on the complex operations database. How will they be measured? We have to start somewhere but we should look at including a common operation in the near future so that all urologists are obliged to report their results for something. No surprises that I think TURBT would be a good one – Its the commonest oncological operation that we do and we know the results are variable and dependent on technique. As it has few complications, the outcome would be relatively simple – 3 month recurrence rate
Colleagues,
Although the underlying principal is laudable and correct that our accurate and balanced data is widely available in the public domain what concerns me is the rapid implementation of this. As previously mentioned the current plan of submitting a few selected databases will not accurately reflect the practice of all urologists.
The current rate of implementation doesn’t allow anytime for the validity and accuracy of the data to be assessed. This then leads to a scenario where we are at risk of publishing data that is not homogeneous among those who have been selected by their area of operative practice rather than accurately reflecting urologists practice as a whole.
We all know that bad data is worse than no data at all and without some safeguards as to the accuracy of data entry we are at risk of publishing poor inaccurate data. This has always been the problem with the Baus data sets in that previously their voluntary nature has made analysis of them for national performance unreliable. (good surgeons input data bad surgeons might not?)
Now we are making data entry compulsory we must have some system of cross checking data prior to it being released into a public domain to ensure all cases are entered, sufficient data is entered to allow individual performance comparison and that the data collected reflects case mix so as to not disadvantage both patients and surgeons.
Can we really safely achieve this in the very limited timescale, that has been forced on us? If inaccurate or poorly balanced data is published the potential harm to an individual or institution may take years to repair.
Anyone under any illusions about what may be on the way and how ugly it could get needs to read the BBC news today re the cessation of surgery in the children’s heart unit in Leeds
https://www.bbc.co.uk/news/health-21974053.
Urologists need to contribute to this process and we need to develop robust methods for measuring success and failure in the operations that are chosen to be studied. otherwise there will be a lot of uncertain patients and unhappy urologists or vice versa !!
I remain perplexed that a paediatric cardiology surgical service (Leeds) can be shutdown stat before being opened again 2 weeks later. Two concerns : i) if the data to ensure competence could be obtained that easily i.e within 2 weeks, was there any need to shut it down in the first place ? or ii) is any data, that can be obtained within 2 weeks, robust enough to re-open a unit that has been closed down ?
Clearly the competence and completeness with which data is collected and handled is vital in this whole process of open publication of results. Which brings me to an interesting chat with a paediatric cardiologist in the sunshine over lunch…
the care with which data is now handled in paediatric cardiology depts is TOTAL. the surgeons enter their data; this is checked by the data manager who tweaks it and adds relevant multiple other factors. This is then triple checked before final submission. The process is time consuming but it is the only way to get it right. Each dept is sent samples of returns from other units to check accuracy of entry i.e. self policing…
One data manager can handle about 400-500 cases a year. This is clearly what will be required in urology if it is to be meaningful. Start writing the job descriptions….