Most nations with developed healthcare systems have a strong interest in audit, both for financial and clinical quality control. Whereas financial control has been a key political requirement for managing healthcare, the use of clinical outcome data has, until recently, taken more of a back seat.
Clinical audit has a long history of describing outcomes and challenging established attitudes or practice.1 Responses to published audits vary. Some clinicians voice criticism of bias as a result of selective reporting, either from a few units, or because of incomplete datasets.2 Attitudes have gradually changed with improved understanding of the role of audit as a tool to examine and refine standards of practice.3 This has been accompanied by a growth in clinical audit across all branches of medicine.
The turn of the century marked a shift towards more widespread clinical audit, with development of political interest in using quality to justify or contain costs. The advent of organisations such as the National Institute for Clinical Excellence (NICE) in the UK saw a growth in the use of research and audit to set standards both for outcomes and processes of care. A good example of this in vascular surgery is the NICE clinical guideline 68, which sets out clear standards for assessment, referral, and treatment of patients with TIA and minor stroke.4 These standards are incorporated into national audits in Europe and reporting now encompasses both outcomes and performance indicators such as timeliness of surgery and cranial nerve injury.5 Such reporting has driven improvement in quality of services by focussing clinicians on key components of high-quality pathways of care.
Vascunet was formed in 1997 as a collaboration of national registries in Europe, New Zealand, and the state of Victoria in Australia, with its first report produced in 2007.6, 7 Since then, the Vascunet group have published comparative data on carotid surgery,8, 9 abdominal aortic aneurysm,10 lower limb bypass,11 and popliteal artery aneurysm.12 One of the key features of these publications has been to describe the variation in clinical practice across neighbouring countries, notable examples being rates of surgery for asymptomatic stenosis and rates of lower limb bypass for intermittent claudication. Variation in outcomes is also reported at a national level.
The value of such reporting was demonstrated by the 2008 Vascunet report. This demonstrated outlying mortality rates after elective repair of abdominal aortic aneurysm surgery in the UK.13 This was a stimulus to a quality improvement initiative14 that sought to standardise practice and improve outcomes. The transparent publication of standards led to their widespread adoption both by clinicians and service commissioners within the UK. Recent publications have demonstrated a marked improvement in UK outcomes.15 This cycle of audit, analysis, standard setting, and re-audit demonstrates the improvement in quality that can follow acknowledgement of poor outcomes. This experience mirrors those in other clinical specialities such as cardiothoracic surgery.
Comparative audits suffer from a number of shortcomings, such as incomplete datasets with potential for bias and misleading interpretation. Most national registries rely on voluntary data contributions from practising clinicians who have varying levels of enthusiasm for audit. It is widely acknowledged that incomplete audit data is a source of bias and may give misleading messages. This has allowed some to ignore the messages from comparative audit, and may be the explanation behind some countries not wishing to participate in data analysis and publication. The recognition of this issue leads to the linking of Swedvasc (and the Helsinki datasets in Finland) data to national administrative datasets to improve accuracy. Similarly in the UK, data for carotid and aortic aneurysm procedures are now compared with national administrative datasets to demonstrate the quality of data. A secondary benefit has been an improvement in data quality, presumably as a result of peer pressure.
Vascunet has recently taken this further with external validation of data subsets in Hungary 16 and Sweden (ongoing project). This pilot demonstrated that national datasets can be validated by expatriate experts, providing an independent and even more robust measure of data quality. It is planned to extend this throughout the Vascunet registries group, if funding can be found.
Clinical audit data is one important source of information, about routine clinical practice, that can be used to highlight inconsistencies in clinical outcomes. The use of data, however incomplete, as an agent for change and as a guide for standard setting is established. We believe that the use of national audit data should be extended to drive quality improvement across geographical boundaries.
There are significant challenges to achieving this, but Vascunet believes that the time is right to embrace this. With increasing financial constraints on healthcare, clinicians need to be seen to lead on issues of quality of care. Part of this requires an open approach to measuring the standards of care, with the aim of improvement, rather than criticism. We believe that we have the support of patients in this aim and that transparent publication of data serves to both inform and educate in the debate about allocation of limited resources.
There is debate about what and how much data should be collected. Most enthusiasts approach clinical audit wishing to obtain a large amount of data to enable a detailed analysis of behaviour. The problem with this approach is that it inevitably relies on busy clinicians collecting the data, often after delivery of care. Unsurprisingly, the levels of enthusiasm for this vary hugely, with some seeing it as an intrusion on their relationship with the patient. The end result is incomplete and unreliable datasets and limited reporting.
An alternative is to collect small amounts of data about critical steps in the patient pathway of care. Such “key performance indicators” (KPI) can be linked to nationally collected administrative data to provide a moderately detailed account of the process of care. An example of this is seeking both outcome data (stroke and death) following carotid endarterectomy and collecting data on symptom to treatment time, to provide a more balanced picture of the quality of care. The resultant “less is more” approach allows for small datasets to provide important information to clinicians.
A number of factors determine how effective clinical audit is at changing clinician behaviours and patient outcomes. There is now a growing science around audit and feedback.17, 18 This states some factors that seem self-evident. For example there is evidence that audit and feedback can be made more effective by setting explicit goals and having a clear and realistic action plan, based on evidence about best practice. There needs to be clarity about the changes required and a commitment to multiple feedback cycles, with availability of peer group data for comparison. Repeated feedback delivered in both written and verbal format by people perceived to be part of the clinical team (i.e. part of the professional group) is much more effective in bringing about change than delivery by outside agencies (e.g. departments of health, commissioners of care). We believe that Vascunet can fill this role as it is made up of representatives from all participating audits.
Open reporting of data in a manner designed to support and encourage change, can be used to drive quality improvement by focussing on a small number of measures associated with a quality service. This has now been happening in many countries for some years. This approach has been used successfully for some years in Sweden, the UK and parts of the USA to demonstrate the quality of service and drive up standards by placing this in the public domain. The focus has been on care delivery within each country. We believe that the next step is for outcomes data analysis and quality improvement in vascular surgery to cross national boundaries, by common reporting of KPI for core vascular procedures. This is why Vascunet has begun validating national registries and has formed links with the North American Society of Vascular Surgery Quality Initiative (SVS-QI).
What is now required is clinical support to agree that it is in both our and our patients’ best interests to support a broadening of clinical audit to provide quality feedback across Europe. This will involve defining agreed datasets and seeking financial support to set up a data centre for analysis and reporting. Data collection should remain a local activity, owned by units and national societies, each committed to collaboration within the Vascunet group. We believe that the time is ripe to use the European registry experience to develop quality improvement initiatives throughout the European society nations and share our experiences openly to the benefit of our patients and clinical practices.