PERFECTS EQA – PERFormance Evaluation for CT Screening
How does it work?
The PERFECTS EQA scheme (PERFormance Evaluation for CT Screening) is a self-assessment and training platform aims to ensure appropriate interpretation of CT scans, to benefit patient outcome, streamline clinician workload and optimise resource allocation.
All participants report a randomised series of collated test sets of cases and automatically receive immediate detailed multimedia interactive feedback comparing their case decisions anonymously to the experts and all other users.
What to expect?
The Targeted Lung Health Checks (TLHC) programme is a new and ground-breaking programme of work in England which aims to diagnose people who have lung cancer at an earlier stage, whilst also correctly identifying those who do not have the disease. The reading of low-dose thoracic CT scans for the TLHC programme is a highly specialised skill that requires clinical experience and judgement, alongside a high perceptive ability.
The PERFECTS EQA scheme is looking to develop these skills further and provide a quality assurance mechanism to ensure radiologists can develop and learn from their reporting practices and highlight areas that require additional training and support.
The PERFECTS EQA scheme is funded by the NHS England and NHS Improvement.
About Version 1 of PERFECTS EQA
Thank you to those who have registered and started to use the PERFECTS EQA (PERFormance Evaluation for CT Screening) platform.
PERFECTS EQA is an external quality assurance tool that is mandated and funded by NHSE/I as part of the TLHC QA standards, specifically standard 15. It has to be completed by readers by May 31st 2023. PERFECTS EQA provides QA validated against reference cases that have been reviewed by a panel of expert radiologists. The process is for 100 exercises (not cases) to be completed each year. This should take 4-6 hours in total and may be completed at different times to avoid fatigue; 100 exercises are required to ensure statistical validity.
PERFECTS EQA complements annual and quarterly radiology reporting audit data as part of the radiology QA process. However, it is different to “real-time” audit because it is focuses on evaluating specific screening reading skills and knowledge, using validated cases with outcomes reviewed by an international panel of expert readers.
- PERFECTS EQA is an important educational tool as it provides immediate feedback on carefully selected challenging exercises.
- It can scientifically evaluate reader performance.
- It provides participants with information about their reading performance, and confidentially provides a comparison with peers and (inter)-national level experts.
- PERFECTS EQA is a faster process to identify any areas where personal improvement could be made. It helps to identify any performance issue early and gives the opportunity to improve practice before any clinical consequences arise. This supports radiologists to ensure their practice is safe. The similar tool for breast cancer screening (PERFORMS EQA) has been shown to correlate with real life data and provide rapid identification of suboptimal performance which can take years to come to light before being identified through “real-time” audit, particularly where reporting activity is low [1,2].
- Key metrics such as sensitivity are easily calculated by PERFECTS EQA by virtue of the enriched cases, but very difficult to obtain from real-life audit [1,2].
- PERFECTS EQA goes a step further than PERFORMS by testing management recommendations.
- This scheme provides 10 CPD credits in accordance with the CPD Scheme of The Royal College of Radiologists. It can therefore form part of the radiologist’s annual appraisal.
PERFECTS EQA is a new tool that complements the new targeted screening programme for lung cancer. Version 1 will be followed by a second version that will incorporate the following:
- Multiplanar reconstruction series.
- Series that incorporate CAD of nodules although the detection exercise specifically addresses nodules that may be missed by CAD (it should be noted CAD should always be utilised as a concurrent or second reader).
It may also include, depending on progress:
- Incidental findings detection and management.
- Additional quantitative AI.
- Specific cases that may be required following evaluation of version 1.
[1] Chen et al. (2020). The Relationship between Mammography Readers’ Real-Life Performance and Performance in a Test Set-based Assessment Scheme in a National Breast Screening Program. Radiology: Imaging Cancer,2(5):e200016. doi: 10.1148/rycan.2020200016
[2] Chen et al. (2021). The relationship between missed breast cancers on mammography in a test-set based assessment scheme and real-life performance in a National Breast Screening Programme European Journal of Radiology, 142:109881, doi: 10.1016/j.ejrad.2021.109881
Latest News
– Dr Yan Chen presented at BTOG 2023 in the BTOG Respiratory/Radiology Symposium at 11:00 on Wednesday 26th on the subject of Quality assurance for a future national screening programme. You can access all the videos and presentations from 21st BTOG Annual Conference 2023 held in Belfast on 26th to 28th April 2023 by becoming a member (HCP membership is free) – https://www.btog.org/register/btog-members/
High standards of radiology reporting are the backbone of a successful lung cancer screening programme. The job of PERFECTS is to establish, maintain and improve those standards.
PROF. DAVID BALDWIN, HONORARY PROFESSOR OF MEDICINE AND CONSULTANT RESPIRATORY PHYSICIAN, NOTTINGHAM UNIVERSITY HOSPITALS AND UNIVERSITY OF NOTTINGHAM
