The Problem
Lung transplant program performance is graded on one-year survival yet current methods of predicting post-transplant mortality are poor. A lung allocation score (LAS) exists at a national level, using overall data reported by all US lung transplant centers, and is determined by the United Network of Organ Sharing (UNOS). This tool is used to help with organ allocation for lung transplant patients, but transplant surgeons and pulmonologists report that it does not provide a satisfying prediction of post-transplant mortality and does not address the risk of early post-operative complications. Notably, the congregated data and reports provided by UNOS are released infrequently (every six months); therefore, the most recent data could be from six months ago. Furthermore, the LAS calculation uses data between that most recent date and the prior eighteen months, such that the oldest data is two years old.
The Duke Lung Transplant Program is one of the largest lung transplant programs in the world with a diverse patient population with more than 2,250 lung transplants since its inception in 1992. The Duke Lung Transplant Program’s higher risk (often declined at other centers) patient population may not be adequately represented with the current LAS model. Currently, the program hosts a weekly committee review in which the extensive evaluation testing is done with inputs from social workers, psychologists, financial coordinators, and pulmonary rehabilitation specialists. Based on this review, the program subjectively deems some patients “high risk.” Early objective determination of high risk would lighten the effort required for the clinical review and, more importantly, help the program prepare patients for the transplant operation. Improving patient preparation and care execution during the transplant operation and hospital admission can dramatically affect the risk of 1-year mortality.
Our Solution
In an effort to address these shortcomings, we built a more comprehensive predictive model for lung transplant patients. In clinical practice, the goal of this model was to be able to provide real-time risk scores of patient mortality and post-operative outcomes to the clinical team during their pre-transplant committee review of the patient. The model was designed to: (1) predict post-operative “textbook”2 outcomes in the pre-transplant setting and (2) predict 90-day and 1-year mortality while incorporating data from lung pre-transplant clinic visits and earlier patient history. Textbook Outcomes were defined as freedom from intensive care unit (ICU) or hospital readmission within 30 days of surgery, organ rejection within 45 days of surgery, Grade-3 primary graft dysfunction within 72 hours, tracheostomy within seven days, inpatient extracorporeal membrane oxygenation (ECMO), inpatient dialysis, and reintubation/extubating more than 48 hours after surgery. This model will help inform and guide patient selection criteria and patient clinical care by stratifying high-risk patients.
The model was selected based on technical feasibility as well the potential impact of the model in clinical workflows. Features were built using patients’ inpatient and outpatient data including, but not limited to, demographic data (age, sex, race/ethnicity, comorbidities), laboratory/imaging data, vitals, and medication administrations during the encounters. LASSO and Random Forest models were tested.
The models predicting textbook outcomes after the lung transplant clinic and on the day of admission had AUCs of 0.69 and 0.70, respectively, and their precision was 0.53 and 0.45. A model predicting 90-day mortality at the time of admission had an AUC of 0.78 and an average precision of 0.16 (Figure 3).
Impact
The project outcome desired is to increase the percent of patients having textbook outcomes (no perioperative complications) from 25% to 30%. Primary textbook outcomes to aim for are to reduce post-operative hospital length of stay, time in the ICU, and tracheostomies. Improving textbook outcomes is expected to reduce hospitalization cost by both reducing the time spent in the hospital and procedures needed. A longer-term outcome and ultimate goal is to reduce the rate of 1-year mortality. Mortality and textbook outcome prediction at the time of committee review will accomplish this by guiding resource allocation decisions, preoperative interventions, surgical scheduling, and awareness of potential complications.
Next Steps
First, we will set and discuss thresholds at which the models best differentiate useful information about risk. Second, we will produce and validate model results for patients in real-time: each Monday on the last two weeks of lung transplant clinic visit patients, and upon admission for the transplant. Third, we will begin to incorporate the use of this model into lung transplant committee review meeting agendas. One person will assume responsibility for stewarding presentation of model outcomes, especially if early versions are tested for use in a digital interface outside Epic (like Tableau on the ACE-DIHI server). Committee review stakeholders will be educated about the model with documents and presentations from project clinical leads. They will have the opportunity and weeks to fully review and validate the model according to individual perceptions before they might use it in the context of their other clinical knowledge.
The next version of the prediction models should incorporate patient donor information and UNOS’ compiled data on Duke patients. We expect this data to strengthen prediction as well as increase its applicability to other lung transplant centers.
Academic Output
Drs. Snyder and Hartwig recently received a NHLBI U01 grant to expand on this model to other transplant centers and incorporate biological data in the prediction.
Development of a Machine Learning Model for Prediction of Mortality in Lung Transplant Patients. Poster presentation at Machine Learning for Healthcare (MLHC). August 2022, Durham, NC. Ochoa T, Knechtle W, Sendak M.
References
- Health Resources and Services Administration. (2022, January 6). Scientific Registry of Transplant Recipients: Program Specific Report. Scientific Registry of Transplant Recipients. Retrieved April 10, 2022, from https://www.srtr.org/PDFs/012022_release/pdfPSR/NCDUTX1LU202111PNEW.pdf
- Halpern SE, Moris D, Gloria JN, Shaw BI, Haney JC, Klapper JA, Barbas AS, Hartwig MG. (2021). Textbook Outcome: Definition and Analysis of a Novel Quality Measure in Lung Transplantation. Ann Surg. DOI: 10.1097/SLA.0000000000004916. Online ahead of print.
