Month: August 2024

As we continue our work on our project, we’re excited to share the latest developments on SimUNet, which we will use in field experiments to test our AI models with OPO professionals.

During this year, we’ve focused on understanding the OPO context and identified core features needed for SimUNet to serve as an effective research tool. SimUNet’s original interface was designed around transplant center clinicians evaluating and responding to kidney offers. The upcoming updates will introduce a more flexible architecture that will allow researchers to create more customized study designs and the ability to conduct experiments in an OPO context.

We’ve already made significant progress in updating the contemporary design of SimUNet and adding important administrative tools to help manage the studies. Currently, the UNOS Software Engineering team is updating SimUNet to include more robust researcher features and adding the OPO study designs.

Our Next Steps:

Our immediate focus is on developing detailed OPO scenarios. These scenarios will simulate the decision-making points of OPO staff, allowing us to see how the tools we are building can assist them in their critical roles. We are also looking to identify key events when the AI tools have the most potential to help our introduce efficiencies.

We plan to conduct a focus group and testing with OPO professionals to refine the tool. Once the prototype of SimUNet OPO is ready, we will continue to refine and test the tool during the first half of 2025.

Why This Matters:

The enhancements we’re making to SimUNet will allow us to empirically demonstrate that what we are developing provides OPO staff with more intuitive tools to manage organ donation processes. By incorporating our AI models, we aim to streamline their workflows, improving both efficiency and outcomes in organ transplantation. Our goal is to start the field experiments in September 2025.

We are excited about the potential impact of this project and look forward to sharing more updates as we move closer to our study start date in September 2025.

Stay tuned for further developments as we continue to refine and test the improvements to SimUNet.

Contributor: Brendon Cummiskey (UNOS)

Please join us at the AI Ethics Symposium: Bridging Disparities in Health Care Using Artificial Intelligence on Friday, November 22, 2024 from 8:30 am to 5:00 pm.   

Location: II Monastero, Saint Louis University, St Louis, MO 

Registration: Free! Please register via: https://forms.gle/kB6tytzJYwKDdJC68  

Physicians, researchers, and scholars, including graduate and professional students, are invited to submit proposals on topics that pertain to practical and ethical opportunities and challenges related to AI’s capacity to address inequality and promote fairness in health care, particularly with respect to organ transplantation. In addition, proposals for panels representing public stakeholders, such as health policymakers and patient populations, would be especially welcome.  

To learn more, please see https://sites.mst.edu/aifortransplant/symposium/

We look forward to seeing you!

AI systems are widely used in healthcare for applications ranging from medical image scan analysis to cancer detection. Human-subject studies are essential for understanding the effects of AI in specific tasks or domains. For us, these experiments are crucial for assessing the impact of Explainable AI (XAI) on factors like task performance, user trust, and user understanding. Conducting these experiments enhances our understanding of human-AI interaction, ultimately shaping AI systems to be more effective and suitable for specific domains, users, and tasks. However, to develop theory, we need to conduct a large number of experiments to iterate on designs.

Unfortunately, in the healthcare domain, it is difficult to conduct a large number of human-subject experiments. Experts, such as doctors and nurses, are busy and don’t have time to participate in multiple studies. Therefore, we find it useful to conduct more general experiments in a non-healthcare context (i.e., an analogy domain) where we can use an online participant pool. Once we find designs that work well in general, we will be able to test them in the kidney transplant context via a SimUNet field trial.

Identifying an Appropriate Analogy Domain

In the kidney transplant process, the proposed AI system will be utilized by surgeons and coordinators with expertise in matching donors and recipients. Therefore, the analogy domain should enable people to leverage their knowledge to complement the AI, rather than blindly accepting its predictions. Expertise can be assessed through self-reported measures, career-related information, and objective knowledge tests.

The analogy domain must also have tabulated information, like the dataset used in the kidney transplant process, which contains elements such as donor age, gender, and serum creatinine levels. The outcome measure should be predictable, not random, allowing users to make informed decisions. Domains like fantasy football or sports betting are unsuitable due to high randomness.

Finally, the domain should involve subjective truth – a decision made on personal expertise and risk preferences, where two experts could genuinely disagree about the appropriate path forward. In the kidney transplant process, surgeons’ decisions to accept or not accept a donor kidney are used as the ground truth in AI’s training. This is different from an “objective truth,” such as the health outcomes that a patient realizes in the future. In many cases, such as when a transplant is not conducted, we don’t know what that objective truth would have been.

Based on these characteristics, we have developed an experimental task in the real estate domain. The decision-making process in real estate involves the assessment of multiple attributes and expert knowledge. This similarity makes it a useful domain for studying how AI explanations affect user decision-making. Many people have familiarity with the process of buying a house, so they understand the decisions that go into it. For real estate, there is data on the house as well as the buyer, which have to go through a matching process. This allows us to test different XAI formats that can eventually be translated to the kidney transplant context.

Contributors: Harishankar V. Subramanian (S&T), Casey Canfield (S&T)