AI systems are widely used in healthcare for applications ranging from medical image scan analysis to cancer detection. Human-subject studies are essential for understanding the effects of AI in specific tasks or domains. For us, these experiments are crucial for assessing the impact of Explainable AI (XAI) on factors like task performance, user trust, and user understanding. Conducting these experiments enhances our understanding of human-AI interaction, ultimately shaping AI systems to be more effective and suitable for specific domains, users, and tasks. However, to develop theory, we need to conduct a large number of experiments to iterate on designs.
Unfortunately, in the healthcare domain, it is difficult to conduct a large number of human-subject experiments. Experts, such as doctors and nurses, are busy and don’t have time to participate in multiple studies. Therefore, we find it useful to conduct more general experiments in a non-healthcare context (i.e., an analogy domain) where we can use an online participant pool. Once we find designs that work well in general, we will be able to test them in the kidney transplant context via a SimUNet field trial.
Identifying an Appropriate Analogy Domain
In the kidney transplant process, the proposed AI system will be utilized by surgeons and coordinators with expertise in matching donors and recipients. Therefore, the analogy domain should enable people to leverage their knowledge to complement the AI, rather than blindly accepting its predictions. Expertise can be assessed through self-reported measures, career-related information, and objective knowledge tests.
The analogy domain must also have tabulated information, like the dataset used in the kidney transplant process, which contains elements such as donor age, gender, and serum creatinine levels. The outcome measure should be predictable, not random, allowing users to make informed decisions. Domains like fantasy football or sports betting are unsuitable due to high randomness.
Finally, the domain should involve subjective truth – a decision made on personal expertise and risk preferences, where two experts could genuinely disagree about the appropriate path forward. In the kidney transplant process, surgeons’ decisions to accept or not accept a donor kidney are used as the ground truth in AI’s training. This is different from an “objective truth,” such as the health outcomes that a patient realizes in the future. In many cases, such as when a transplant is not conducted, we don’t know what that objective truth would have been.
Based on these characteristics, we have developed an experimental task in the real estate domain. The decision-making process in real estate involves the assessment of multiple attributes and expert knowledge. This similarity makes it a useful domain for studying how AI explanations affect user decision-making. Many people have familiarity with the process of buying a house, so they understand the decisions that go into it. For real estate, there is data on the house as well as the buyer, which have to go through a matching process. This allows us to test different XAI formats that can eventually be translated to the kidney transplant context.
Contributors: Harishankar V. Subramanian (S&T), Casey Canfield (S&T)