Artificial Intelligence in Graduate Medical Education Applications

摘要

I am not a robot, and I can prove it…just by clicking on all the picture tiles with bicycles or buses. These CAPTCHA tests, or Completely Automated Public Turing tests to tell Computers and Humans Apart, protect websites from bots and spam. Will residency program directors soon need their own form of CAPTCHA to protect the integrity of the residency application process from artificial intelligence (AI)? The 2023-2024 residency application season is the first in which generative AI has been a popularly recognized and easily accessed tool. While its use has advantages, it also threatens the sincerity and reliability of important data points in application evaluation.In November 2022, the San Francisco-based tech company OpenAI released ChatGPT—a free online chat bot capable of writing prose that is nearly indistinguishable from human-generated text. It is now one of several such services available online. Generative Pre-trained Transformer (GPT) describes large language model artificial neural networks that are “trained” on large data sets and can perform natural language processing tasks such as generating new text and analyzing text for specified content.1 After the March 2023 update, OpenAI announced on its website:The use of AI was quickly adopted by university students. An informal survey of nearly 5000 Stanford students found that 17% used ChatGPT in fall 2022 for assignments and examinations (the term in which ChatGPT became publicly available).3 Over 25% felt that AI use, even if answers are unedited, was not a violation of honor code.3 While there are no data on AI use by health professional students, one must assume parallels with the general university population.Because AI use is widespread at universities, many applicants will consider using it when writing a personal statement (PS). The PS has long been the applicant’s opportunity to narrate their journey through medical school and to their specialty of choice, and to highlight their unique qualities and motivations.4 Using AI to assist in writing them can improve efficiency, help with brainstorming, and improve communication for people for whom English is a second language.5 It also improves equity in access to application assistance services, which can cost in the thousands of dollars.6On the other hand, multiple online forums advise against using AI for the PS because it seems “inauthentic and unoriginal—and therefore not as good as” an applicant’s own work.7 An AI-written PS lacks the narrative voice, context, and specific details that make it personal.8 Another notable drawback of AI-generated work is its propensity for lengthy and potentially excessive text, a factor that poses a barrier to reader efficiency. No data yet exist on the applicant’s perspective on the use of AI in PS generation.Through letters of recommendation (LORs), faculty advocate for students and highlight their longitudinal relationships.9 Similarly to the PS, AI can assist in generating an LOR, which can improve writing efficiency and reduce the use of biased language,10 with which LORs have long been fraught.11 There are no studies of how frequently AI is used to generate a professional school LOR. An article in The Atlantic, “The End of Recommendation Letters: Professors, Like Their Students, Use ChatGPT to Get Out of Doing Their Assignments,” reveals one popular sentiment against the use of AI in letter writing.12 Because AI-generated letters usually lack the personal connection, emotional intelligence, nuances, and subjective judgement of a human-written letter,13 heavy editing is advisable if AI is used for the first draft. If AI is used in text generation, authorship is not attributed to the AI; at least this is the stance for scientific journals.14 Ultimately, the signer of an LOR takes responsibility for its content, which has always been the case.Program directors may also look for assistance from AI. Application inflation necessitates improved efficiency in application review.15