Artificial Intelligence Standards

Artificial Intelligence is ever-changing and evolving, so it is important to know the current standards and research around AI. Check out this page to learn more!

AI Research Interests: Division of Digital Psychiatry

Artificial Intelligence and mental health have been intertwined since the very first AI tools ever created. As such, the different domains of ongoing research in MH-AI are both widespread and varied. Our work focuses on a few main areas that we have identified as critical foundations to build out in an effort to facilitate future discoveries.

  • We have done a large amount of work completing thorough literature reviews and analyses across domains. These range from user-focused tools like chatbots to more historic and technical systems involving machine learning. We feel that much existing research has been siloed and advanced without appropriate pause to review the spread of what currently exists, and work hard to fill this ever widening gap.

  • We focus heavily on bias, stigma, and social impact through our research. Understanding the different preferences that artificially intelligent tools surface is a critical research area in AI alignment. Our work focuses on everything from simple output bias assessment all the way to more cutting edge interpretability research to ensure that we are learning about the biases that emerge at all levels of AI systems.

  • Current research efforts surround developing an evaluation schema for chatbots in the context of mental health/therapy. While we recognize there are a multitude of AI evaluation frameworks, our team brings unique expertise in evaluating digital mental health tools from a clinical and research-backed standpoint. However, the complexity of AI expands the nature of typical digital health tool evaluation — its tendency to mimic human behavior requires a tiered approach, where AI technology is evaluated as both a technology and human clinician. It is important to understand the underlying technological features of a chatbot, but also its ability to operate within the bounds of a licensed clinician.

    Furthermore, we aim to build our evaluation schema on the foundation of existing frameworks and provide concrete methods to assess necessary components such as safety, reliability, data quality, and transparency. Through this, we hope to facilitate responsible and informed AI development, integration, and use in mental healthcare.

Relevant Publications

2025:

Torous, J., & Greenberg, W. (2025). Large Language Models and Artificial Intelligence in Psychiatry Medical Education: Augmenting But Not Replacing Best Practices. Academic psychiatry : the journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry, 49(1), 22–24. https://doi.org/10.1007/s40596-024-01996-6

2024:

Blease, C., Worthen, A., & Torous, J. (2024). Psychiatrists' experiences and opinions of generative artificial intelligence in mental healthcare: An online mixed methods survey. Psychiatry research, 333, 115724. https://doi.org/10.1016/j.psychres.2024.115724

Chen, K., Lane, E., Burns, J., Macrynikola, N., Chang, S., & Torous, J. (2024). The Digital Navigator: Standardizing Human Technology Support in App-Integrated Clinical Care. Telemedicine and E-Health, 30(7), e1963–e1970. https://doi.org/10.1089/tmj.2024.0023

Flathers, M., Smith, G., Wagner, E., Fisher, C. E., & Torous, J. (2024). AI depictions of psychiatric diagnoses: a preliminary study of generative image outputs in Midjourney V.6 and DALL-E 3. BMJ mental health, 27(1), e301298. https://doi.org/10.1136/bmjment-2024-301298

Lee, C., Mohebbi, M., O'Callaghan, E., & Winsberg, M. (2024). Large Language Models Versus Expert Clinicians in Crisis Prediction Among Telemental Health Patients: Comparative Study. JMIR mental health, 11, e58129. https://doi.org/10.2196/58129

Kim, J., Leonte, K.G., Chen, M.L. et al. (2024) Large language models outperform mental and medical health care professionals in identifying obsessive-compulsive disorder. npj Digit. Med. 7, 193. https://doi.org/10.1038/s41746-024-01181-x

Torous, J., & Blease, C. (2024). Generative artificial intelligence in mental health care: potential benefits and current challenges. World psychiatry : official journal of the World Psychiatric Association (WPA), 23(1), 1–2. https://doi.org/10.1002/wps.21148

2020:

Wisniewski, H., & Torous, J. (2020). Digital navigators to implement smartphone and digital tools in care. Acta psychiatrica Scandinavica, 141(4), 350–355. https://doi.org/10.1111/acps.13149