Article

Comprehensive Evaluation and Design Strategies for Japanese Counseling AIs Using Large Language Models

2025-05-19

Abstract excerpt

<p>This study simultaneously evaluated three types of systems in Japanese counseling dialogues: counselor AIs (GPT-4-turbo in a zero-shot setting with a Structured Multi-Dialogue Prompt [SMDP] and Claude 3-Opus with SMDP), a client AI, and counseling-evaluation AIs (o3, Claude 3.7-Sonnet, and Gemini-2.5-pro). Introducing the SMDP significantly improved human experts’ global ratings based on the Motivational Interv...

Topics

Open a Topic to create a Post that cites this publication.

Uncategorized literature

Identifiers and source

Literature Corpus work: 6ca1212f-aa04-5a98-804a-6599b844f3ce
DOI: 10.31234/osf.io/j2fsq_v4

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Comprehensive Evaluation and Design Strategies for Japanese Counseling AIs Using Large Language ModelsDOI 10.31234/osf.io/j2fsq_v4Read

Comprehensive Evaluation and Design Strategies for Japanese Counseling AIs Using Large Language ModelsSemantic proximity Read

Comprehensive Evaluation and Design Strategies for Japanese Counseling AIs Using Large Language ModelsSemantic proximity Read

Comprehensive Evaluation and Design Strategies for Japanese Counseling AIs Using Large Language ModelsSemantic proximity Read

Large Language Models Acting as Psychotherapists: A Scoping ReviewSemantic proximity Read

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation BurdenSemantic proximity Read

Deterministic Retrieval-Grounded Language Models for Clinical Counseling: Large-Scale Multilingual Evaluation with Cryptographically Verifiable PipelinesSemantic proximity Read

ΔAPT: Can we build an AI Therapist? Interdisciplinary critical review aimed at maximizing clinical outcomes using Large Language Models for AI Psychotherapy.Semantic proximity Read

A Study on the Accuracy of Pre-Treatment Consultation Responses for Adult Orthodontic Patients Based on Large Language ModelsSemantic proximity Read

Conversational trajectory degrades large language model detection of suicidal ideation relative to clinicians: a preregistered studySemantic proximity Read

Introducing CounseLLMe: A dataset of simulated mental health dialogues for comparing LLMs like Haiku, LLaMAntino and ChatGPT against humansSemantic proximity Read

Human vs. Machine: Comparing AI-generated and Human-written Psychological ReportsSemantic proximity Read

Human vs. Machine: Comparing AI-generated and Human-written Psychological ReportsSemantic proximity Read