Article

Evaluating ChatGPT’s Semantic Alignment with Community Answers: A Topic-Aware Analysis Using BERTScore and BERTopic

2025-04-23

Abstract excerpt

This study evaluates the semantic alignment of ChatGPT’s responses with human-selected best answers in an open-domain question answering (QA) setting, using data from the Yahoo! Answers platform. Unlike prior research focused on domain-specific or educational datasets, this work investigates ChatGPT’s general-purpose QA capabilities across a diverse topical landscape. We apply BERTopic to extract latent themes fro...

Topics

Open a Topic to create a Post that cites this publication.

COVID-19 diagnosis using AI

Identifiers and source

Literature Corpus work: e0fef16c-513e-5f2a-8691-93c3f57fe2ae
DOI: 10.20944/preprints202504.2000.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Evaluating ChatGPT’s Semantic Alignment with Community Answers: A Topic-Aware Analysis Using BERTScore and BERTopicDOI 10.20944/preprints202504.2000.v1Read

ChatGPT vs. Bard: A Comparative StudySemantic proximity Read

KM-Chat: A Large-Scale Synthetic Question-Answer Dataset for Open-Domain Conversational AISemantic proximity Read

Enhancing Chatbot User Satisfaction: A Machine Learning Approach Integrating Decision Tree, TF-IDF, and BERTopicSemantic proximity Read

ChatGPT's Expanding Horizons and Transformative Impact Across Domains: A Critical Review of Capabilities, Challenges, and Future DirectionsSemantic proximity Read

A Categorical Archive of ChatGPT FailuresSemantic proximity Read

Conversations From Make-Believe: An Attentive Encoder–Decoder Chatbot Trained on Scripted DialogueSemantic proximity Read

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models: A Case Study on ChatGPTSemantic proximity Read

LLMs4All: A Review of Large Language Models Across Academic DisciplinesSemantic proximity Read

Chat-WanJuan: A Fine-Tuned Compact Language Model for Multiple- Choice Question Generation in Medical EducationSemantic proximity Read

Generative AI Chatbot Using CSV FilesSemantic proximity Read

Dynamic Multi-Hop Retrieval-Augmented Generation Framework for Professional Domain Question AnsweringSemantic proximity Read

ChatGPT: A Brief Narrative ReviewSemantic proximity Read