Article

Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation

2024-10-30

Abstract excerpt

This work focuses on the efficiency of the knowledge distillation approach in generating a lightweight yet powerful BERT-based model for natural language processing (NLP) applications. After the model creation, we applied the resulting model, LastBERT, to a real-world taskclassifying severity levels of Attention Deficit Hyperactivity Disorder (ADHD)-related concerns from social media text data. Referring to LastBE...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 63870417-fa87-502f-a571-0bcd124a2097
DOI: 10.1101/2024.10.30.24316411

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillationDOI 10.1101/2024.10.30.24316411Read

Upscaling A Smaller LLM to More Parameters via Manual Regressive DistillationSemantic proximity Read

Boosting Domain Knowledge Understanding of LLMs through Fine-Tuning with a Novel KNN AlgorithmSemantic proximity Read

Knowledge Accuracy and Reducing Hallucinations in LLMs via Dynamic Domain Knowledge InjectionSemantic proximity Read

Zhixin: An Instruction-Tuned Large Language Model for Mental Disorder DiagnosisSemantic proximity Read

Text Classification of German Mental Healthcare Data: A Hands-on Tutorial on Natural Language Processing Techniques.Semantic proximity Read

SuperGLUE: The AI raceSemantic proximity Read

Large-Scale Psychiatric Concept Extraction from Electronic Health Records: A Comparative Study of Encoder-Based Language ModelsSemantic proximity Read

A Comprehensive Review of DeepSeek: Performance, Architecture and CapabilitiesSemantic proximity Read

MedThink: Enhancing Diagnostic Accuracy in Small Models via Teacher-Guided Reasoning CorrectionSemantic proximity Read

Synthetic Data Distillation Enables the Extraction of Clinical Information at ScaleSemantic proximity Read

Semantic Contextual Embedding for Domain-Specific Task Transfer in Large Language ModelsSemantic proximity Read

REFINE: Closing the Loop Between Large Language Models and Symbolic Rules in Clinical NLPSemantic proximity Read