Methodological Considerations for Large Language Model-Based Symptom Extraction in Neuro-Oncology Electronic Health Records.

Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks, including information extraction from complex, unstructured text. A recent study applied LLMs to extract symptoms of brain tumor patients from clinical notes in electronic health records.1 The authors first validated LLM performance using a manually annotated dataset, and demonstrated strong agreement with expert review, and then applied the validated pipeline to large-scale real-world data. These results highlighted the promise of LLMs for scalable symptom surveillance in routine clinical practice. Here, we note several methodological considerations as shown in Figure 1, which may help clarify the interpretation of the reported findings and inform future applications of similar pipelines. First, the authors formulated symptom extraction as a predefined set of nine treatment-related symptoms, with additional experiments incorporating an “other” category to flag symptoms outside this set. While this design enables consistent benchmarking and strong performance, it constrains symptom capture to anticipated categories and limits coverage of clinically relevant symptoms documented in free text. From a clinical perspective, the authors explicitly excluded core disease-related manifestations common in neuro-oncology, such as seizures, aphasia, and focal weakness, to focus on treatment-related symptoms.2 However, these disease-driven symptoms affect a substantial proportion of brain tumor patients and may overlap clinically with treatment toxicities (e.g. tumor-related headache versus treatment-associated cerebral edema). As a result, symptom extraction limited to predefined categories should be interpreted as identification of explicitly documented target symptoms rather than comprehensive symptom discovery.3 In addition, reliance on a single expert annotator precludes assessment of inter-rater variability for subjective or ambiguously documented symptoms. Framing symptom extraction as an open-ended process where symptom mentions are first identified and subsequently normalized using ontology-driven mappings, could help distinguish disease-related and treatment-related symptoms and improve generalizability without altering the underlying data source.