NCI Data Jamboree (Project Abstract Submission): Submission #38

serial: '38'
sid: '189004'
uuid: 8cc7a68a-4501-443a-a361-e5a02dffd307
uri: /nci/datajamboree/abstractsubmission
created: '1785076838'
completed: '1785076978'
changed: '1785159546'
in_draft: '0'
current_page: ''
remote_addr: 10.208.24.192
uid: '0'
langcode: en
webform_id: nci_data_jamboree_abstracts
entity_type: node
entity_id: '2272'
locked: '0'
sticky: '0'
notes: ''
metatag: meta
data:
  list_of_additional_authors:
    - add_author_letters: Ph.D.
      affiliation: 'Central Florida University'
      first_name: Jaeyoung
      last_name: Park
    - add_author_letters: 'PharmD, Ph.D. Student'
      affiliation: 'West Virginia University'
      first_name: Samuel
      last_name: Ayemere
    - add_author_letters: 'PharmD Cadidate'
      affiliation: 'West Virginia University'
      first_name: Abdullah
      last_name: Anbari
  category: 'Developing, refining, or validating tools, methods, algorithms, and pipelines'
  degree_s_: 'Ph.D., M.A., M.S.'
  email: lei.guo@hsc.wvu.edu
  first_name: 'L. Raymond'
  keywords_abstracts: 'Algorithmic Bias, Head and Neck Cancer, Digital Twins, Large Language Models'
  last_name: Guo
  middle_initial: ''
  organization: 'West Virginia University'
  organization_address:
    address: ''
    address_2: ''
    city: Morgantown
    country: ''
    postal_code: ''
    state_province: ''
  summary: |
    Scientific & Technical Questions: Large Language Models (LLMs) are increasingly explored for clinical decision support, yet their vulnerability to hallucination and demographic bias in oncologic decision-making remains underexplored. This project constructs EHR-derived patient digital twins, synthetic, representative clinical profiles synthesized from real-world health records, to stress-test open-source local LLMs (e.g., Llama 3 via Ollama). We evaluate model accuracy, hallucination rates, and algorithmic bias when processing clinical narratives, treatment protocols, and prognostic reasoning in Head and Neck Squamous Cell Carcinoma (HNSCC) across varying HPV statuses (p16 positive vs p16 negative) and demographic variables.

    Community Impact: HNSCC management critically depends on HPV stratification under AJCC 8th edition guidelines. Using synthetic digital twins provides a privacy-preserving framework to identify where generative AI fails across demographic sub-populations (e.g., age, sex, rural/urban disparities) before deploying LLM tools in real-world clinical workflows. This approach will also explore and characterize gaps and limitations in general EHR resources such as MIMIC-IV for specialized oncology AI applications.

    Datasets & Repositories: We will utilize MIMIC-IV (and MIMIC-IV-Note/ED), a publicly accessible, de-identified real-world Electronic Health Record (EHR) database hosted on PhysioNet, to extract clinical parameters and construct HNSCC patient digital twin profiles.

    Tools & Computing Requirements: The workflow leverages local LLM deployments (Ollama / HuggingFace), Python (PyTorch, Pandas, Scikit-learn) for synthetic digital twin generation and statistical benchmarking, and GitHub for open-source code sharing. Laptop or cloud compute with local GPU acceleration is sufficient.

    Assembled Team: We have assembled a multi-disciplinary team from West Virginia University comprising faculty expertise in health informatics, real-world evidence, and pharmaceutical policy (Dr. L. Raymond Guo [Lead], Dr. Jae Park [Co-Investigator]), a PhD student in Health Services Research, and a PharmD student specializing in clinical oncology regimens.) 
  title: 'Assistant Professor'
  ttile: 'Stress Testing Open-Source Language Models on Head and Neck Oncology Cohorts: A Digital Twin Benchmark Framework'