Medical AI Eval
Medical AI Evaluation and Modeling
A cross-institution medical AI project covering NIH R01 text evaluation and osteoporosis modeling with migration validation.
medical AILLM Evalmodelinghuman baseline
Problem
Medical AI work is fragile when text-review criteria, expert baselines, data missingness, and cross-institution drift are not made explicit.
Workflow
- 01Preprocess NIH R01 proposal text and call six models under controlled evaluation conditions.
- 02Collect expert-professor scores as the human baseline for comparing model bias and stability.
- 03Build an end-to-end osteoporosis modeling pipeline with feature engineering and ensemble models.
- 04Validate across China / US and institutional data differences while using low-barrier physiological features.
Evidence
Collaborators
Project context involved UTHSC and St. Jude domain experts; public proof keeps sensitive research material private.
NIH R01 evaluation
Six models and 23 research proposals were evaluated against expert scoring as human baseline.
Osteoporosis modeling
Cross-country / cross-institution migration validation addressed inconsistent standards and missing clinical data.
Boundary
- This does not replace clinical expert judgment.
- The public site does not publish patient data, raw proposal text, model outputs, or manuscripts under submission.
- The project is NIH R01 text evaluation plus osteoporosis modeling; it is not a pancreatic-cancer project.
Role Mapping
- LLM Eval: turns subjective text judgment into auditable rubric and baseline comparison.
- Healthcare data product: handles sparse, shifted data and clinically meaningful low-barrier features.
- AI-assisted decision systems: keeps human baselines and review boundaries explicit.