Coastal painting by Paul Signac

Jingying Wang

Hi, I am Jingying Wang, a Ph.D. candidate from the Computer Science and Engineering department of the University of Michigan. I am co-advised by Prof. Xu Wang from the UM CSE, and Vitaliy Popov from UM Medicine.

My research lies at the intersection of Human-Computer Interaction (HCI) and medical education. I develop human-ai system for externalizing surgical expertise including perceptual cognitive skill, psychomotor skill, and attentional skill. My projects includes systems include multimodal sensing in OR, XR and computer vision, and interactive learning.

Jingying Wang

News

  1. Started my summer internship at Sony CSL Kyoto, mentored by Jun Rekimoto, working on CueOnGaze.

  2. Honored to be selected to receive the Barbour Scholarship! Article ↗

  3. Excited to kick off our new project, supported by the NSF award SCH: Multimodal Techniques to Enhance Intra- and Post-operative Learning and Coordination between Attending and Resident Surgeons, advised by Xu Wang, Vitaliy Popov, and Anhong Guo!

  4. Our paper “Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction” got an honorable mention award in CHI2024

Research & Publications

An iceberg illustrates increasingly tacit surgical expertise alongside surgeons using eye trackers in an operating room. More
articulated
Partially
tacit
Highly
tacit
eXplainMR (CHI ’25) SurgGaze
CueOnGaze compares an expert’s recorded actions and attention with a trainee’s live actions and gaze, providing audio and visual guidance when attention is misaligned during a tea ceremony.

CueOnGaze

Paper in submission

A gaze-guided learning system that compares a trainee’s actions and visual attention with an expert’s recorded action–attention template. When gaze is misaligned with the next task, the system provides audio and visual cues to guide attention to the relevant object.

Mentored by Jun Rekimoto.

SurgGaze gaze calibration method

SurgGaze: Implicit Calibration for Accurate Gaze Analysis in Operating Rooms with Wearable Eyetrackers

arXiv preprint · 2026

Jingying Wang, Rosiana Natalie, Keyuan Hu, Wenqian Xu, Brian George, Vitaliy Popov, Anhong Guo, Xu Wang

Paper ↗

SurgGraph surgical scene graph pipeline

SurgGraph: Quantitative Laparoscopic Video Understanding via Geometry-Grounded Scene Graphs

arXiv preprint · 2026

Jingying Wang, Rosiana Natalie, Marquise D Singleterry, Filippos Bellos, Brian George, Gurjit Sandhu, Jason J Corso, Anhong Guo, Vitaliy Popov, Xu Wang

Paper ↗

eXplainMR mixed reality ultrasound learning

eXplainMR: Generating Real-time Textual and Visual eXplanations to Facilitate UltraSonography Learning in MR

ACM CHI Conference on Human Factors in Computing Systems (CHI 2025)

Jingying Wang, Jingjing Zhang, Juana Nicoll Capizzano, Matthew Sigakis, Xu Wang*, Vitaliy Popov*

Paper ↗Video ↗

Surgment video based surgery learning

Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning

ACM CHI Conference on Human Factors in Computing Systems (CHI 2024)

Jingying Wang, Haoran Tang, Taylor Kantor, Tandis Soltani, Vitaliy Popov, Xu Wang

Paper ↗Video ↗

Surgeons’ visual needs during instruction

Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and Instruction

ACM CHI Conference on Human Factors in Computing Systems (CHI 2024)

Xinyue Chen*, Vitaliy Popov*, Jingying Wang, Michael Kemp, Gurjit Sandhu, Taylor Kantor, Natalie Mateju, and Xu Wang

Paper ↗

SketchSearch reference map exercises

SketchSearch: Fine-tuning Reference Maps to Create Exercises In Support of Video-based Learning for Surgeons

ACM Symposium on User Interface Software and Technology (UIST 2023) · Demo

Jingying Wang, Vitaliy Popov, and Xu Wang

Paper ↗Video ↗

Stylized facial animation

Real-time Facial Animation for 3D Stylized Character with Emotion Dynamics

ACM International Conference on Multimedia (ACM Multimedia 2023)

Ye Pan, Ruisi Zhang, Jingying Wang, Yu Ding, and Kenny Mitchell

Paper ↗

Automatic blendshape generation

Fully Automatic Blendshape Generation for Stylized Characters

IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2023)

Jingying Wang, Yilin Qiu, Keyu Chen, Yu Ding and Ye Pan

Paper ↗Video ↗