ReflectLive

Real-time Analysis and Feedback of Communication Behaviors in Video Meetings to Support Better Communication.

ReflectLive is a real-time, WebRTC-based communication system designed to help participants improve the way they present themselves and communicate to others. To accomplish this, the system analyzes a meeting’s audio and video streams to provide real-time feedback to participants about their own non-verbal communication behaviors.

Video teleconferencing allows people to connect and communicate across distance, but users are not always aware of their own non-verbal behaviors (such as how much they speak, interrupt others, look away from others, etc.). These types of behaviors can have a large influence on communication, and the limited view of speakers and their environment in video meetings makes these non-verbal behaviors particularly salient.

We designed and built ReflectLive, a system that analyzes the audio and video streams of a WebRTC-based video conference and provides real-time feedback (in the form dynamic data visualizations) about a speaker’s own speaking amount, eye gaze, interruptions, face position, and other non-verbal behaviors important for effective communication. ReflectLive runs in each user’s browser, maintaining privacy and reducing additional use of bandwidth. With ReflectLive, we investigate whether real-time feedback presented live to each user during the video meeting can help them adjust their communication behaviors to match their own expectations or goals, while at the same time minimizing distractions from the feedback itself.

Technical Contact

Related Publications

2017
Publication Details
  • Computer-Supported Cooperative Work and Social Computing
  • Nov 1, 2017

Abstract

Close
Video telehealth is growing to allow more clinicians to see patients from afar. As a result, clinicians, typically trained for in-person visits, must learn to communicate both health information and non-verbal affective signals to patients through a digital medium. We introduce a system called ReflectLive that senses and provides real-time feedback about non-verbal communication behaviors to clinicians so they can improve their communication behaviors. A user evaluation with 10 clinicians showed that the real-time feedback helped clinicians maintain better eye contact with patients and was not overly distracting. Clinicians reported being more aware of their non-verbal communication behaviors and reacted positively to summaries of their conversational metrics, motivating them to want to improve. Using ReflectLive as a probe, we also discuss the benefits and concerns around automatically quantifying the “soft skills” and complexities of clinician-patient communication, the controllability of behaviors, and the design considerations for how to present real-time and summative feedback to clinicians.