Emoji Buttons Gauge Emergency Department Sentiments in Real Time

PHILADELPHIA – Simple button terminals – featuring “emoji” reflecting a range of emotions and sentiments -- stationed around emergency departments (EDs) are effective in monitoring doctor and patient sentiments in real time, a Penn Medicine study found. Traditionally, surveys are mailed or sent electronically to evaluate patient experiences, but response rates can be low and those that respond do so well after the visit. Using touch terminals could help inform immediate adjustments in the ED that would not just better serve patients but also the clinicians treating them. The findings of this trial were published today in Annals of Emergency Medicine.

“This study begins to shed light on simple, fast ways of identifying trends and providing high-level information on how patients and providers are feeling in real time,” said Anish Agarwal, MD, an assistant professor of Emergency Medicine at Penn Medicine. “We wanted to see if people would even notice these buttons, which they did — and they pushed them a lot.”

The terminals Agarwal and his fellow researchers used are similar to those used to gauge visitor satisfaction in sports arenas and airports. Each terminal features four buttons, ranging from very positive (green, with a happy face) to very negative (red, with a frowning face). They were established in three different locations in an urban ED to capture the sentiments of three specific groups: doctors, nurses, and patients. As such, one was set up near physician workstations, another at nurses’ workstations, and the last at the patient exit.

During the five-month study period in 2018, nearly 14,000 sentiments were recorded across the three terminals, with roughly 68 percent coming from the provider-facing terminals. The nurses’ station terminal recorded the most sentiments, accounting for 53 percent of those captured for the study but, across all three terminals, an average of 108 sentiments were recorded per day, numbers which were encouraging to the researchers.

“This work suggests that we can collect real-time provider and patient feedback that we haven’t previously been able to identify,” said the study’s senior author, Raina Merchant, MD, the director of the Penn Medicine Center for Digital Health and an associate professor of Emergency Medicine. “This can allow for support when things are going well and addressing challenges when they occur.”

Although primarily a feasibility study, the researchers did uncover some associations between the recorded sentiments and things like identity and patient volume. For instance, high satisfaction numbers were associated with the patient exit terminal, where positive responses outnumbered the negative each day by 25 percent, on average.

On the negative side of comments, doctor and nurse sentiments were moderately associated with a higher number of patients waiting to be seen, and strongly correlated with an increased number of patients being boarded in the ED. This, Agarwal believes, could be tied to capacity strains that limit providers’ ability to see patients.

“I think our staff are happier when things are working smoothly, times where we can quickly treat people and keep things moving,” Agarwal said.

Now with evidence that button terminals are used at a high level in the ED, Agarwal hopes to dive deeper on his findings with future work to determine why sentiment shifts happen and determine the best ways that they can be quickly responded to.

“Frictionless feedback is key in helping drive quality improvement in healthcare, and we need to explore more ways in which we can expand it,” Agarwal said. “I would argue that the day-to-day frustrations — and joy — clinicians experience likely contributes to their long-term satisfaction or burnout. So we need to rethink how we engage with providers and patients.”