Measuring emotions: Geneva Emotion Wheel vs. PrEmo

Here at OTOinsights, one of our research goals is to determine a reliable and informative method for measuring the emotional impact media has on users.  Until recently, we had been using the Geneva Emotion Wheel (GEW) in our research to measure participants’ emotions.  The GEW was developed in 2005 by Klaus Scherer who works in the Geneva Emotion Research Group at the University of Geneva.  The GEW is a visual wheel with 20 spokes where each spoke is associated with a type of emotion (10 positive emotions and 10 negative).  For example, involvement/interest and embarrassment/shame are two of the emotion groups.  The spokes of the wheel are made up of five circles which allow the participant to choose the level (essentially, a 1-5 Likert scale) for which they felt that particular emotion.  Here is Scherer’s original paper.  In our research, we had the participants choose two or three emotion groups and mark the appropriate circle to indicate the intensity of the emotions.  The results of the GEW include an overall emotional score on a scale of -10 to +10 (for two emotions marked) and a radar diagram listing the average score for each emotion on the GEW.  While the score was useful for use in our Quantemo Engagement Index (QEI), the radar diagram was typically met with blank stares from our stakeholders.  “What does all of this mean?” was a common question.  The problem was that there are too many emotions on the GEW, so the results become very spread out.  The confusion intensified when there was only a slight difference in the outcomes between stimuli.  Additionally, the GEW allows participants to write in an emotion if it is not listed on the wheel.  This introduces variability into the final interpretation.  Overall, the GEW was useful for our initial purposes, but was really quite hard to explain to stakeholders.

Then I had a serendipitous moment while attending the CHI conference this year.  The session entitled “Beyond Usability: Evaluating Emotional Response as an Integral Part of the User Experience” was presented by researchers at Salesforce.com and Stanford University.  In their research, they used Emocards to measure the emotions that participants’ associated with Web page tasks (their paper can be found here).  Using this tool, they showed that there was a distinct difference in the emotional impact that the different tasks had on the users.  Emocards is a tool developed by Pieter Desmet at the Delft University of Technology (the original paper can be found here).    The Emocards show faces that depict various emotions and the user gives a 0-4 rating for each of the emotions.  The first advantage that Emocards have over the GEW is that it is a cross-cultural tool – facial emotions are typically universally recognized.  Intrigued, I started looking into Emocards and how I can incorporate them into the OTOinsights research.

I was pleased to find that Emocards have transitioned into PrEmo, an online tool that applies the same concepts as Emocards, but with a slightly different treatment.  In PrEmo, there are 12 characters displaying different emotions (see picture above)  – 6 positive and 6 negative.  When the user clicks on the character, it briefly animates and plays a sound, both of which convey the particular emotion to the user.  The user then clicks on a 0-4 scale to indicate the intensity of that emotion they felt when presented with the stimulus.

Now, thus far I have only run one study using PrEmo, but am pretty pleased with the results.  They have a wonderful interface for setting up the study and the results come back to me in an easy to parse Excel file.  This is much better than manually coding the GEW, which was filled out on paper.  Additionally, I found that I did not have to give additional explanations to the users while they were filling out PrEmo – they understood it right away.  This was certainly not the case for the GEW, which warranted many explanations.  Sometimes, the users did not know what a word on GEW meant and asked about it.  There were probably plenty of users who didn’t know what something meant and just skipped over the emotion without asking about it, which would confound the results.

PrEmo also simplifies the interpretation of the results and offers emotions that are more geared to digital media.  Whereas the GEW contains 20 emotions (10 positive and 10 negative), PrEmo contains 12 emotions (6 positive and 6 negative).  Some of the emotions in the GEW weren’t really applicable to digital media.  How many Web sites have you visited that made you feel guilt/remorse or embarrassment/shame?  Probably not too many.  So, I anticipate that the results we will get from PrEmo will be more applicable to our research.

I look forward to using PrEmo more and reporting the results here.  It seems like changing our emotional instrument from the GEW to PrEmo will give more relevant results that can be easily interpreted.  I guess it should be mentioned that there is one advantage that the GEW has over PrEmo: the GEW is free and there is a fee associated with using the online PrEmo tool.  But, so far, I think it’s worth the price.