Here is Part II of our User Testing PeopleHunt post.
What we needed:
To test functionality of the new hunt feature in PeopleHunt.
What we wanted to know:
- would it be possible for people to find one another in a crowded room?
- how would they react to being hunted and approached?
- would people hunt more than once?
- is it better for everyone to hunt at the same time, or stagger the activity?
What we needed in order to test:
A group of 30+ with smartphones
Mobile version of PeopleHunt with new feature
Food and drinks for group
Location to hold the event
Feedback survey
Prizes for winners
Camera to record interactions
Luck:
Met Aya from
Bing and they offered to sponsor the event and give us advice from their usability team in Seattle.
On the night:
We let people know we were testing
We let people play without too much intervention
We interviewed players
We held a discussion group afterwards
Aya from Bing explaining how they helped
What was good about it:
We managed to gather all the data we needed to identify the good and bad points of the current feature set.
Players getting to know each other better by playing PeopleHunt
We were able to see the hunt in action and get a feel for how people were using the product, which was slightly different from how we expected. We imagined two people searching for each other at the same time, synchronously, which did happen, but we also saw people participating in a one sided hunt, and if players couldn't find their target, they just hunted for another one.
It was awesome to watch people connect deeply, hear positive things about the product, and have the chance to record people's reactions.
One of our fans, Libby!
What was bad about it:
It was not an exact use case. PeopleHunt works best as a session held during a longer conference, not just at a solo networking session, so the players had different expectations. In the end it meant that we had to watch their behaviour more than listen to what they wanted, which is an important thing to consider when creating an artificial use case to test something.
What we learned about the hunt:
We learned that the hunt is an awesome addition to the product, minus a few bugs. It actually took me a week to digest the fact that after all of our experiments, we have indeed created an environment where approaching strangers is acceptable. No need to train as a pick up artist!
Conclusions:
This form of user testing is an important way to test a feature, and to find unexpected hiccups in your product functionality that may be having a negative effect on the user experience. It is great to help you polish a product. However it is limited in how useful it can be to figure out the direction a product should take, or as a replacement for real consumer feedback. Advice, feedback and suggestions for what the product could, should or might be in the future, no matter how well intentioned, should be kept to a minimum!
The winners!
Thanks to all our guinea pigs, and to
Bing for helping us with advice and sponsorship! We really enjoyed and benefitted greatly from watching you use our app. Hopefully we can participate as guinea pigs in other people's events in the future. Do get in touch if you want more details - email me at ellen@crowdscanner.com.
