User Testing

Experience Prototype – Version 0.2

I quickly moved on from the first prototype to start thinking about what would be valuable to test in the second iteration of the prototype. Using the creative thinking design method cards I pulled together the following assumptions to test.

  1. How successful are the structured responses which Facebook messenger can offer?
  2. What do users think of interacting with a business in this manner?

Dreaming up a business

Joe's Wing Palace

To test these assumptions I had to think of a business which could be used for this experiment. With my fondness for chicken wings I quickly created a new Facebook page for ‘Joe’s Wing Palace’ – a restaurant that solely focused on delivering the best chicken wings in Dublin 🙂

I found a really great service called ChatFuel that allows you to build up Facebook messenger chat flows. The really great thing about this service is to ability to integrate much richer media like images, links, maps, etc. So the whole conversation is that bit richer for the user.

Let’s make a reservation and learn…

I setup a test for my user group to find Joe’s Wing Palace in Facebook messenger and to make a reservation. I was keen to see how they interacted with the UI, how they found the experience of interacting with a business and what opportunities or challenges did they see.

The tests were carried out over a few days and I picked up the following big insights.

Structured responses failed

This was the most surprising insight of all the testing. Thinking back on it I don’t know if it was because of the previous test I carried out or if it was people’s natural instinct to just chat to the bot. The majority of users (at least 75%) initially responded to the first interaction by not using the button UI. Luckily the system was setup to capture this and the user did not know any difference to complete the reservation.

Paper Trail

When I spoke to Michael Owen Liston he had mentioned a really key insight he picked up in his project. That for businesses there is an ‘explicit paper trail of intention’. Which for me was most obvious in this task. Through Facebook messenger I could see each and every conversation in real time of when a user was speaking to the bot. You could see what interactions they took, what questions they asked and how they went reached their goal (or not in some cases). There is a huge opportunity here for businesses to work closer with their customers.

Users wanted more

The majority of the users I tested with got very excited with the potential of this UI. In the post experiment interviews they were keen to share what else they would like to see,

This is Awesome. It would be great if you could share the conversation with someone else. I would like to setup an event for the booking and message it to my friends

You can see the test version of this app here:


Leave a Reply

Your email address will not be published. Required fields are marked *