After drafting our four tasks and creating our prototype, we were ready to commence our user testing. We each tested two people which we all moderated in-person. The target audience was people in their 20’s, so we focused on that age demographic for our testing to generate the most informative feedback. The test involved a structured interview based on four tasks.
We wanted to gain both qualitative and quantitative information from the user testing. For the qualitative data, we were looking to a more emotional response of what they thought of the new features we added to the redesign. With the quantitive data, we wanted to measure the task completion rate and the efficiency of how they navigated through them.
From gathering the data, we had a 100% completion rate of our tasks. Which was good; however, there were some issues some people found certain aspects confusing and the dwell time on particular tasks were longer than we would have liked. More notably though we found the most significant error was in our task script. It was a bit rough and ready and not considered enough, which resulted in having to go off-script at times. So we did not gather the most insightful data from here. But we did discover some pain points where people struggled, which we were able to redesign. We also gained some qualitative data where people found the ‘boards’ idea “very handy” and liked the “stripped-back” interface.
So from the testing, we were able to refine the prototype two a few areas where people were struggling. The biggest issue lay in the third task, where people had to add a product to the board they just created. There was an overload of information here, and people did not find the process straightforward. So we pared it back, making it clear what to do. Another change we made was to introduce the Alexa brand into the design. Brining in the logo for every time Alexa introduced herself, gave consistency to the interactions. Using the italic serif font, which was part of the brand guidelines for Alexa also gave it a more personal touch. The third change was to highlight Alexas recommendations helping it stand out more and understand that it was from Alexa.
We then conducted another round of user testing with the new updates. For this round of testing, we made several changes to the script as the previous one was a bit vague. We included directions on what products to click as part of the user testing this time; we neglected this the last time, and from speaking to Holly this was still a feature that needed to be measured. These updates helped improve the experience, coupled with the updated task script; we collected more conclusive data.
Amazon Redesign 