SUMMARY

Overview

• I led the remote usability studies for a mid-fidelity web-based prototype in enhancing "company X" marketplace experience using "company X"'s usability lab located in Amsterdam in The Netherlands.
• My responsibilities included:
  • Designing a pre-test survey to understand seller's experiences and motivations with the current product.
  • Acting as an observer and moderator in different usability sessions that were conducted remotely.
  • Gathering feedback on user satisfaction points about prototype design and incorporating blank-page technique (participatory design) to help sellers think aloud and co-design the potential new design.
  • Analyzing quantitative and qualitative data from pre-test survey, notes/sketches during the session, post-task survey, and post-test survey.

• To evaluate the usability of a mid-fidelity web-based prototype.
• To elicit feedback from sellers to inform design direction:
  • How do sellers understand the information presented?
  • How do sellers interact with the prototype?
  • What features would sellers like to see in the next iteration?

• Requirement from "company X" was they wanted to learn from sellers who were outside of local area.
• Technical issues encountered during the remote usability studies.
• Without a fully working prototype it made it more difficult to gather user's real feelings and behaviors (especially in a remote setting).

• Usability study report with 17 detailed findings of user pain points and corresponding recommendations for the redesign.
• The impact was to direct the business development team about what worked and what did not in terms of user flows and web interfaces and incorporate research findings into the design by presenting design recommendations to the design team. The design concepts were discussed for company's strategic planning in 2018.

* I can't present the specific findings and design recommendations due to NDA, but I can share the process.

The testing environment was shown in the following figure, including: 

• An observation room where the observer was in charge of recording the video, taking notes, and taking screenshots from the video about the user sketches from participatory design activities (blank-page technique)
• A moderation room where the moderator shared the screen with participants through video conferencing, to see how they would interact with the prototype by the given tasks and then ask follow-up questions to understand why.

Our team was fortunate to use "company X"'s usability lab located in Haarlem, 9 miles west of Amsterdam, to conduct the remote usability testing with both qualitative and quantitative research methods in order to gather a comprehensive understanding of user behaviors.

• The blank-page technique was very helpful in encouraging participants to think aloud because they have to explain the meaning and logistics of their sketches, ideas, or expectations when interacting further with the prototype design, which also allowed us, researchers, to have the opportunities to ask follow-up questions about the why underneath. 
• It was challenging to build rapport with participants when conducting user studies remotely. Even I tried to involve participatory design activities (blank-page technique) with them, some of them were reluctant to sketch anything. One strategy to improve this might be to learn about their backgrounds in advance, such as their experiences selling on "company X", selling categories, etc, which might be a better way to start the conversation with them.
• Our team didn't encounter any difficulties in recruiting since our sponsor did a really good job helping us, it would always be helpful to recruit more users since no-show rates for the remote testing can be higher than for in-person studies.