One design problem, 6 students, and an eager first-time user experience tester. The result? Not what I expected…
A Design Problem
When we first began using Moodle, we chose the fan-favorite Essential theme. Although this was a great starting place, we soon realized how easy it is for students to get lost in the labyrinth of buttons and clicks. (I think it took something like 8 clicks for students to view their first course.) The design problem became this: how can we make Moodle easier to navigate and reduce the amount of “clicks” a user makes in the site? The idea of a Moodle redesign was born.
Many people choose Moodle because it is free and open source. However, there are hidden costs, one of which may include the site’s usability. Fortunately, we are in a position to work with an external company for the redesign. This company uses an agile approach to design, which means they make prototypes based on our ideas and feedback. We go through several phases like this, until the project is finished. This approach helps both parties develop a project without getting too far ahead. We can make changes as needed, without slowing down the entire build or going in a different direction than anticipated.
Usability Testing FTW
One of the top priorities of an instructional designer is to get feedback from the people intended to use the product. In my case, the majority of people using Moodle are students. So my usability testing group needed to include our students. For simplicity’s sake, I will refer to this group as a “focus group” from now on.
To prepare for testing, I did my research. First, I wrote down stumbling blocks that I knew students and I struggled with in the original Moodle site. I wanted to make sure the old issues weren’t still a problem in the new site. I used these issues to write a short list of tasks for the focus group to complete. I made the tasks simple and straightforward, but there was still opportunity for them to explore the new site and provide general feedback.
I also thought about how I wanted to run the focus group. I am a huge fan of Steve Krug, whose passion and ability to explain usability testing is enviable. I cannot recommend Rocket Surgery Made Easy enough. I used his script to introduce the idea of usability testing to students. I think it made them feel comfortable with the process, and helped them realize I wasn’t testing their abilities. Mainly, I was interested in their experience with the new Moodle site! As it happened, I had six students in the focus group. Five of them had never even heard of Moodle before, whereas one of them had actually taken a couple classes in our old Moodle site. It could not have worked out better!
The Nitty Gritty
To run the focus group, I set up 6 tablets in a booked meeting room and loaded the Moodle site onto each. Thank goodness I brought my laptop, because one of the tablets died. Side note: Always prepare for tech failure!
I didn’t offer them too much direction, other than using parts of Krug’s script and providing context about how and why we use Moodle. I made it clear to students I was available to answer their questions, and that I would walk around the room observing their actions. When I noticed they got stuck, I asked them questions and didn’t immediately offer answers. I found this approach worked quite well; I wrote notes based on their informal feedback. At the end of the meeting, I asked if they had any general feedback. This strategy didn’t work very well. Although I had provided them with pens and paper, none of them had taken notes. After 25 minutes, they seemed ready to pack it in. Plus, it is always difficult to offer feedback in front of peers! So no one really said anything at the end.
Overall, the testing was a good experience, and it made me realize we don’t do this enough when it comes to projects and curriculum planning. And to be honest, preparing for the usability testing and setting up the focus group was not very difficult. This process could easily be replicated for many different deliverables. It’s a good reminder to get feedback from the end users at various points of development and design.
I also really enjoyed getting feedback from staff members. I sat down with 3 co-workers, because some of them will be taking Moodle courses. Plus, it’s always good to get a second opinion. They caught things that I and the focus group had missed, and they asked good questions to help me consider why we were adding or taking away certain elements from the Moodle site. Don’t forget to have a Marketing team member review the build before delivery!
Finally, this experience was a good reminder that while it’s important to get feedback from end users, it’s necessary to weigh those opinions with sound instructional design principles. There were a few times when I felt tongue-tied after receiving feedback. I felt like I had to respond right then. A better strategy is to collect feedback, ask questions when necessary, and then sit with the feedback.
For Next Time
- Screen record testers as they complete the tasks. This allows you to share feedback with developers or re-watch the feedback to make crucial decisions.
- Give members of the focus group a bit of privacy. I think some students felt pressure to “keep up” with others. What was meant to see the barriers of usability turned into a bit of a competition (as depicted in the “high jump” photo from the beginning of the post.)
- Make it easy for the focus group to share findings. This idea came to me right after the testing was over (of course), but next time I will provide each member of the group with 3 Post-It notes, so they can easily share a few ideas. This way, there’s no awkward, non-existent discussion at the end!