Let’s talk about testing!
We are on the point where we have designed a raw prototype with Figma. Our startup idea, Variance Team Building, is finally seeing some daylight. As we have written before about the lean method, the key factor for success is to test the product to improve it constantly. We announced our process last week at the Proakatemia day and now we will have people testing our product for the first time. So exciting!
To do it right we have had mentoring sessions with Mikko Voima and Nea Ohvo, who are experts on that field. Before the testing begins there are important steps to follow to make sure that the results coming out from the testing be valid. We need to make sure that we are acting in the same way for each tester and be careful not to do manners, which could affect the outcome.
Before doing the testing on live, we need to do a form with the questions we are trying to find the answer to. We did a raw sketch and asked for feedback about the way we were asking the questions. We wanted to be sure that the questions aren’t leading the tester to answer in a way we are hoping for. These could be questions like “Do you care about the environment?” Most people would answer yes, just because they know that the majority of people think the environment is an important matter, and having an opinion that doesn’t follow the common rule is something, we are careful to say out loud. These feelings may affect the outcome since people might answer something else than they really think.
Also, one good example when interviewing is always to ask about the past, not about the future. That’s how you are able to find the real way how the person is acting instead of what the person thinks how he/she would act. The question “Do you recycle?” wouldn’t really give much information, instead “How often have you recycled in the past two weeks?” It gives you a real answer to work with. People tend to think themselves and their actions in a polished way, better way. That is just the way human brains are working.
So here, most people would answer that they do recycle, but when asking how often they have done it in the past, they might tell you: “Well, lately not really.”
Why is it so important to learn about the past? Well, while validating your product and trying to figure out how often would people actually use it, it can be deadly if you have asked the question in a way where you have been affecting the outcome. Then you might end up in a situation where you have your product ready to go you find out that there are no customers.
What are the things we want to know at this point?
As written (here) about the MVP (minimum viable product), we want to know if the person understands the value we are delivering. That’s why our first question will be “What do you think the platform is offering?” and the person will have an opportunity to write it in their own words. If here we would give different options to choose on, we would unintentionally force the person to answer to the directions we have set already. So actually, if there would be options like “Helping people to know themselves better / Find new courses to take” we would be blind for the information that if the person didn’t know it clearly and just write something that made sense the most.
Then for the other questions, we will be using the Likert – scale, where the person can choose from 1 to 5 how he/she is feeling about the arguments proposed. The argument could be “I think that I would like to use this system frequently.” And then the person needs to choose: 1) I strongly disagree 2) I don’t agree 3) I don’t have opinion 4) I agree 5) I definitely agree
The quantitative options will give us numbers to work with and then we will be able to have reliable data that can be tested if the information is valid.
To make sure the information would be valid we will be asking the following questions to find out the real user experience:
I found the system unnecessarily complex.
I thought the system was easy to use.
I think that I would need the support of a technical person to be able to use this system.
I found the various functions in this system were well integrated.
I thought there was too much inconsistency in this system.
I would imagine that most people would learn to use this system very quickly.
I found the system very cumbersome to use.
We will ask the “same” things in different ways, because of the risk of someone to misunderstand or to reduce the factors we are not aware of that would mislead the results. With the different answers, we are able to compare the results among each other and maybe to find similarity occurred from the same subject.
After asking about the experience it is important to have an open question box to find out if there is something you haven’t notice and the tester wants to tell about. Questions could be “What was the most exciting thing? Was there something that you didn’t understand at all?” The minus with open-ended questions is, that you will need more recourses when going through the data. We will also ask about the icons. Are they easy to understand? How about the size and coloring and fonts?
During the testing
At the beginning of testing, it is important to say the instructions in the same way. The easiest way remotely is going by the script which needs to be done before the actual testing. We will tell the tester that we are testing the product, not the tester, to make the tester feel more in ease. It’s important to make the tester feel relaxed so the tension or anxiety doesn’t affect the outcome.
We will be asking the person to speak what is he/she doing and thinking while going through the prototype. The person could say “I’m not sure what that icon means, or I don’t see the text properly because it is so small. Now I’m trying to find where to click to go back.” We will take notes and that is why one person needs to be the one who is just observing and writing down the observations. Also, if possible, the best would be if the session would be recorded to be able to go back later to check things if needed. We could also give the tester some tasks, for example, to go to the front page and after that to go to her/his profile. That is how we are able to know precise information about UI.
Psykologian tutkimustyöopas – Petri Paavilainen
Voima M. 2020. Mentor
Ohvo N. 2020. Mentor