top of page

Evaluation

The prototypes could not be tested and evaluated properly due to the limited time available for this project. Despite this, there are certainly recommendations on aspects of our product that could be improved or added. These include the development of supplementary items to support the service and the improvement of the already existing concept.
Evaluation Plan

Evaluation Plan

The goal of this test is to find out whether a user from the target group is able to understand how the prototype works and is able to complete certain tasks. As the test can be done by only one person, the test is not representative of how the prototype would function for the rest of the target audience. For this, testing with a larger sample size would be needed. 

 

It is relevant to test the task completion because it assists in evaluating the product. If for example, it comes out that the user has difficulty completing certain tasks, certain parts of the product/app should be redesigned. 

App Test for the User

The user needs to carry out and understand the following points when using the app: 

​

  • The user understands the menus on the bottom of the screen.

  • The user understands that you first have to fill in the mood before clicking yes or no.

  • The user is able to find past moods on the calendar screen.

  • The user is able to understand the charts.

  • The user is able to change the language on the settings screen.

  • The user is able to change the theme colour on the settings screen.

  • The user is able to allow or deny push notification on the settings screen.

  • The user is able to change the mood entry of the Moodbabble. and to turn them on or off on the settings screen.

 

​

To see if the user understands all the previously mentioned points, we need to carry out a usability test. During this test, we will ask the user to perform certain tasks. The time and the mistakes that are made will be tracked as the tasks are performed. Some important actions can be written down in the notes section.

​

After the test has been completed, the person leading the test will go through the list of tasks and will ask about the test participant’s experience, paying extra attention to any tasks a participant may have failed to complete or struggled with. This will be an unstructured interview, with only the tasks from the exercise as guidance. 

App Test Plan

Click to enlarge

Interface Test (Moodbabble.)

The user needs to carry out and understand the following points when using the physical product:

​​

  • The user understands how to fill in the mood.

  • The user understands how to fill in if the user wants to talk.

  • Understand that you use the OK button after filling in the mood and if you want to talk, confirm the settings.

  • The user understands that all the previous filled in moods can be found in the app.

  • The user understands that the settings (language and the times of the mood entry) on the physical product can be changed in the app.

  • The user understands what to do with the feedback given by the product.

​

After the test has been completed, the person leading the test will go through the list of tasks and will ask about the test participant’s experience, paying extra attention to any tasks a participant may have failed to complete or struggled with. This will be an unstructured interview, with only the tasks from the exercise as guidance.

Prototype.png

Click to enlarge

Test Materials

  • Physical prototype (to show beforehand, will not actually be used during testing)

  • Digital prototype 

    • Representing the physical product (preferably on a tablet, otherwise on laptop)

    • Representing the app (preferably on phone, otherwise on tablet or laptop)

  • Materials required for notetaking

    • Laptop

    • Stopwatch

Test Conditions

The test conditions should stay the same throughout the test. If a test would be done with more participants, that would be extra important. The test conditions should somewhat resemble a home environment. It is unnecessary to recreate an entire apartment, but it should be calm (not too noisy, no people walking by), without too many distractions. 

There should be two ‘examiners’ present, one to communicate with the participant, the other to take notes. They should not speak while the participant performs the tasks. 

Shape

Evaluating the Aesthetics

While no usability testing is possible for the product’s aesthetics, it is still important to us that the looks of the product match the user’s wishes. Mainly for the physical product: this will be placed somewhere in the house and will become part of the user’s home. 

In order to evaluate the design (in a more structured manner than merely through conversation), a short questionnaire was created. This questionnaire, consisting of Likert scale questions, provides more insight into the opinion of the end-user. Likert scales are commonly used in questionnaires: they are 5 step scales, with options ranging from ‘strongly disagree’ to ‘strongly agree’ (see image).  Therefore, the questionnaire has to be filled in by potential users. 

 

Statements:

  • The product looks friendly

  • The product looks as if it is easy to use

  • The product would fit with my style

  • The product seems appropriate for my age

  • The different buttons look understandable

  • I see this product fitting with my friend’s style

  • The product would look good in my home

likert.png
Reflection

Reflection and Recommendations

Account for Caregiver and Client

At the moment, the concept and prototype for the mobile App are focussing on the version the primary stakeholder gets to see. In other words the main user of the physical product and the service. The mobile App concept has not been worked out for other stakeholders, like the caregivers that play an important role in the interaction of our product. The mobile application could be given separate account options for example, one for the user (the client) and one for the caregiver. This will enhance the privacy of the system and make sure all stakeholders have one centralized mobile App.

Product at the Front Door

In the current state of our product, only the caregivers can see what mood Wendy is in. Occasionally, the neighbours come to her with problems and make her feel overwhelmed. To resolve this problem we thought out a solution to inform the neighbours and caregivers of Wendy her mood. This will indicate when people from outside her room can visit Wendy or not at a certain time. 

 

This solution is related to the traffic light previously seen in the ideation phase. The downside to this idea is that you are dealing with the privacy of a person and that everyone can see how she feels. We proposed this idea to our co-designer at the end of the concept phase as an additional/supplementary product. She informed us that she would like a solution like this, but that it would only be necessary to show her mood when she is in the orange or red zone. This would indicate that she does not want to be disturbed at that moment. This new additional product could be added to our already existing product and app, but this is something for future development.

Set up of the Voice/Words of the Physical Product

The final product has an implemented speaker that realizes the vocal interaction of the product. As the user indicates her feelings and if he or she wants to talk, feedback in the form of audible speech can be heard from the speaker.

 

For the prototype, this form of audible feedback included fixed phrases as advice. It might be desirable to give the stakeholders, the caregivers in particular, the option to record and submit custom audio feedback that works best for the client. This will greatly enhance the effectiveness of the personal support the product has to offer.

Co-designer

Meeting with the co-designer was a bit of a challenge. During the week, there was only one day that the co-designer could have a meeting together with her mentor. So we would have preferred to get in contact with Wendy more often. 

 

During the evaluation with the co-designer, she mentioned that she liked the style of the product. The interface was very clear for her as well. She also felt like the product suited her needs and that we listened to her.

bottom of page