DOI : https://doi.org/10.18517/ijaseit.8.6.6428

Evaluating Usability of Serious Games for Therapeutic Purpose: A Study of ASAH-i

Nadia Akma Ahmad Zaki (1), Tengku Siti Meriam Tengku Wook (2), Kartini Ahmad (3)
(1) Universiti Pendidikan Sultan Idris
(2) Universiti Kebangsaan Malaysia
(3) Universiti Kebangsaan Malaysia
Fulltext View | Download
How to cite (IJASEIT) :
Ahmad Zaki, Nadia Akma, et al. “Evaluating Usability of Serious Games for Therapeutic Purpose: A Study of ASAH-I”. International Journal on Advanced Science, Engineering and Information Technology, vol. 8, no. 6, Dec. 2018, pp. 2627-34, doi:10.18517/ijaseit.8.6.6428.
Citation Format :
Usability has been determined to be connected to achieving predetermined practical goals by identifying usability problems during the development cycle of a product or system. In order for the next and future versions of the systems to be enhanced and improved, usability flaws and errors must first be identified. On the same stroke, identifying flaws encountered by the end users will indirectly steer other developers from committing the same mistakes. This paper reveals findings from a usability assessment conducted on the serious game, which was developed for therapeutic purpose known as ASAH-i, which was custom-made to develop and encourage the perception and cognitive skills for children with speech and language delay (CSLD). Two groups of users were formed, namely, test group and control group. Twelve participants were selected which include six CSLD as participants for test group and another six normal children for the control group. In a controlled environment, observation and think-aloud methods were applied to analyse ASAH-i. The data gathered were then evaluated accordingly to the task success, time of the user assignment accomplishment, error rates, and satisfaction in regards to the utilisation of ASAH-i. The result proved that there is no profound disputes between the two corresponding groups, where almost all participants for both groups can play ASAH-i prototype effectively since no major usability issues were found. With regards to competency or efficiency, the result reveals that the control group can play ASAH-i prototype more competently and efficiently than the test group. However, it is interesting to note that both group members also succeeded to play ASAH-i with very little mistakes. This factor yields a strong base that provides towards the group members’ contentment and enjoyment, in which most group members in both teams were very much happy and satisfied with ASAH-i. With this fact in mind, we propose an endorsement or recommendation study which includes details and emphasises on future development to enhance the usability of ASAH-i.

S. Mader, S. Natkin, and G. Levieux, “How to analyze therapeutic games: the player/game/therapy model,” Entertain. Comput. 2012, pp. 193-206, 2012.

M. Wrzesien, M. A. Raya, C. Botella, J. Burkhardt, J. Breton-López, and A. R. Ortega, “A pilot evaluation of a therapeutic game applied to small animal phobia treatment,” in Serious Games Development and Applications, 2014, vol. 8778, pp. 10-20.

N. Yusof, R. M. Rias, E. H. Yusoff, and M. Sciences, “Embedding Cognitive Behavior Therapy in Therapeutic Game Design Model for Depression Faculty of Medicine,” J. Converg. Inf. Technol., vol. 10, no. September, pp. 1-11, 2015.

C. I. Springer, G. Colorado, and J. R. Misurell, “Structured therapeutic games for nonoffending caregivers of children who have experienced sexual abuse,” J. Child Sex. Abus., vol. 24, no. 4, pp. 412-428, 2015.

H. M. Toh, S. E. Ghazali, and P. Subramaniam, “The acceptability and usefulness of Cognitive Stimulation Therapy for older adults with Dementia: A narrative review,” Int. J. Alzheimers. Dis., vol. 2016 (no p, no. 5131570, 2016.

V. H. Carvalho, T. Martins, F. Soares, and M. Araíºjo, “Total Challenge : A Serious Game for Stimulating Cognitive Abilities,” Int. J. Adv. Corp. Learn., vol. 9, no. 1, pp. 4-11, 2016.

A. Shapi’i, N. A. Mat Zin, and A. M. Elaklouk, “A game system for cognitive rehabilitation,” Biomed Res. Int., vol. 2015, 2015.

J. Lumsden, E. A. Edwards, N. S. Lawrence, D. Coyle, and M. R. Munafí², “Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy,” JMIR Serious Games, vol. 4, no. 2, p. e11, 2016.

M. E. Kelly, D. Loughrey, B. A. Lawlor, I. H. Robertson, C. Walsh, and S. Brennan, “The impact of cognitive training and mental stimulation on cognitive and everyday functioning of healthy older adults: A systematic review and meta-analysis,” Ageing Res. Rev., vol. 15, no. 1, pp. 28-43, 2014.

D. Mapelli, E. Di Rosa, R. Nocita, and D. Sava, “Cognitive stimulation in patients with Dementia: Randomized controlled trial,” Dement. Geriatr. Cogn. Dis. Extra, vol. 3, no. 1, pp. 263-271, 2013.

N. A. A. Zaki, T. S. M. T. Wook, and K. Ahmad, “Analysis and classification of serious games for cognitive stimulation,” in International Conference on Electrical Engineering and Informatics (ICEEI), 2015, pp. 612-617.

N. A. Ahmad Zaki, T. S. M. Tengku Wook, K. Ahmad, N. A. A. Zaki, T. S. M. T. Wook, and K. Ahmad, “Towards developing a therapeutic serious game design model for stimulating cognitive abilities: A case for children with speech and language delay,” in Computational Intelligence in Information Systems. CIIS 2016. Advances in Intelligent Systems and Computing, vol. 532, S. Phon-Amnuaisuk, T.-W. Au, and S. Omar, Eds. Cham: Springer International Publishing, 2017, pp. 259-269.

J. Nielsen, “Usability 101: Introduction to Usability,” Nielsen Norman Gr., p. Articles, 2012.

International Organization for Standardization, “International Standard 9241-210, Ergonomics of human-system interaction - Human-centred design for interactive systems,” Int. Organ. Stand., 2010.

N. Bevan, J. Carter, J. Earthy, T. Geis, and S. Harker, “New ISO standards for usability, usability reports and usability measures,” Lect. Notes Comput. Sci., vol. 9731, pp. 268-278, 2016.

N. Bevan, J. Carter, and S. Harker, “ISO 9241-11 revised: What have we learnt about usability since 1998?,” Lect. Notes Comput. Sci., vol. 9169, pp. 143-151, 2015.

J. Rubin and D. Chisnell, Handbook of usability testing, 2nd ed. Indianapolis, 2008.

J. Lazar, J. H. Feng, and H. Hochheiser, “Usability testing,” in Research Methods in Human Computer Interaction, 2017, pp. 263-298.

A. Druin, “The role of children in the design of new technology,” Behav. Inf. Technol., vol. 21, no. 1, pp. 1-25, 2002.

B. Zaman, “Evaluating games with children,” Proc. Interact 2005 Work. Child Comput. Interact. Methodol. Res., pp. 9-11, 2005.

J. Nielsen, “Paper versus computer implementations as mockup scenarios for heuristic evaluation,” in Proceedings of the IFIP TC13 Third International Conference on Human-Computer Interaction, 1990, pp. 315-320.

C. Boletsis and S. McCallum, “Augmented reality cubes for cognitive gaming: Preliminary usability and game experience testing,” Int. J. Serious Games, vol. 3, no. 1, pp. 3-18, 2016.

A.-W. Lyan, A.-G. Amjad, A.-Z. Shaden, and A.-N. Khalid, “A usability evaluation of Arabic mobile applications designed for children with special needs — Autism,” Lect. Notes Softw. Eng., vol. 3, no. 3, pp. 203-209, 2015.

J. Nielsen, Usability engineering. California, 1993.

N. Jacob, “Estimating the number of subjects needed for a thinking aloud test,” Int. J. Hum. Comput. Stud., vol. 41, no. 3, pp. 385-397, 1994.

W. Barendregt and M. M. Bekker, “Guidelines for user testing with children,” Eindhoven, Netherlands, Tech. Rep, pp. 1-4, 2003.

S. N. S. da Cunha, X. L. Travassos Junior, R. Guizzo, and C. de Sousa Pereira-Guizzo, “The digital memory game: An assistive technology resource evaluated by children with cerebral palsy,” Psicol. Reflex. e Crit., vol. 29, no. 1, 2016.

N. Nasiri, S. Shirmohammadi, and A. Rashed, “A serious game for children with speech disorders and hearing problems,” 2017 IEEE 5th Int. Conf. Serious Games Appl. Heal. SeGAH 2017, 2017.

M. A. Khanum and M. C. Trivedi, “Take Care: A study on usability evaluation methods for children,” Int. J. Adv. Res. Comput. Sci., vol. 3, no. 2, 2012.

M. L. Guha, A. Druin, and J. A. Fails, “Designing with and for children with special needs: an inclusionary model,” Interact. Des. Child., pp. 61-64, 2008.

T. Porcino, D. Trevisan, E. Clua, M. Rodrigues, and D. Barbosa, “A participatory approach for game design to support the learning and communication of Autistic children,” Int. Fed. Inf. Process. 2015, vol. 9353, pp. 238-251, 2015.

L. Benton and H. Johnson, “Widening participation in technology design: A review of the involvement of children with special educational needs and disabilities,” Int. J. Child-Computer Interact., vol. 3-4, pp. 23-40, 2015.

R. Yí¡ñez-Gómez, D. Cascado-Caballero, and J. L. Sevillano, “Academic methods for usability evaluation of serious games: a systematic review,” Multimed. Tools Appl., vol. 76, no. 4, pp. 5755-5784, 2017.

N. Mat Diah, M. Ismail, S. Ahmad, and M. K. M. Dahari, “Usability testing for educational computer game using observation method,” Int. Conf. Inf. Retr. Knowl. Manag. Explor. Invis. World, CAMP’10, no. April, pp. 157-161, 2010.

A. Donker and P. Markopoulos, “A comparison of think-aloud, questionnaires and interviews for testing usability with children,” in People and Computers XVI - Memorable Yet Invisible: Proceedings of HCI 2002, F. X., F. J., and D. F., Eds. London: Springer London, 2002, pp. 305-316.

N. A. A. Zaki, T. S. M. T. Wook, and K. Ahmad, “Therapeutic serious game design guidelines for stimulating cognitive abilities of children with speech and language delay,” J. ICT, vol. 2, no. 2, pp. 284-312, 2017.

J. Sauro and J. R. Lewis, “Estimating completion rates from small samples using binomial confidence intervals: Comparisons and recommendations,” Proc. Hum. Factors Ergon. Soc. Annu. Meet., pp. 2100-2104, 2005.

T. Tullis and B. Albert, Measuring the user experience: Collecting, analyzing, and presenting usability metrics, 2nd ed. 2013.

T. S. M. T. Wook and S. S. Salim, “Visual interface for searching and browsing children’s WebOPAC,” Univers. Access Inf. Soc., vol. 13, no. 4, pp. 367-385, 2014.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).