PigScape Game

A full-body motion-controlled therapeutic game for children with ADHD

Overview

The project aimed to explore and evaluate the potential of using mid-air gestural interactions as an engaging therapeutic gaming medium for children diagnosed with attention-deficit/hyperactivity disorder (ADHD).

The PigScape game was designed to promote self-regulation, behavioral control, and attention skills through active, gesture-based gameplay.

By combining jumps and freezing tasks, the game also encouraged physical activity, focus, and impulse control, supporting the development of cognitive and communication skills such as listening, following instructions, and maintaining attention.

Goal

The goal was to evaluate and optimize usability and therapeutic effectiveness of mid-air gestural interaction, ensuring the interface was intuitive, engaging, and suitable for children with ADHD. The design aimed to support both therapeutic outcomes (improving impulse control and attention) and engaging gameplay.

Key focus areas included:

Designing seamless gesture-based controls (jumps and freezing postures) tailored to children’s physical and cognitive abilities;
Understanding children's engagement, challenges, and preferences in full-body motion gaming;
Refining the user experience based on behavioral insights, gesture performance data, and qualitative feedback.

This project contributes to innovative UX solutions in therapeutic gaming, bridging the gap between human-technology interaction and cognitive training for neurodiverse users.

Role

Interaction designer, Developer, UX researcher

Tools

Development: C++, Microsoft Kinect SDK

Data analysis: SPSS, Python, behaviour coding (manual video analysis & coding frameworks)

Team

Two UX researchers, one developer, two psycologists

Timeline

Feb - May 2018 (4 monthes)

Photos from Julia Kuosmanen

Emphathy & Define

During the ideation phase, our team closely colaborated with neuropsychologists and observed therapy sessions to clarify and define essential game requirements tailored specifically for children with ADHD.

These expert insights allowed us to identify two critical needs for the design:

Balance physical engagement and cognitive challenge. Based on clinical feedback and prior studies, it was important to maintain a balance between active physical play (to increase arousal and engagement) and cognitive tasks such as impulse control and attention focusing.
Design intuitive, robust gestures. Full-body gestures should be selected as natural, easy to perform for most children, and offer consistency that supports reliable gesture recognition.

Ideate

Brainstormed gesture options balancing physical activity, ease of execution, and gesture recognition reliability;
Prioritized playful, engaging game scenarios with clear feedback and motivational elements (e.g., "flying pig" activated by double jump)

Participatory Design:

A participatory design workshop was conducted with four children.

During the workshops, children contributed extensively to:

Customization of the game’s visual assets (avatars and environment graphics);
Development of the game’s narrative elements and storyline;
Adjustment of difficulty levels;
Creation of the gesture vocabulary

These contributions directly informed the design of avatars, environmental graphics, the storyline, and control gestures.

Slide from Julia Kuosmanen

Prototype

A functional prototype was developed, combining a custom-built PigScape game engine with a full-body montion-controled gestural interface powered by Microsoft Kinect SDK.

The game integrates two core gameplay phases:

Active Periods featuring short intervals of physically engaging gameplay where players control an avatar by performing full-body gestures.
Freezing Tasks designed as therapeutic exercises to help children practice restraining excessive movement during moments of high arousal — a key skill in ADHD treatment.

The game supports paired play, allowing children to participate competitively or cooperatively, fostering both friendly competition and collaborative teamwork.

During the active period, the game’s protagonist — a pig avatar — runs an obstacle course moving from left to right across the screen. Players steer the pig by performing jumps and jumps with hands up gestures to avoid obstacles and collect coins and treasures. A skeleton avatar of each player is displayed on the left side of the screen to help children maintain proper positioning within the sensor’s view.

After a set duration, gameplay transitions to the freezing task, where players are presented with a target posture silhouette alongside their Kinect-tracked skeleton. Players must replicate and hold the posture to complete the task successfully. Varying levels of posture difficulty were incorporated, including easy, moderately easy, moderate, and difficult, to enhance the game’s level of challenge and sustain player engagement.

Interaction gestures in game

Simple jump

Users steer the pig avatar by jumps and, thus, avoiding obstacles and collecting coins and other items to earn points and treasures

Jump with hands up

Jumps with hands up grants the avatar a superpower, namely wings, and increases the height of the jump to reach further collectibles

Freezing postures

The children are required to replicate a predeterminated posture by remaining motionless in that posture for a predetermined amout of time

Usabilllity Testing

Participants:

Pilot Study: 11 typically developing children (ages 6–13);
Clinical Study: 18 children diagnosed with ADHD (ages 8–12) across 6 game sessions, allowing us to observe the effects of prolonged mid-air interaction in the game.

Methods:

Active Period Evaluation: Monitored simple jump and jump with hands up gesture performance, success rates, and user preferences.
Freezing Task Assessment: Freezing succes rate and duration of posture holds.
Data Collection: Automated gesture tracking, behavioral observation, questionnaires, and semi-structured interviews covering ease of use, enjoyment, and perceived difficulty.

Quantitative Results:

Gesture recognition accuracy 92% for simple jumps, 95% for jumps with hansa up, and 91% for freezing tasks was highly accurate and robust for the intended gaming purpose and supporting the gameplay.
In acive period, jumps with hands up were preferred, used 53% more often than single jumps.
In freezing tasks, freezing succes rate was 96%, children successfully repeated the predefined postures in 83% tasks and staied motionless in 73%.

Qualitative Insights:

Most participants rated the game as easy or moderately difficult and maintained a positive opinion throughout all sessions.
Positive feedback: Many children described jumping and collecting points as their favorite activities, where the flying aspect related to jumps with hands up was especially joyful. Several children also found the freezing tasks enjoyable.
Negative feedback: Some reported fatigue during long freezing periods.

Observational Findings:

Most participants developed a consistent gesture strategy, often favoring jumps with hands up for engagement and reward maximization.
Children frequently expressed joy, excitement, and playful behavior, including dancing while matching postures during freezing tasks. However, moments of frustration occasionally occurred when physical fatigue set in.
Most quickly understood the game mechanics without additional instruction.

Usability issues that we identified in usability tests:

Issue 1. Accidental jumps detection

Accidental simple jumps appeared when children reacted emotionally to the game events.

Solution: This issue would require conditions that help to prevent accidental system engagement. For instance, jumps with hands up could be one of the solutions. Jumps with hands up, with their distinct phases of initial preparation, take-off, and landing, offer a more intentional, controlled gesture, thus reducing the likelihood of accidental activation.

Issue 2. Difficulty maintaining complete stillness

Our observations suggest that staying completely motionless for a set duration may be can be challenging for some children, especially those with attention and impulse control difficulties.

Solution: Allowing minor movement within predefined boundaries improved task feasibility and reduced frustration.

Outcomes

Children with ADHD successfully engaged with the game’s gesture-based controls, maintaining positive feedback throughout multiple sessions.
The combination of active (jumping/double jumping) and cognitive (freezing) tasks promoted improvements in impulse control, attention regulation, and physical engagement.
Qualitative feedback and observational insights led to iterative improvements in gesture recognition, task difficulty balancing, and user feedback mechanisms.
The game became a permanent part of the neuropsychological group intervention for children with ADHD at the Psychology clinic PSYKE, Tampere University.

Publications

Gizatdinova, Y., Remizova, V., Sand, A., Sharma, S., Rantanen, K., Helminen, T., & Kylliäinen, A. (2022). PigScape: An embodied video game for cognitive peer-training of impulse and behavior control in children with ADHD. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22), Article 68, 1–4. New York, NY, USA: ACM. https://doi.org/10.1145/3517428.3550401

Remizova, V., Sand, A., Špakov, O., Lylykangas, J., Qin, M., Helminen, T., Takio, F., Rantanen, K., Kylliäinen, A., Surakka, V., & Gizatdinova, Y. (2023). Exploring Mid-air Gestural Interfaces for Children with ADHD. In Proceedings of the 16th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG ’23), November 15–17, 2023, Rennes, France. Association for Computing Machinery, New York, NY, USA, 10 pages. https://doi.org/10.1145/3623264.3624449

Page updated

Google Sites

Report abuse