The Pepper Project

Meet Pepper

Pepper, the humanoid robot, is a familiar face on campus at Saint Joseph's University. Typically used to greet prospective students and families for Open Houses and Accepted Student Days, Pepper is arguably one of the most memorable parts of a student's first visit to SJU. To the SJU community, Pepper is known as "Iggy", a student-body selected name inspired by Saint Ignatius of Loyola. However, the impact of the robot goes beyond Hawk Hill.

Through a collaboration with Bancroft in New Jersey, Pepper is also being explored as a tool to support individuals with autism and intellectual and developmental disabilities. This project examines how AI-enabled features in Pepper can help create meaningful, personalized experiences through custom activity sessions tailored to participants’ physical and cognitive abilities, interests, and support needs.

During these sessions, Pepper’s tablet displays visual activities, while built-in speakers can play sounds, music, or spoken prompts that narrate what appears on the screen. Pepper also has cameras that allow the robot to detect faces and turn its head toward participants, helping make interactions feel more responsive and engaging.

Source: Kimberly Paynter / WHYY Philadelphia

Activities

Pepper uses AI to generate activity sessions based on who is participating. These activities fall into four different categories.

Cognition: Pepper reads out a scenario, and individuals must use critical thinking skills to figure out what comes first in a small sequence of events
Recognition: Pepper plays a sound or a song, and it must be correctly matched from four different options
Recreation: Pepper can give individuals the space to talk freely, or sing karaoke, and dance
Basic Skills: Pepper teaches about proper hygiene and health habits

Future Goals

In the final vision for Pepper Project, Pepper will be able to move freely around a space in order to greet and interact as many people as possible. It will be able to detect and recognize people, greet them, and hold a brief conversation. In order to support staff members, Pepper will act as a second pair of eyes, ensuring that no behavioral shift or emotional change goes unnoticed.

Session generation would also be refined, with even more constraints in session generation. When Pepper detects an individual, it will run the image through a database to find a face that matches. Once there is a match, a file will load with an individual's interests, abilities, time schedule, typical emotional state, and any notes that have been logged by direct support staff. These would then be used to select the best activities for that exact moment.

As for activity expansion, Pepper would offer an entirely new set of therapeutic and physical activities, which prioritize physical and emotional well-being. These would include deep breathing exercises to calm someone down, or something similar to chair yoga to keep individuals physically active.

Once the battery begins to run low, Pepper would navigate back to its home base to charge, before continuing to uplift and support individuals.