Jabberstamp: embedding sound and voice in traditional drawings

Hayes Raffle, Cati Vaucelle, Ruibing Wang* (Cornell University) and Hiroshi Ishii
MIT Media Laboratory
20 Ames Street, Cambridge, MA 02139 USA
{hayes, cati, ishii} @media.mit.edu, *rw98@cornell.edu

ABSTRACT

Children in our culture are accustomed to creating people and things and places - with implied context - in their drawings. Since the first days they draw, parents will ask "who is that? Where are they? What are they doing?" From early on, children have learned through drawing to provide the information necessary for an audience to understand the story that is going on in their drawing. Conversely, learning how to contextualize an oral or written story in the absence of images is a much slower learning process for children, and children's ability to use language to communicate when and where their story takes place is considered a milestone in literacy development.

Jabberstamp is the first tool that allows children to synthesize their drawings and voices. To use Jabberstamp, children create drawings, collages or paintings on normal paper. They press a special rubber stamp onto the page to record sounds into their drawings. When children touch the marks of the stamp with a small trumpet, they can hear the sounds playback, retelling the stories they created. In a series of studies, children ages 4-8 use Jabberstamp to convey meaning in their drawings. The system allows collaboration among peers at different developmental levels. Jabberstamp compositions reveal children's narrative styles and their planning strategies. In guided activities, children develop stories by situating sound recording in their drawing, which suggests future opportunities for hybrid voice-visual tools to support children's emergent literacy.