The HandySinger system is a tool developed at ATR Intelligent Robotics and Communication Laboratories to naturally express a singing voice controlled by the gestures of a hand puppet. A singing voice morphing algorithm smoothes out the strength of expressions delivered with a singing voice.
The researchers recorded a young woman singing a Japanese nursery rhyme, “Furusato” (“Hometownâ€?), in four types of expressions: normal, dark, whisper and wet voice. Dark emphasizes expressiveness like that produced by an opera singer, whisper is a hoarse voice like a lullaby, and the wet expression is used in pop music for temporally emotional emphasis.
The hand puppet consists of a stuffed penguin and a sensor-covered glove that acts as a capturing device. For sensing the motion of the puppet’s gestures, this system has 7 bend sensors and 2 pressure sensors. The sensors’ analog signals are sent to an A/D converter and changed into MIDI signals. A sound control program produces a singing voice.
Users can control singing voice expression with their gestures: voice volume, type of voice expression (dark, whisper, or wet) and strength. For example to sing with “dark” expression, make expansive opera-like gestures for emphasis in a song, “whisper” is mapped to a drooping gesture of the head, and “wet” is obtained through a stretching gesture of the arm as done by pop singers.
PDF of the project.
More puppets: Sock Puppet Theatre, MIDI controller sock puppet.