"Augmented Body and Virtual Body" was performed in, "Utopiales," a festival in Nantes, France in November 9, 2005.
Suguru Goto: Concept and Composition
Yann Bertrand: 3D Image
Ippei Hosaka: BodySuit Performance
Francois Leonarte: Stage Direction
Michèle Trotta: Coordinator
Francois Leonarte and Antonin Artaud: Voice
Texts: "A Thousand Plateaus" by Gilles Deleuze & Felix Guattari, and "To Have Done With The Judgment Of God" by Antonin Artaud
The big idea is to put a human being into a system where many machines act as an extension of the body. I am a composer and I want to control a robot orchestra, video images and videotext using a "BodySuit". All this within the conceptual framework of Music Theatre and a musical composition (Fig.1).
Fig.1: The performance of "Augmented Body and Virtual Body" in Nantes, France, in November 2005. (C) Utopiales
1. About the work
In the project, robots play musical instruments. The robots are controlled by the BodySuit in real time (Fig.2).
The robots ("RoboticMusic") are basically simple mechanical actuators (Fig.3).
Fig.3: There is a special sort of springs in the arm of the robot. At the end of this, it holds a mallet.
They do however, suggest human body parts: arms and hands, which play the instruments (Fig.4).
Fig.4: RoboticMusic. From the left to right, Gong, Bass Drum, Tom-toms, Snare, Pipes.
(C) Raphae¨l Chipault
There will be a sufficient number of these robots to suggest a small orchestra. The robots are both musicians and actors (Fig.5).
Fig.5: Pipe Robot appears behind the Cymbal Robot on this photo. Pipe robot changes the pitches according to the speed of spins.
Central to the project is the composition: a musical composition and a set of algorithms, which will enable me to realize this composition in real time.
2. Music Theater
In this Music Theater the robots and BodySuit are the main protagonists with live players, computer sound, video images, and text in the secondary roles. This is not a multimedia super-sensorium. On the contrary: the whole project concentrates on a single-minded reinforcement of the central theme: Augmented Body and Virtual Body. All structured by a richly-textured musical composition (Fig.6).
Fig.6: This system is applied to a music theater work. There is a 3D image (by Yann Bertrand) behind RoboticMusic.
Like the video image and the sound, the text is also used to reinforce the theme Augmented Body and Virtual Body. The text is treated both as visual material on the video screens and spoken by a real or synthetic voice. Both versions of the text are gradually altered by logical rules - as if playing a game - and are slowly morphed into another meaning. My initial idea was to process the text, “A Thousand Plateaus” by Gilles Deleuze & Felix Guattari, and "To Have Done With The Judgment Of God" by Antonin Artaud (Fig.7).
Fig.7: A performance with "BodySuit" and "RoboticMusic." A photo from its rehearsal.
4. The Theme: Augmented Body and Virtual Body
Having explained the ingredients, I will explain how they will be used to realize the theme Augmented Body and Virtual Body.
The Music Theater weaves a pattern but it is an iridescent and puzzling pattern, interpenetrating and recursive. Augmented Body and Virtual Body convey a sense of duality, a sense of opposition and a sense of similarity. Take, for example, a robot playing a percussion instrument in a duet with the "BodySuit". Similar musical materials will be produced by equivalent gestures. This reveals that a robot plays with an efficient but mechanical gesture, while the "BodySuit" performs naturally and with expression. The point is not whether one is inferior to the other but that they coexist, are equivalent, and different. The materials of Virtual Body and Real Body are both virtual and real, depending on the situation and on definitions. Unlike a human performer playing a Virtual Musical Instrument, the BodySuit is real and controls real physical robots, but these robots are designed to suggest imaginary human bodies. Furthermore the realistic body images or semi-realistic computer graphics seen as video are only one electron thick and so don't have much material existence at all whereas the players, the robots and the performer are real enough and on stage.
The composition emphasizes and exploits the differences between real instruments and robotic instruments and synthetic computer sounds. The computer sound largely contains the texture sound, but a succession of blocks can be dangerous. It may be simplified to provide a clearer texture. Rhythm, pulse, continuation can be clear and emphasized. A clearer horizontal and vertical line can be established, instead of mere heterophony. Heterophony like texture is based upon a single material, and then it is repeated and juxtaposed over and over, like making flaky pastry. The problems appear when the texture changes in another section. It will be heard as a lack of consistency in composition. Less repetition, less juxtaposition, and more materials are one of the solutions. This creates a clearer texture. When several materials are integrated in one section, it becomes easier to transform to a new section and a new texture. The ratio of appearance in events can be controlled by algorithm. If this happens at a micro-level, and it changes too often, the texture becomes too complicated. When these complicated textures change, one may find a difficulty to change one to another. Another solution is for the duration of each material to be sustained longer. Another one is to retain similar textures, instead of changing radically. However, the danger is the music can contain an impression of being rough. The fundamental question can be resolved by a sense of balance, a sense of logical process and by taste.
As the section changes, the combinations will be changed as an ensemble:
a. Robot solo.
b. Computer sound solo.
c. Robot and computer
d. Voice solo
e. Instrument solo
f. BodySuit and robot
g. BodySuit and computer
h. BodySuit, robot, and computer
Fig.6: Upper half of body with BodySuit. The 12 bending sensors are placed on each joint of body.
The robots are controlled by the following algorithms:
a. Orchestra, as one
b. Interactive with movement of body with sensors
c. Mathematical algorithm in a complex manner
d. Each robot is treated as an individual
e. Or, the complement of the whole is treated as one big creature
6. Work Points
The project is a discussion of:
- Existence and physical reality
- The group and the individual
- The physical world and a body
- The body and perception
- Augmented Body and Virtual Body