Gary Hill's Writing Corpora

Client: Gary Hill
Date: February 2012
Exhibition: Active Presence: Action, Object and Public
Curators: Sergio Edelsztein and Kathleen Forde
Location: Museo MARCO, Vigo, Spain

Writing Corpora is an artwork by Gary Hill that uses physical gestures triggering text, video and audio relating to idiomatic phrases that refer to the human body (i.e. “put your foot in your mouth”), in both English and, in this version, Galician (the native tongue in Vigo). The piece is a continuation of the artist’s conceptual work focused on the convergence of body and language utilizing our current era’s ever evolving new technologies for self-expression.

Our team created a software framework that allowed the artist to play back media elements throughout the gallery space with no physical controller other than the performers movements. The result is a full-featured toolkit that can trigger any audio and/or video media with practically any body pose or gesture.

We created two systems of custom software for Writing Corpora. The first was a fluid-dynamics “touch-floor”, an alphabet soup of letters that congeal into legible phrases and words when certain regions of the floor projection are stepped on, which is also an interactive element that remains for current and future gallery attendees. When a phrase formed in the text fluid it also simultaneously played an audio clip on overhead speakers of the same idiom being spoken in Galician if the text was in English and vice-versa.

The other software, used only during the performance by the artist, tracked skeleton data from a Kinect depth camera for real-time control of audio/visual elements with physical body gestures and movements. This tracked the physical distance between almost every possible combination of skeletal joints (elbow to head, foot to torso, knee to neck … you name it) and played back specific audio and/or video clips on one of three projectors in the room while a forth displayed this tangle of gesture data overlaid on the user’s tracked skeleton.