Designing a hyperinstrument with gesture interface for musical performance

[摘要] ENGLISH ABSTRACT: The field of gesture based research and the interaction between human and computer with the focusfalling on musical applications, is well established internationally. However, in South Africa,research in this field appears dormant. The reasons for this state of affairs are complex and can beargued from different angles covering socio-economical, philosophical and educationalperspectives.This document describes the design, creation and implementation of an operational gesture interfaceenvironment which holds the potential to be expanded in the future. The implementation draws oncost-efficient hard- and software in the design of elementary to more advanced musical and evennon-musical virtual environments (VEs) harbouring potential for further research and performance.Hard- and software available at Stellenbosch University's Konservatorium were used together withselected free downloadable software from the internet in creating VEs, which to a degree simulateother techniques of sound manipulation. The choice of software was guided by the availability ofsupport and prominence in terms of usage. Software basically had to incorporate hand movementtracking and the mapping of data to manipulate several parameters.Three independent systems, each representing a different VE, were studied, experimented with andprogrammed in order to validate the thesis. The first system manipulates a complete electronicmusical instrument. The second system incorporates the simulation of a real-life musicalperformance and the third system focuses on manipulating specific sequencing software by a basicalternative computer mouse implementation.The outcome of this thesis provides an environment within which several programming techniquesare treated and combined to form a template for teaching this field, and future development andresearch. These techniques incorporate the manipulation of digital audio, deal with a digitalcommunication protocol, basic computer graphics and other necessary programming algorithms. Inaddition, the thesis strives to provide an outline for the understanding, design and implementationof a VE installation.The three systems will be installed for operation during a presentation of this thesis. Together withthe three operative systems, this document strives to act as an initial platform from which excitingfuturistic research and activity can be launched.

[发布机构] Stellenbosch University