My first week in New York city has been so busy that I have not had much time to do a retrospective of my eventful week, from August 15th to 22nd at EMPAC in Troy, NY.

I am happy to report that I was successful in creating a “visual music” presentation on 3 screens with one of the aspects controlled in real time using gesture control.

Adafruit shipped my Arduino boards to EMPAC and I was able to research on the internet and receive help from others in the workshop to understand how to make the system work. I chose the infra-red distance sensor and used information from the Adafruit website to help me program the board. My trusty Ubuntu computer handled all of the Arduino code uploading while I ran the Isadora program on the Macbook.

My performance was thrilling for me, but I was inspired and informed by the work that everyone else in the class showed on Saturday, August 21, 2010.

The first work we saw was by Joff, a theatre instructor who has a company that puts on plays in Second Life. The work included a grassy field with images of human figures that were short videos. The figures would fade in, seem to be sleeping, then move in some way, then fade out. We were invited to interact with the figures, but I found it fascinating just to watch them.

It was eerie and engaging as an installation and Joff might develop it further as a performance or to be more interactive.

There were two other installations, one by Ian (pronounced Ion) that consisted of a table and two chairs with a set of seemingly random objects on the table. The screen that faced the participants in the chair included a delayed video of individuals who had visited in the past mixed and overlaid with video of the present visitors. The more people that participated the more interesting the exhibit became as the past visitors would show up on the screen seeming to sit opposite the live participants.

Ian also staged an interactive performance and he was very generous in assisting with coding other people’s projects.

I hope to have more time to process the photos I took and I hope that Johannes (pictured here enjoying interacting with Ian’s installation) will be able to produce some video excerpts from the documentation of the Saturday events.

It was a full schedule and everyone worked very hard to create interesting and innovative work.

One of the largest collaborations was Sarah’s (shown activating one of the messages left) project. This was a complex installation that interacted with the visitor. As the person moved on the lighted squares the text and the audio would respond by changing. This process required complex programming, audio and text preparation and a lot of testing. The final showing was a great demonstration of the power of collaboration and the strong idea that Sarah realized.

Victor contributed to many projects as a programmer, electronic music composer and performer and finally, he revealed his own powerful statement.

Victor is a karate master who tracked his movement with a light controlled with infra-red tracking. It was a surprise to everyone who had watched all of the other projects evolve during the week with Victor seen as a helpful computer coder and audio designer. Victor amazed everyone with his grace and intensity as he put on a very effective performance.

More information about the activities of the busy week can be found on the workshop blog titled, EMPAC Live. Media+Performance Lab.

I do have more material, but no time at present. When I return home and I am shut in during the rainy season, memories of this wonderful week will pull me back to share more with you.

Share on Facebook

UniverCity Net - digital art by Victoria Gibson

Research into the Immersive Environment Instrument (IEI) continues and this post is about the visual surround aspect of the presentation. For background information on this project click here. There are lots of links to examples, but none I have the right to re-post. My digital artwork, UniversityNet, has become a kind of logo for the IEI, so I post it to break up the text heaviness of research.

My goal is to bring the cinematic experience of film into live performance within a virtual reality simulation. Exploring a non-linear narrative that references three time periods — past, present and future, to construct a communication that is open to individual interpretation. Resonating with an audience on an emotional level, the performer improvises each show as a unique construct. Using a sensor array, events are triggered with dance-like motions that communicate emotional intention through gesture.

The University of California,  Santa Barbara is the host institution for the AlloSphere, an incredible facility designed by Dr. JoAnn Kuchera-Morin.  Dr. JoAnn Kuchera Morin assembled a team that included world renowned architect, Robert Venturi, to design and construct this feat of engineering. She talks about her work on TED with a really interesting video.

In 2000 she began the creation, design, and development of a Digital Media Center within the California Nanosystems Institute. The culmination of her design is the Allosphere Research Laboratory, a three-story metal sphere inside an echo-free cube, designed for immersive, interactive scientific and artistic investigation of multi-dimensional data sets. She serves as Director of the Allosphere Research Laboratory and Center.  . — XMedia Lab

There is a lot of interesting research in Santa Barbara that may apply to the construction of the Immersive Environment Instrument, but it is not portable and it was designed to facilitate research, not performance. There is no room for an audience, but compositions could be shown on video as seen on the TED site. The challenge would be to translate as much of the effect of this multi-million dollar facility into a portable touring show.

Many of the elements of the IEI have been used in performance before, as the idea of surround sound has been in use since the introduction generally credited to Stockhausen. The idea of surround video has been used by a collective, Workspace Unlimited, among others. Their example, Hybrid Space 360, is really interesting because it has already been presented at EMPAC – Experimental Media and performing arts Center Rensselaer in New York, 2008. There are photos on their site, but as it in Flash, you have to navigate yourself, click on projects to view the data . . .  I cannot even link directly.

Their interesting concept is of an interactive installation, not a performer controlled environment. The Immersive Environment Instrument will tour with Victoria Gibson in Girl Can Dream, the first composition for the instrument. Later, other musicians will be invited to join Victoria to expand the sensor array and integrate other personality styles.

A lot of recent development in software has made still photos panoramic and there are even very inexpensive point and shoot cameras that offer the ability to select the nodes (nodes are individual photos — see Wikipedia) required to stitch together photos in this way. With the proper viewer, an on-line panorama can be achieved. See the Apple VR site for examples of Quicktime panoramas (plug-in required) .

Ubuntu Users — The most promising thread in this area seems to be freepv not found in a repository by Synaptic yet. Build instructions on the site.

Anyone with the quicktime plug-in can view movies made of panoramic photos on this company’s site — Studio 360. These examples are not the photo itself, but a movie of the panoramic photo.

The concept of surround video is tricky, so I propose to set up separate nodes, the same way panoramic photos are really separate photos. This will also give me more flexibility in using lights to soften the edges of each node to aid in blending or to use the nodes a unique, distinct images. I do not expect surround video to be available soon as large studios are having enough of a challenge with Imax, Omnimax and conventional 3D films.  I think that it is more cost effective and realistic to project each node independently and let the instrument create a Cinematic Virtual Reality (CVR) illusion. Cinematic Virtual Reality is a term I created for the visual component to the Immersive Environment Instrument.

Share on Facebook

UniverCity Net – digital art by Victoria Gibson

As an Integrated Media Artist, I have many project ideas that I am constantly working on. My most recent project proposals have been seeking support to develop a new instrument for me to play in performance. My current goal is to develop a performance instrument that works using gesture control and sensors to create an alternative reality environment.

Although my core training is in music, I have studied dance and motion and have spent years of my life in on-stage performance as a musician. In my recent presentations, I have played computer based instruments and I realize that a lot of the impact of musical presentation is about the gesture of producing the sound.

If I am to continue to use the computer as a performance instrument, I need to develop a controller that responds to dance-like movements. The vision of being able to focus physical energy through dance and control audio and visual elements was crystallized into my need for an Immersive Environment Instrument (IEI).

My research into this topic has been detailed and I will construct a page to organize information about the IEI. [UPDATE: Find it Here] This post is to show the work that is already being done by Arizona State University, School of Art, Media and Engineering that uses motion tracking technology and dance with light and projections.

Here is a link to their Gallery page, I have no permission to repost, and I am very sensitive about these issues. So you have to go and look from their site — it is worth a visit to see the dancers and the light responding to their movements. Video and photographic documentation is available from the link.

What I want to do is different because it is based in Integrated Media performance where dance is a significant portion, but it is used to focus energy that results in other events. My concept is to immerse the audience in an environment that surrounds them with sound+light+motion to create a beautiful virtual reality experience that brings dream to life.

Screens will surround the audience to the limit of their peripheral vision and surround sound will envelope them; because we cannot see behind us but we hear in 360 degrees, there is no need to completely surround  with the visual elements. The artists will become part of the projection as the light will be projected on to them and their shadows create a similar effect to the shadow play of Indonesia as they move.

The most practical scenario for the screens that I can envision is for them to show images in panels, with each panel having an individual projector, so that the image/video can can be changed in location or a fresh image brought forward in real time.

The light will fill the areas between the projections with colour that I also want to control in real time. Parameters like diffusion, colour, intensity and brightness can all be computer controlled with vari-lites.

The sound will surround the viewer from all directions and will be generated using traditional sound sources such as voice, percussion and guitar augmented, effected and accompanied by computer generated sound and pre-recorded samples.

The musicians will use dance-like gestures to control all the parameters simultaneously. Some parameters may be cross-linked to allow simultaneous activation of related image/video/light/sound events.

Once I have the IEI constructed I will compose works that will use pre-organized materials and ideas presented in an improvised flow that responds to the energy of the moment.

To truly realize this concept, I plan to link all performances to a virtual reality computer world, such as Second Life, so that the performance may be shared in mixed reality. The virtual reality may also be projected for the live audience to share the unique effects that can only be experienced virtually. This will be accomplished using the amazingly low latency software developed by Stanford University under the direction of Chris Chafe. This software is based on the jackd audio connection software used in Linux OS, such as the Ubuntu kernel that I use. The software that is used for Telematic concerts, such as the one I wrote about in the post about Chris Chafe, is called jacktrip. This software can be used to link musicians in remote locations with very low latency — depending on the speed of the connection. Before I arrived in Banff, John Adams had recorded a multitrack session using 24/96 sampling rate. This astounded me and the results I witnessed myself, at a lower sampling rate, were exceptionally good.

Due to the interesting work done by dance technologists in the world of motion tracking, I have now joined dance-tech.net and you can visit me there. I am freeing my inner dancer after all these years of trying to limit myself. In the music world all the business people and sucessful artists on panels tell you to limit yourself and categorize your work in a recognizable way so you can market it sucessfully. I did try, but at this point in my life, it has become important to me to do the work as I perceive it. Commercial success has become a lot less important than the satisfaction I get from achieving my goals.

Go to Immersive Environment Instrument Page

Share on Facebook