The Visual Recital is an exciting new approach to integrating live art music with dynamic imagery, animations, and synchronized video clips, where the performing musician directly controls the digitized visualizations as an enhancement of the musical performance. With the visuals serving as a powerful narrative lattice, art music becomes immediately accessible to audiences young and old who may have never experienced a live recital setting.
Visual Pianist Hugh Sung has developed a system that integrates cutting-edge visualizations with a simple foot-switch trigger that enables the Art Musician to perfectly synchronize visual effects that illustrate the narrative sweep and structure of each musical composition
NEW! Watch a sample from a performance at the Darlington Arts Center featuring selections from Charles B. Griffin's "Vernacular Dances", with visuals projected onto a 9' x 7' pvc frame rear projection screen:
Cellist Mary Artmann performs "Bunraku" by Toshiro Mayuzumi at the Rocky Ridge Music Center in Estes Park, Colorado, with dynamic visualizations by Charmain Schuh and Hugh Sung. Hugh Sung (www.HughSung.com) has developed an audio-reactive engine using Pure Data to trigger visual effects within the presentation program Liquid Media. Please visit www.VisualRecital.com for more details on the techniques used to create this Visual Recital performance.
No Trackbacks |
Chopin Project: To-Do List
Here is a list of artwork that we will need to create according to the storyboard outline proposed by the Neptune High School art students:
Scene 1: Introduction/Once Upon a Time
3 sunrise scenes at a lake with trees
- one with landscape in darkness
- one with sun just emerging
- last one with sun rising and light filling scene
Scene 2: The Main Theme
Picture of old man walking by the lake
Scene 3: An exciting secret revealed
Wind blows, revealing scarf on old man representing memories of lost love
Scene 4: Impending Danger
Pictures of approaching storm scene
- clouds darkening
- trees swaying
- leaves flying
- waves forming at lake
- lightning in skies
Scene 5: A distant memory
Pictures of old man in youth and lake from long ago
- smaller trees
- perhaps more people walking around?
Scene 6: Love Theme
Pictures representing love and life, such as
- cherry blossoms
- 2 swans swimming together in lake
- young woman wearing old man's scarf?
Scene 7: Main Theme returns
Autumn scene at lake, with leaves on trees turning colors?
- Picture of the old man in middle age with scarf - looks rich and powerful?
Scene 8: A Grand Dance
Pictures of a town center
- perhaps with a fountain?
- buildings surrounding town square similar in composition to how trees surround lake?
Scene 9: A flurry of excitement
Pictures of town filled with busy people, activities
Scene 10: Distant Memory joins the Love Theme
Party scene?
- Man meets woman wearing his scarf
- pictures of man and woman dancing together
- closing scene with man left alone with scarf in his hands
Scene 11: The Main Theme returns, in greater danger than before
Winter scene at lake with old bare trees, empty skies, and old man in advanced age
Scene 12: A ferocious battle
Winter storm scenes by lake
- snowstorm?
- old man in coat and scarf struggling against wind?
Scene 13: A tragic ending
Pictures of scarf floating on lake
- night scene at lake with moon?
Once again, here is the flash player for the musical references:
No Trackbacks |
The Chopin Project
The Chopin Project is a Visual Recital workshop collaboration with the students at Neptune High School to explore the Ballade No. 1 in G minor, Op. 23 by Frédéric Chopin. We'll be using this website as a project blog to share notes on the development of the storyboard and visualizations created for this piece. Many thanks to Superintendent Dr. David Mooij, Principal Richard Allen, Director of Curriculum, Assessment and Instruction Edward Dever, supervisors Nancy Moore-Fuss (Arts and Music) and Claudia Mooij (Special Education), Department Chair Brian Dougherty, Director of Instrumental Music Raymond A. Kelly, and Art Teacher Michelle Bowers for helping to bring teachers and students together!
We started today's workshop with a brief introduction to Polish-born composer Frédéric Chopin (1810 - 1849):
- he lived in Paris and enjoyed a career as a piano teacher and composer
- he composed the Ballade No. 1 in G minor when he was 25 years old
- his music is considered to be part of the Romantic period in Classical Music
We talked about how all music tells a story, and how we would be using Chopin's music as the basic script for the visualizations that we would be creating.
I then played through the work in its entirety, simultaneously projecting descriptive text of the following sections:
1. Introduction - Once upon a time...
2. The Main Theme
3. An exciting secret revealed
4. Impending Danger
5. A distant memory
6. Love Theme
7. The Main Theme returns
8. A Grand Dance
9. A flurry of excitement
10. The distant memory joins the Love Theme
11. The Main Theme returns, in greater danger than before
12. A ferocious battle
13. A tragic ending
You can listen to each of the above sections with the flash MP3 player below:
We then brainstormed on possible story and character ideas for each section. We also selected some landscapes that had been created by an earlier art class, matching them with some of the opening sections of the music as follows:
After some more brainstorming, the class came up with some amazing storyboard concepts as follows:
create the setting of a man by a lake - the man represents the main theme, and the lake could represent the old man's life
each time the main theme returns, the man gets older (starts with him as a young child)
the lake in turn also ages as the piece progresses - trees get bigger, seasons pass - various seasons can represent shifting moods for different sections (springtime, blooming flowers, storms blowing leaves off of trees, bare branches, water levels changing from full to muddy, etc.)
Love themes can be represented by swans, cherry blossoms, sunsets & sunrises (and other scenery changes)
objects representing love, loss, such as a stopwatch? a top hat?
Shifting realities - in head? memory?
Tragic ending - represented by the object of love left alone on the scene - hat on the beach? floating in the water?
photographs of nearby lake to use either as scenery or as artwork templates - pictures of sunrises/sunsets would be amazing!
Participating students will be invited to comment and contribute artwork, photos, sketches and story ideas through this blog. Additional blog articles highlighting specific sections will be created as needed.
In this video, I presented a Visual Recital workshop at the Greenport School in New York, featuring movements from Modest Mussorgsky's "Pictures at an Exhibition" and artwork by 3rd, 4th, and 6th grade students under the supervision of art teacher Anne McDonald. The workshop and concert were made possible by the Greenport Music Festival.
No Trackbacks |
Visual Recital Workshop at the Hamilton School
i can't think of many teachers better than good ol' experience, and certainly this past week's Visual Recital workshop at the Hamilton School proved to be a gold mine of learning on all fronts!
The session began with an initial visit with a group of the school's 4th - 6th graders. Cellist Susan Babini and pianist Michael Mizrahi had been introducing the group to the first movement of the Sonata in F major, Op. 99 by Johannes Brahms. Realizing that we were going to have limited class time with the kids, i chose to "pre-empt" the design of the visualizations by selecting a general theme and creating the visual backgrounds ahead of time. I wanted to present ideas that would be as contextually familiar to the West Philadelphia students as possible, so Bonnie Slobodien and i brainstormed the idea of "Two Views of the Schuylkill River" (wow, you know you're from Philadelphia if you can not only pronounce that word correctly - "Skoo-Kill" - but also spell it from memory!!) The basic idea was to show contrasting elements in the music, such as:
fast
slow
peaceful
majestic
The Schuylkill River is the main body of water that runs through Philadelphia, featuring a major highway on one end and a lovely park on the other. Initially i was going to stick with just a riverside highway scene and a blank riverside park trail background, with the idea that the students would be encouraged to draw things that move fast on the highway (cars, trucks, motorcycles and whatnot), and contrasting things that you would find in a peaceful riverside park (trees, flowers, clouds, ducks, boats, etc.) Analyzing the Brahms made me realize that almost 8 minutes of music comprised the first movement, and that there was plenty of opportunity to feature other contrasting sections. I eventually came up with the following added backgrounds:
the Fast scene, featuring the Schuylkill Expressway
the Slow scene, featuring Kelly Drive (the scenic route next to the river)
a busy street scene, representing a part of town that lies next to the river and under some big highway overpasses - this was going to be the "busy" theme
Boathouse row, a picturesque series of crew houses used by local universities and crew clubs right along the river - this would be the "peaceful" theme
The Art Museum - this would represent the "majestic: grand" theme
The Ben Franklin Bridge at night - this would represent the "majestic: exciting" theme, with the students primarily drawing fireworks for this scene
We had two back to back classes to work with, for a total of about 60 students. Given the time restraints, we kept the art medium simple: crayon pencils on white paper, to be cut out and pasted with glue sticks onto black construction paper backgrounds.
What an amazing output of creativity! Giving the students empty scenes to work with, they all vied to produce several items for EVERY one of them!
Note to self: i thought that pasting the cut out pictures on black paper would make it easier to isolate the images for transparent backgrounds, but it turns out to be actually more tedious. Time can be saved by eliminating the cutout/gluestick actions, and the scanned images can be "lassoed" manually, copied and pasted onto transparent backgrounds, and saved as PNG image file formats.
Unlike the Mad Cow visual recital workshop in Colorado, i at least had about a week to scan in the images and place them into the scenery. Well, a jam-packed week so it turned out, what with the Greenfield Competition finals, all the rehearsals in preparation for that, and Karate graduations for the kids and me...it was a challenge to find the time to get this all done (which led to late, late night programming sessions...which then led to - kaff kaff - this yucky cold i came down with...)
Primary programs used for creating the visual backgrounds:
ArtRage 2, a fantastic program that can realistically simulate paint, markers, crayons, pencil, and a host of other physical media - works exceptionally well with Tablet PC's
Inkscape, the open source vector drawing program. I really fell in love with the simplicity of use with this one - i had been a longtime CorelDraw user back in the old days, so this was like working in old familiar territory. The "technical drawings", such as the Ben Franklin bridge, the highway and the yellow dividers, and anything else that required symmetry or precision was best crafted with Inkscape.
Primary programs used for cropping, cleaning, and in some cases making animations with the students' pictures:
Macromedia Fireworks - i'm sure i could use GIMP to extract the scanned images and paste them onto transparent backgrounds, but i just work faster in Fireworks...
Macromedia Flash - several kids came up with pictures that were almost identical (sunshine, flags, etc.), so i took advantage of some simple Flash layering and alpha fades to make animated blends between the pictures and exporting them as animated GIF's
Initially, i was only going to have static backgrounds with all of the image "actors" moving in automated loops within each scene. The only trigger points would be to advance to the next scene. Fortunately, i was able to get some great help from the developer of Liquid Media to create an "unlimited ammo" trigger system. Using the X-Keys USB 12-port switch interface, i was able to incorporate 3 pedals - one to advance the scene, and two others to be used by the musicians to trigger events within specific scenes. For example, in the Fast highway scene, there was an active background of the highway zipping along and cars traveling over it. If the cellist stepped on her pedal, a series of special cars would drive by at a faster speed. If the pianist stepped on his pedal, helicopters would fly by the sky overhead.
Here are some pictures from Monday's VR workshop:
Mr. Guy Cannon's music classroom setup at the Hamilton School:
Susie giving a cello lesson to a curious student:
Michael surrounded by eager pianists:
Our indefatigable director of Education and Outreach from Astral, Bonnie Slobodien, encouraging the students to "respect our friends by listening quietly" - well, that lasted for a few seconds at least...
The main man himself, Mr. Guy Cannon - a cooler music teacher i have yet to meet!
Michael and Susie prep the students for the world premiere of "Brahms on the Schuylkill River":
Michael demonstrates how the cars can be triggered to zip by on the highway scene:
Student balloons float over the Art museum and Susie's head:
Susie and Michael performing during the Boathouse Row scene:
One concern that Bonnie and the musicians had was having the visuals overwhelm the students' reception of the music. I think we came up with some great activities to balance the excitement of having one's own artwork animated to live music and the need to encourage stronger listening skills:
I came up with the "Ta-DA" game on the spot, where the students were challenged to recognize and count the number of times the "Ta-DA" theme was played - ie, the 16th note to tied quarter notes motif that runs throughout the theme (highlighted in yellow):
When the theme fragment returns at the beginning of the development section, the "Ta-DA's" turn into "Oh-NO's!", highlighting the change to minor and its resulting shift in mood:
The Ta-DA's come back at the end as "Hoo-Ray" (or something to that effect), reflecting the heroic final statement of the motif in the last few measures - we thought of the association with something proud, noble, and majestic - like fireworks!
Interesting to point out how a rhythmic fragment can change somewhat, turning from 16ths -> half notes into 8ths and quarters.
Another listening game involved everyone closing their eyes and the musicians playing a random section of the movement. The students would have to guess which scene the music was associated with, then open their eyes to see if their guesses matched what was on the screen. It was quite remarkable to see how quickly they matched the musical associations we established with their artwork!
The most fun was having two students at a time come up and press the pedals to trigger the action themselves in synchronization with the musical motifs. This worked so well that i'm going to try to design future Visual Workshops with more pedals so that larger groups of students can get involved in activating the visual triggers in conjunction with the elements they hear in the music.
(Another note to self: kids remember everything they draw. I mean, EVERYTHING - i fit in almost all the pictures, but had to contend with a handful of disappointed faces when i didn't have time to scan in this turtle or that car or that balloon...next time, put EVERYTHING into the Visual Recital!!)
I had several video cameras running, but no hands to actually push the "record" button - what with all the activity and excitement, i just didn't get a chance to lay down a lot of video or audio. What i need in the future is a team of interns to help me set up the documentary equipment...well, i'll go through the few minutes that i did manage to record and see if i can pull together a short clip. If not, i'll make sure that i plan the videotaping at the next Visual Recital workshop in Greenport, NY more carefully.
whew...quite a long-winded blog post today! Lots of exciting ideas, lots of stuff learned - i'm already excited about the next project!
Visual Recital Workshop Jan. 12, 2008 2:00 pm in Willow Grove, PA
Presented by the Northern Delaware Valley Chapter of PMTA, the Visual Recital Workshop comes to the Jacobs Music Store in Willow Grove, PA on Jan. 12, 2008 at 2:00 p.m.! Join pianist Hugh Sung as he demonstrates artistic and pedagogical techniques for combining synchronized visuals with live performance.
Admission: $10 for adults, $5 for students (free admission for members of the Northern Delaware Valley Chapter of PMTA)
This is a terrific performance of "Escuala" ("The Shark") by Astor Piazzolla, arranged by the remarkable Accordionist Lidia Kaminska and performed by her in collaboration with Saxophonist Brian Sacawa and pianist Hugh Sung at the Great Auditorium in Ocean Grove, New Jersey. Enjoy!!
Partnering with the Notes at 9,000 Emerging Artist Series, we developed a pilot Visual Recital Workshop for the 2007 Mad Cow Festival, where amateur artists created mixed media visuals for synchronized live performance to the music of Debussy. I collaborated with winners of the Emerging Artist Series competition in a fantastic concert combining dance, music, and visuals in a format that one listener proclaimed as "the future of live classical concerts!" With the Visual Recital Workshop, the audience member is immersed in the performance and plays a vital participatory role as co-creator with the musician. Many thanks to SoYoung Lee, Amy J. Clark, Charmain Schuh, and the creative team at The Dairy for this innovative approach to the live concert experience! Here's the video storyboard:
Pianoteq is an amazing piano simulation program that allows for unprecedented musical expression on digital pianos. If you're new to digital pianos and virtual instrument plugins, this episode will help untangle all those wires and break down the setup process into easy steps. I start with simple diagrams for either a computer or dedicated sound module setup - aka, the Muse Receptor - then follow with a "live video" example of a computer setup with the help of a talented young pianist, Katie Tran.
In this semi-documentary, i collaborate with three musicians from Astral Artistic Services to present a pilot outreach program for the Meade School in North Philadelphia, utilizing my Visual Recital techniques. A behind-the-scenes look at how the show is set up, followed by the presentation itself. The kids had a blast, particularly with the visualizations for Paul Schoenfield's "Cafe Music"! Other pieces featured were a movement from Beethoven's Piano Trio in C minor, Op. 1 No. 3, and the Adagio movement from Brahms Piano Trio in B major, Op. 8.
Musicians:
Jennifer Curtis, Violin
Susan Babini, Cello
Michael Mizrahi, Piano
Note: this is a smaller file size version for faster playback at reduced video quality
In this semi-documentary, i collaborate with three musicians from Astral Artistic Services to present a pilot outreach program for the Meade School in North Philadelphia, utilizing my Visual Recital techniques. A behind-the-scenes look at how the show is set up, followed by the presentation itself. The kids had a blast, particularly with the visualizations for Paul Schoenfield's "Cafe Music"! Other pieces featured were a movement from Beethoven's Piano Trio in C minor, Op. 1 No. 3, and the Adagio movement from Brahms Piano Trio in B major, Op. 8.
I've been receiving invitations from a growing number of venues for the Visual Recital concept, and i wanted to post some of the workshop proposals that i'm offering in conjunction with VR performances:
Visual Recital Workshops
A. Visual Recital Art Workshop - I've been invited to give a Visual Recital workshop/concert in Colorado this summer. The neat thing about this will be the involvement of about 70 kids - i will introduce some classical piano works, then direct the students in small teams to create artwork depicting their feelings and reactions to assigned sections of a particular piece. Their artwork will be digitally scanned, then incorporated into my Visual Recital framework and performed "live" for the kids and their families! Perhaps something like this could be explored with a group of music students, or even collaboratively between a team of music students and a team of art students? Lots and lots of interesting artistic possibilities here! I've been invited to present a similar workshop in Long Island in the fall.
B. Visual Recital How-To - How about a nuts and bolts workshop for the teachers themselves? I could give a short demo and then a technical overview of the various components involved in creating a Visual Recital presentation. This might be a wonderful time for discussion and brainstorming to think of transforming ways classical music can be presented both in the teaching studio and in public performance venues.
C. New Possibilities with Digital Instruments - I'm starting to work pretty extensively with Digital Pianos, thanks to a fantastic new program that's recently been upgraded called Pianoteq (developed in France). This program is actually a piano simulator, as opposed to a "sampler" that comprises the architecture of virtually every digital piano on the market. What makes this program so exciting is the fact that real, acoustic piano phenomena can be accurately simulated on a digital piano for the first time - everything from sympathetic vibrations, to fully continuous pedal effects, to the way the hammers interact with vibrating and non-vibrating strings, cross string vibration effects - the full parameters are pretty breathtaking, but the really amazing aspect is how expressive the program actually feels. This could become a powerful - and affordable - new tool for teachers looking for a way to offer serious instrument alternatives for students with more modest budgets. One possibility would be to incorporate a Visual Recital performance simultaneously demonstrating the Pianoteq program.
Here are some suggested lecture/workshop/performance applications of the above ideas:
1. Barebones technical presentation - video demonstration and Powerpoint-style slideshow with brief lecture and discussion on the Visual Recital concept and how it can be implemented in pedagogical settings. Approximate duration: 30-45 min.
2. Live performance Visual Recital demonstration (~12-15 min.) using front projector and small screen and existing piano at venue. Powerpoint-style slideshow with lecture and discussion on the Visual Recital concept and how it can be implemented in pedagogical and performance settings. Approximate duration: 45-60 min.
3. Live performance Visual Recital demonstration (~12-15 min.) with full technical setup, employing 7' x 9' rear projection screen and digital piano demonstrating cutting edge Piano simulation software (Pianoteq). Powerpoint-style slideshow with lecture and discussion on the Visual Recital concept and other supporting technologies for innovative pedagogical and performance practices. Approximate duration: 60 - 90 min.
Please note that running times are approximate - they can be extended or shortened as needed.
Yesterday, we had a run-through of my new visualizations designed for 3 piano trio works: the first movement of Beethoven's Trio in C minor, Op. 1 No. 3; the slow movement from Brahm's Trio No. 1 in B major, Op. 8; and the first movement of "Cafe Music" by Paul Schoenfield. During my initial Visual Recital planning meeting with Astral Artists Jennifer Curtis (violin), Susan Babini (cello) and Michael Mizrahi (piano), Jennifer expressed a desire to participate in the 'pedal pushing' activities (she apparently has a lot of experience with a drumset!) My initial idea had been for the pianist to do all the video triggers himself, but getting the other players involved made so much sense!
The initial problem was finding a hardware solution to allow for multiple pedal and keystroke assignments into a single computer. Fortunately, i found a great product from P.I. Engineering: a USB Switch Interface that allows for up to 12 push button or foot tredle inputs. A friend of mine suggested using standard MIDI Damper pedals with 1/4" - 1/8" mono adapters, and it turns out that they work beautifully as input triggers!
Each pedal was assigned a letter keystroke ("A" for the piano, "B" for the violin, and "C" for the cello).
Splitting the pedal trigger tasks among 3 players makes for a much easier job, particularly in situations with tricky page turns. It also opens up the possibility for much greater complexity and interactive visualizations!
Michael was a brave soul, not only using the pedal trigger for the visual triggers, but also tackling the use of my Tablet PC as a digital music reader with a second footswitch for turning pages!
We're aiming for our first Astral/Visual Recital show at one of the Philadelphia public schools sometime next month. In the meantime, Michael will be borrowing both my Toshiba Tablet PC (which will run the visualizations via Liquid Media) and my backup Fujitsu ST5022D tablet pc as a digital music reader ("Kaylee"), along with the USB Switch Interface and the pedals for the group to practice on. Generous soul methinks i yam, eh? Well, the excitement really comes from seeing other musicians having an opportunity to try out some cutting-edge technology and seeing the true benefits that come from these tools. That can only happen with hands-on experience.
BTW, special thanks go to David Michie for taking time to show the wares of his beautiful Violin shop (right next to my office!), and to Rich Galassini for the fantastic tour of Cunningham Piano Factory. I'm using pictures taken from both locations for the Beethoven Trio movement, giving a visual "story" of how trees become musical instruments. I can't wait to show you clips from the 'finished product' once we play this show in public!
Music Meets Tech #24: Visual Recital in Brazil, Part 1 - Pictures at an Exhibition
I finally received the DVD video and CD audio files for my performances at the I Virtuosi Festival in Recife, Brazil this past December! The stage crew did a fantastic job getting the Visual Recital setup together - and what a terrific setup, with the images filling up the entire back wall of the theater!
Here is the first portion of Mussorgsky's "Pictures at an Exhibition", with the following sections to be presented as consecutive video podcasts. The entire performance will eventually be available as a hi-def video download.
This video performance is being presented as a Quicktime Video file (approximately 52.5 MB) and as an iPod-compatible Video Podcast (77 MB for the video iPod format). Running time is appoximately 9.5 minutes. .
