An open source audio-responsive VJ application that we (Joel Pitt, Kelly Cheesman, Will Marshall) have been building in Processing. This application uses a multitouch control surface to trigger videos, mask layers, control motion and capture live video. Visual inspiration came from a video by Kate Moross for SIMIAN MOBILE DISCO .

INSPIRATION

CONCEPT

We wanted to to make sound-responsive shapes mask video in real-time.

We used Joel’s JazzMutant Lemur as a control interface (shiny awesomeness), as it supports OSC and has a built-in physics model, allowing us very hands-on control of our visuals.

Features include:

  • Four live camera feeds
  • Preset patterns for the shapes
  • OpenGL blending for layering video
  • Beat detection
  • Interface to select from a grid of videos
  • Use of gifs as layer masks

DEMO

SHOWS

July 2010 Maya

EXAMPLES

[SinglePic not found]

Suggestions, contributions and pings welcome.

TODO

  • Video controls:
    • loop points
    • scrubbing (manual frame control)
  • Package media and make available for download
  • Make all points respond to same frequency and allow frequency response to be configurable.
  • Better presets which save frequency responses and size of points.
  • Refactor demo mode so that each component has a doDemo method or something similar
  • Make responses dependent on intensity (stop stuff going nuts when there is hardly any noise)
  • Allow other backgrounds (GIFs, images, plain, colour)
  • Add more point shapes and images.
  • Add a PointMotion that makes things bounce up and down.
  • Convert Rorshach into a PointArtist object
  • Scale point size range to resolution of frame.

7 Comments

  1. Tweets that mention kellective -- Topsy.com
    October 4, 2010

    [...] This post was mentioned on Twitter by Joel Pitt, Jodi. Jodi said: Well done! RT @kellective Speed of SoundProject has been chosen as a finalist for the NZ Open SourceAwards 2010. http://bit.ly/bRZdmU [...]

    Reply
  2. cellstorm
    October 8, 2011

    hello. I am trying to run sv on linux. I have gotten quite far, but have no clue where to put the movies

    Loading background movies…
    No movies!
    Exception in thread “Animation Thread” java.lang.ArrayIndexOutOfBoundsException: 0
    at SpeedOfSound$JMCMovieBackgroundArtist.init(SpeedOfSound.java:1049)
    at SpeedOfSound.setup(SpeedOfSound.java:267)
    at processing.core.PApplet.handleDraw(Unknown Source)
    at processing.core.PApplet.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:662)

    is what I get right now…

    please give me some hints….

    thanks in avance…

    Reply
    • admin
      October 9, 2011

      Hi Cellstorm, first of all check that your movies are in a folder called ‘data’. The version on Github already has a demo movie in that folder and should work… please let me know if this still doesn’t work and I will try help you

      Reply
  3. cellstorm
    October 9, 2011

    yeah sure I have this folder. and there is station.mov in it.

    it has this codec:

    Format : Sorenson 1
    Codec ID : SVQ1
    Codec ID/Info : Sorenson Media Video 1 (Apple QuickTime 3)
    Duration : 4s 303ms
    Bit rate mode : Variable
    Bit rate : 1 050 Kbps
    Width : 160 pixels
    Height : 120 pixels
    Display aspect ratio : 4:3
    Frame rate mode : Constant
    Frame rate : 23.976 fps
    Bits/(Pixel*Frame) : 2.282
    Stream size : 551 KiB (99%)

    I also have Movs with

    Format : DXDI
    Codec ID : DXDI
    Duration : 10s 0ms
    Bit rate mode : Variable
    Bit rate : 21.8 Mbps
    Width : 640 pixels
    Height : 480 pixels
    Display aspect ratio : 4:3
    Frame rate mode : Constant
    Frame rate : 25.000 fps

    I have put them into the data folder, and tried to run it agan, but I am getting the same error..

    Reply
  4. Joel Pitt
    October 9, 2011

    Hi cellstorm,

    I’m afraid that non-OSX development is a bit tricky. Certain codecs would work okay, and others wouldn’t.

    I would experiment with JMC Movie and various Linux codecs to see what works, and then use ffmpeg or mencoder to convert all the videos to that format.

    Sorry I can’t be more help, it’s been a couple of years since I worked on this!

    Reply
  5. cellstorm
    October 9, 2011

    so you think its all about codecs and I do not have to alter code? I hope you are right, I will try some movs, oggs etc. with the jmc demo applications for playing videos and then try again in sv

    Reply
  6. cellstorm
    October 9, 2011

    well nothing works so far with jmcvideo…
    I emailed angus forbes now. I can only suspect that it is because I am on Maverick64 , and javafx 1.2, which is jmcvideo build upon, is only available in 32bits. Now I have inspected the native gstreamer library (libGStreamer.so in linux) and I have all the libraries installed in 32bit, which it links to, so I thought that part should work fine… maybe I am wrong here, and it needs sth. else.

    meanwhile, do you remember why You guys did not succeed in letting sv run on linux? I only ask because you explicitely say so on the page…

    and if jmcvideo is only needed for loading videos (and thats hopefully not really a big part of sv), would it be possible for a processing – newbie to hook up annother library for doing the video loading? http://gsvideo.sourceforge.net/ also uses gstreamer, as jmcvideo does through javafx….

    Reply

Leave a Reply