Use MIDI to control video playback speed and match location

Hey folks,

I know almost nothing about computers (except that they hate me), but I know what I'd like to do, and hope MT can do it.

My GF has a head-cam video of her jogging through the park at a set BPM (steps per minute, actually). When she runs on the treadmill at home, it's at varying speeds, matched to the music BPM she's playing (she uses the music to pace herself). I'd like to have the video playback not only match the speed of the music she's playing but also sync her steps in the video to the beat.

The computer is running Win 7/64.

I use VLC for video playback, but could use other programs.

I use Traktor DJ software for music playback. It has MIDI out.

Again, I don't know much about computing, and will have someone else do the hard/programing work, I'm just trying to see if ANYTHING will do what I need.

It seems to me that since Traktor supplies MIDI out, it's actually aware of where the beat is (4/4 time, for example), and may be able to relay that info via the MIDI.

VLC can change it's playback speed in very small increments with a single line of code, easily able to accommodate full, half, or even quarter BPM changes.

If there was a way to embed the BPM info into the video, then perhaps a program could read the incoming audio speed and BPM, and match the video speed and BPM to it. Thus, each of her steps in the video would match whatever beat speed the audio is playing.

Thanks

Dave

Hi Dave

Since VLC doesn't receive MIDI information in a live sort of way (receiving MIDI clock), I don't think MIDI Translator can help. I will leave this up here, though, in case anyone in our community is interested in taking this on.

Hope you get the help you need
Jared

Thanks Jared.

I'm not stuck with VLC, I just use it because I like to support open-source. If there is another video player that is known to play nice with MT, I can switch. Again, I don't know much about computers, I just know what I want to accomplish.

Also, I came across this while doing my research. I don't know if it helps, as I don't speak computer.

MIDI with VLC media player

Every so often, someone asks if VLC could support MIDI, the de-facto industry standard for digital music instruments.
Input

As a matter of facts, VLC already supports MIDI - to some extent. The Standard MIDI Files (*.MID) parser plugin was published in October 2007. In April 2008, RIFF files (*.RMI) support was added. That plugin was in VLC releases since version 0.9.0. At the time of writing (July 2009), VLC does however not support hardware MIDI inputs neither networked MIDI over RTP.

Also, support for other file formats is missing, including:

Karaoke files (*.KAR) and
Extended MIDI files (*.XMI), found in many old games.

As a matter of facts, I am looking for legal and good documentation on the XMI format (not XMF!), so that support could be added as well.
Synthesis

But then, VLC still needs to synthesize sounds from the MIDI events. Unfortunately, hardware synthesis with the audio adapter would be rather tricky. Indeed, the VLC audio mixing and output core subsystem was designed for PCM samples (with special support for S/PDIF). Also, while FM synthesis was common in the good old days (notably DOS games), most modern (cheap) audio adapters, such as AC'97 audio controllers or Intel HDA cards, lack hardware synthesizer anyway.

So VLC resorts to software synthesis with a sound font file (*.SF2) through the Fluidsynth open-source library. Timidity is more commonly used, but it is not available as a C programming library out of the box.

That raises two potential problems:

First, the Fluidsynth plugin must be enabled. Many binary packagers omit it. It should be visible in the list of codecs from the preferences dialog, if it is supported by your particular VLC build.
Second, a sound font must be obtained separately, and configured. If you know a decent compact GPL-compatible sound fount, please contact me so we can include it in the future VLC releases...

If not included in your version, you may need to complain to your packager, and build VLC from source yourself. You can easily find free sound fonts websites from Google; then tell VLC where to find it from the preferences dialog (Codecs / Fluidsynth).

Now you can play MIDI files!

http://www.remlab.net/op/vlc-midi.shtml

Hi Dave

It looks like you should be able to use .MID files that are exported from a software like Ableton or Logic. This should provide a little more info. I don't think this is something you would use MT for, but you could definitely use a DAW to provide a .MID file and use that to load into VLC, pending VLC can pull the tempo. I'm not 100% sure, though, as I've never seen something like this.

J

A little confused… But I think I can help.

(NOTE:) I have many years in video and music and might help, though I have always been more of a creative than an engineer…. I'll try.

MY QUESTION:
Sooo….. Are you saying that you want to take video she recorded while jogging to the beat (likely 4/4) and then create a video that tightens up the syncing for a more musical videographic effect? More "music video" style, instead of the original recording.
OR
…are you saying that the syncing in the original video is fine, but you want to simply change the tempo to match new music, perhaps to mashup (montage) her various videos to a single song?

These pose two very different problems and the first one is difficult, and the second one is potentially very easy.

Could you clarify what your end goal is? One of these two, or something else.