h a l f b a k e r y
add, search, annotate, link, view, overview, recent, by name, random
news, help, about, links, report a problem
or get an account
Quantizing in audio is nothing new. This is where an audio
recording of a drum track for instance is moved around as
necessary to fit into a precise, predetermined matrix.
So if you have a sloppy drummer, you quantize the
so it moves late beats backwards and early beats forward.
the rhythmic equivalent of autotune for singers.
The proposed idea is to have the audio track for a video
quantized, but retaining its alignment to the video track.
Easiest example would be to show a drummer playing
sloppily, the audio/video quantize function would be
activated and you would hear AND SEE a drummer playing
perfectly on beat.
This effect is currently done by hand where people
cut up videos and drop them into audio tracks to make
novelty, usually humorous music videos.
So this could be done automatically or with some manual
editing ti fine tweak it. You'd take a video of your aunt
bunion, click "analyze audio track" and it would move the
main "beats" of her conversation to the nearest quarter
for instance giving it the appearance that she's rapping.
Alternatively, you could manually look at the peaks and
valleys of the audio signal over-layed on a beat matrix and
you could move it by hand. Once it sounded right, you'd hit
"play" and watch the video of the person wrapping.
Doesn't have to be just people talking either. Any video
sound could be made a rhythmic performance piece.
You could also add auto tune to the audio track and make
any video where people were talking or even making noise
a music video.
This is an example of the effect where everything was done manually.
The idea would be to do this automatically. [doctorremulac3, Jan 30 2016]
Another great example.
Brilliant. Ripoff of this idea coming in 3...2...1... [doctorremulac3, Jan 30 2016]
Only vaguely related...
[normzone, Jan 30 2016]
Pretty much this but only for videos of people speaking [notexactly, Feb 10 2016]
Video Rewrite paper
SIGGRAPH 1997 [notexactly, Feb 10 2016]
Disney Research VideoSnapping
Pretty much this but for temporally aligning one video to another. SIGGRAPH 2014 [notexactly, Feb 10 2016]
Pretty much this for generating CGI video temporally aligned to existing sound. SIGGRAPH 2014 [notexactly, Feb 10 2016]
||Doing it manually in Final Cut Pro X within a segment,
blade the timing track (not the video itself) just
before the hit point. Go to the retime menu and turn
on Speed Transitions with ripple turned off. Adjust
the chopped timing slightly until it lines up the visual
hit point with the audio waveform hit point. This
gives you an adjustable speed ramp. The segment
can maintain the overall duration (i.e. no ripple).
||Its still a lot of manual work, but because of the
Speed Transitions menu, its so much more effective.
||My idea would be to have a audio/video linked
stretchable timeline where audio peaks would be
automatically moved to the next grid point.
||Fine tuning could be done with a graphic interface,
moving left or right to taste.
||You could have a music audio track as the guide for
your video audio track. Instant rap video complete
with choreography moving to the music.
||You could jump around waving your arms like reciting
the alphabet and it would come out a rap/dance
video moving exactly to the beat.
||See my [links] for prior art. Inverse-Foley Animation is the
one I remembered, but I didn't remember the title, so I did
a Google search for [siggraph video audio sync] and found all