Over the Frail Dream
https://forum.overthefraildream.net/

arrow vortex syncery
https://forum.overthefraildream.net/viewtopic.php?t=52
Page 3 of 4
Author:  iesuoykuot [ Sun Mar 08, 2026 2:07 pm ]
Post subject:  Re: arrow vortex syncery

grapheme wrote: *  Sun Mar 08, 2026 11:32 am
btw when i say this shit does it make sense to anybody
i can understand it fine, except maybe DFT and FFT

Author:  grapheme [ Sun Mar 08, 2026 3:55 pm ]
Post subject:  Re: arrow vortex syncery

that does not inspire confidence!

Author:  Windoze [ Sun Mar 08, 2026 6:22 pm ]
Post subject:  Re: arrow vortex syncery

grapheme wrote: *  Sun Mar 08, 2026 11:32 am
grapheme wrote: *  Wed Mar 04, 2026 9:27 pm
mine also has the 1.5x problem but you can detect it somewhat reliably if the onsets strengths follow the music stress by looking at the DFT of the onset times (when you only care about a couple frequencies you don't have to do a full FFT)
btw when i say this shit does it make sense to anybody
i did fft stuff like a few years ago for a script that estimated bitrate via compression shelves. thats about it for my FFT knowledge though

though while i was in the middle of writing about how i didnt get it i realized what "looking at the DFT of the onset times" meant. at first it made no sense because "DFTs work on sequences wtf would be the point of applying it to a single onset" but then i realized that the onsets as a group are the sequence and i guess the frequency of the onsets gets you something that's like a tempo if you squint hard enough. if thats all correct then the only part i'm confused on is what exactly music stress is if it's something distinct from the onsets

Author:  grapheme [ Sun Mar 08, 2026 7:56 pm ]
Post subject:  Re: arrow vortex syncery

an easy example of musical stress and onset strength not aligning is _234_234_234_234. you hear the gap as the beat

that's assuming the onset strengths are reliable indicators of what people hear as musical stress, which, since it fails to fire at all on some material you can easily hear drums in, is not the case generally. but it's good enough

anyway yea the onsets are a sequence but, unlike the sequences in most expositions of the DFT/FFT you'll find, the samples aren't equally-spaced. but you can just say the times we don't have samples for are 0 and they drop out of the sum. so you can use the DFT directly. like i mean go to wikipedia, go to discrete fourier transform, scroll down to definition, translate that one equation into code, done. you can't really do the FFT cause it wants operate on all these 0-valued samples we don't have. since it comes from audio what you could do is take the original audio buffer, zero it, stuff the onset strengths into the sample corresponding to the onset time, then FFT. now if you do that in a sliding window instead of the entire file you get something similar to how people were approaching this problem 20 years ago. i never bothered with any of that stuff

Author:  Windoze [ Mon Mar 09, 2026 12:18 am ]
Post subject:  Re: arrow vortex syncery

grapheme wrote: *  Sun Mar 08, 2026 7:56 pm
an easy example of musical stress and onset strength not aligning is _234_234_234_234. you hear the gap as the beat

that's assuming the onset strengths are reliable indicators of what people hear as musical stress, which, since it fails to fire at all on some material you can easily hear drums in, is not the case generally. but it's good enough
i shouldve clarified that i get it conceptually but not how you would programmatically detect that. iirc you said all of this is relying purely on the onsets, so how is musical stress detected if it's something distinct from an onset? (unless if that whole thing was posed as a hypothetical. in that case s/how is/how would and whatnot)

Author:  grapheme [ Mon Mar 09, 2026 1:35 am ]
Post subject:  Re: arrow vortex syncery

for the actual syncing i don't and just assume onset strengths are good enough. hmm having ideas again

for the DFT thing it doesn't matter. it'll just extract whatever periodicity is there. that'll tell you if a rhythm at whatever bpm is present in the onsets you have. so really any kind of rhythmic variation means this weakly flags a lot of possible bpms and so this way of detecting bpms just dumps a bunch of plausible BPMs on you. but i know the bpm by, , , i dont want to get into it. but my method has the same 1.5x ambiguity AV's bpm detection has. maybe not as bad

but you can use the DFT for that cause if you already know the BPM you want to test you can compute the X[k] for that bin and the 1.5x bin and, if the onsets are good enough, that'll tell you if you have

1 2 3 4 5 6 7 8 9 10 11 12
or
1 2 3 4 5 6 7 8 9 10 11 12

and you can adjust the bpm accordingly if you got it wrong. doesn't always detect a wrong bpm but when it does its always right i think

i was about to add a bit here about why the DFT picks that out but i decided against it. the first book i read on the FFT was julius o smiths and he begins it by being all, okay, imagine a 3d vector, okay, now, imagine a big N-dimensional vector, look, it's so simple! and at the time i was like wtf is this. and now here today i was like, how could I explain this, okay imagine a 3d vector,

Author:  iesuoykuot [ Mon Mar 09, 2026 11:36 am ]
Post subject:  Re: arrow vortex syncery

grapheme wrote: *  Mon Mar 09, 2026 1:35 am
i was about to add a bit here about why the DFT picks that out but i decided against it. the first book i read on the FFT was julius o smiths and he begins it by being all, okay, imagine a 3d vector, okay, now, imagine a big N-dimensional vector, look, it's so simple! and at the time i was like wtf is this. and now here today i was like, how could I explain this, okay imagine a 3d vector,
can i have an elaboration on the 3d-vector vs n-vector, i don't understand

Author:  grapheme [ Mon Mar 09, 2026 2:00 pm ]
Post subject:  Re: arrow vortex syncery

so glad you asked

theres two ways to understand fourier transforms. one understanding starts with the continuous + calculus and the other with the discrete + linear algebra. they both end up at the same place. on the linear algebra it makes sense to talk about dimensions; its the number of components in your vector, the number of columns in your matrix. on the continuous side if you want to talk that way you start talking about infinite dimensional spaces. this makes a lot of sense if you know what it means, but nobody spoke this way when fourier came up with this stuff, so let us set it aside

historically continuous was easiest to understand cause computation was difficult. now computation is easy so discrete is easier for programmer types. if you get calculus drilled into you early in life then continuous is still easier for many people

on the continuous side you can justify the fourier transform by observing what happens when you take an integral like integrate[ f(x)g(x)dx ] and making f(x) and g(x) two applications of the exponential function that treat x as an imaginary number. this requires some mental infrastructure to be developed for it to be a good time

on the discrete side we use linear algebra. now historically linear algebra was developed to understand the f(x)g(x)dx pattern more generally but in this case the generality means it can plug directly into your geometric intuitions. so the fourier transform becomes a lot easier to understand

discretely the integral becomes sum[ f[n]g[n] ] or in code
float dot_product(float[n] f, float[n] g) {
    float result = 0.0f;
    for (i : n) result += f[n]*g[n];
    return result;
}
n is the number of dimensions

in geometry if you have a 3d vector and you want to know its x component you can do this
dot_product(v, { 1, 0, 0 });
{1,0,0} is direction in space that points in the x direction. this works for any direction, not just the x y and z directions. So doing
dot_product(v, dir) * dir
will isolate the part of v pointing in the dir direction

in the DFT if you want to isolate a given frequency you do
dot_product(x, e_f) * e_f
where e_f is exp(i*f), sampled at whatever sample rate x is at. that dot_product is giving you the value that goes in a given FFT bin, and the multiply is turning it into the sine wave you wanted to isolate

and mathematically its the same except in nice 3D geometry you have 3 dimensions, which is sensible, and in the DFT you have as many dimensions as you have samples in your signal, which is also sensible but may be a new meaning of dimension than you are used to

the reason why you can talk about every frequency's corresponding e_f independently with this, and do a big stack of dot products without worrying about other frequencies interfering, is the kind of thing linear algebra gives you the machinery for understanding. this is largely the same mental infrastructure i mentioned before, that needs to be developed for it to be a good time

Author:  iesuoykuot [ Mon Mar 09, 2026 7:39 pm ]
Post subject:  Re: arrow vortex syncery

grapheme wrote: *  Mon Mar 09, 2026 2:00 pm
so glad you asked

theres two ways to understand fourier transforms. one understanding starts with the continuous + calculus and the other with the discrete + linear algebra. they both end up at the same place. on the linear algebra it makes sense to talk about dimensions; its the number of components in your vector, the number of columns in your matrix. on the continuous side if you want to talk that way you start talking about infinite dimensional spaces. this makes a lot of sense if you know what it means, but nobody spoke this way when fourier came up with this stuff, so let us set it aside

historically continuous was easiest to understand cause computation was difficult. now computation is easy so discrete is easier for programmer types. if you get calculus drilled into you early in life then continuous is still easier for many people

on the continuous side you can justify the fourier transform by observing what happens when you take an integral like integrate[ f(x)g(x)dx ] and making f(x) and g(x) two applications of the exponential function that treat x as an imaginary number. this requires some mental infrastructure to be developed for it to be a good time

on the discrete side we use linear algebra. now historically linear algebra was developed to understand the f(x)g(x)dx pattern more generally but in this case the generality means it can plug directly into your geometric intuitions. so the fourier transform becomes a lot easier to understand

discretely the integral becomes sum[ f[n]g[n] ] or in code
float dot_product(float[n] f, float[n] g) {
    float result = 0.0f;
    for (i : n) result += f[n]*g[n];
    return result;
}
n is the number of dimensions

in geometry if you have a 3d vector and you want to know its x component you can do this
dot_product(v, { 1, 0, 0 });
{1,0,0} is direction in space that points in the x direction. this works for any direction, not just the x y and z directions. So doing
dot_product(v, dir) * dir
will isolate the part of v pointing in the dir direction

in the DFT if you want to isolate a given frequency you do
dot_product(x, e_f) * e_f
where e_f is exp(i*f), sampled at whatever sample rate x is at. that dot_product is giving you the value that goes in a given FFT bin, and the multiply is turning it into the sine wave you wanted to isolate

and mathematically its the same except in nice 3D geometry you have 3 dimensions, which is sensible, and in the DFT you have as many dimensions as you have samples in your signal, which is also sensible but may be a new meaning of dimension than you are used to

the reason why you can talk about every frequency's corresponding e_f independently with this, and do a big stack of dot products without worrying about other frequencies interfering, is the kind of thing linear algebra gives you the machinery for understanding. this is largely the same mental infrastructure i mentioned before, that needs to be developed for it to be a good time
so it's almost like buffering data then looking through it and evaluating it with a numerical filter?

Author:  grapheme [ Mon Mar 09, 2026 8:54 pm ]
Post subject:  Re: arrow vortex syncery

yeah one way of thinking about the DFT on an N-sample window/signal is it does N filters independently and the FFT is an optimisation that reduces a lot of redundant work between the filters. filters is a pretty broad term but for linear filters and dsp and stuff this isn't an analogy and is exactly what is happening. since sines/cosines/complex eponentials go on forever theyre a little unusual but in terms of transfer functions* they are very easy to interpret

*wikipedia not linked to preserve the innocent

Page 3 of 4 All times are UTC
Powered by phpBB® Forum Software © phpBB Limited