Over the Frail Dream

and bpm dragging

old https://files.catbox.moe/n8h8pt.zip

enjoy

----

update

grapheme wrote: *

Sat Mar 07, 2026 10:24 pm

https://files.catbox.moe/1l8axa.zip

fixed rmb + d/del to delete on drag + drag from any beat line + fixed catbox nuking the zip contents

the cereal is flying out of my mouth as we speak

will test this out on some stuff and post thoughts

i think this shit is perhaps cooked, especially on this song where it just cannot morally decide if it is syncable (is 192.67 bpm)

speak boomer please

iesuoykuot wrote: *

Wed Mar 04, 2026 12:36 pm

i think this shit is perhaps cooked, especially on this song where it just cannot morally decide if it is syncable
fuck5.png
(is 192.67 bpm)

it does that sometimes, in fact it actually detects the correct bpm and another BPM 1/3 higher or lower (example) in the vast majority of cases

however i'm yet to find a case where it cannot detect the BPM at all, barring live performances and BMS ports, which naturally do not have a set BPM (and even then AV seems to do quite well for one single BPM)

few tests (audio files linked so you can check yourself if curious)

some general notes:
- rmb to cancel bpm movement didnt work for me
- a keybind to insert bpm changes would be nice. maybe this exists already in nu-av but its not documented anywhere
- it'd maybe be nice if one of the difficulty settings just tried to put the bpm changes on downbeats only to make ddream style whole beat adjustments more intuitive
- if there was a way to delete bpm changes in-place (by adjusting the prior bpm to preserve the positions of everything after) that would make adjusting sync on the higher difficulty settings easier sometimes (specifically for deleting oversensitive changes)
- i was pleasantly surprised by being able to drag multiple bpms at once
- likewise for being able to tab select with autosync to only do specific parts
- not too rough generally seems the most reliable
- most common issue is it syncing to the 8ths
- when this works it definitely seems faster overall than manually syncing
- not too rough has unfortunate acronym overlap (i guess hntr is fine?)
- i couldnt tell the difference between warp (holding alt) and just dragging the beat normally

Kraken of the Sea (catbox keeps nuking the link for this instantly for some reason) - really seems to struggle with syncing this no matter the settings. it's mostly constant bpm but has a slight floatiness. gets it pretty much completely wrong on single bpm or not too rough, on hurt me plenty and ultra violence it sometimes gets it right but sometimes bugs out and does a bunch of bpm changes where it shouldnt. not too rough was decent as a base for manual adjustment but sometimes it was a bit too sparse when i wanted to do beat-by-beat adjustments, whereas with the higher "difficulties" i had the opposite problem where there were so many bpm changes on offbeats that it got in the way of trying to fix things. i could probably make it synced using either method but i think i would be faster with the traditional AV wrench syncing method

Sagat - similar song to kraken, vaguely floaty old video game music with not-obvious downbeats. unsurprisingly because of how weird the snare rhythm is the autosync has a really hard time finding the beat on hmp/uv. funny enough not too rough sets the whole thing to a single bpm

Spoon- figured this would be more fair for the autosync. not too rough actually does a pretty good job here. sometimes it's slightly off but never enough that i think anyone would notice during gameplay. same goes for hmp (which mostly produces pretty reasonable adjustments on the 8ths). i did notice it gets tripped up at 2:24.551 though because there's more intense 16th drums right before. fortunately this is very easily fixed. (also, not too rough doesnt get tripped up here). ultra violence makes a lot of mistakes unsurprisingly due to syncing to a lot of small offbeat onsets detected randomly

Deep Water - at first i was going to write this off as a complete failure but then i realized it wasnt too hard to adjust the beats on hurt me plenty to be right. i think the main issue with this is that a lot of the time it places the downbeat on the 12th before. again probably another case of "cant distinguish the downbeat" because its just guitar strumming

The Shining Path - very surprised at how good it did on this one. not too rough with a 1.5x bpm multiplier perfectly syncs a lot of this, but occasionally it trips up and starts syncing an 8th off, though usually it readjusts. unfortunately all of the sync methods completely fail the ending slowdown but still, much better performance than expected

Volcano - similar to the shining path, gets it surprisingly right but sometimes messes up and starts putting the downbeat on the 8ths

Woodpecker No. 1 (volume warning) - believe it or not, mostly works. once again there's the isssue where a few times it starts going an 8th off but it mostly gets it right. bpm is surprisingly stable even on uv a lot of the time. this is the first time i noticed woodpecker has bpm changes

maybe i'll write more later but this is probably decent for now. definitely recommend giving this a shot whenever you're syncing something floaty; sometimes it wont work but when it does i think it will save a lot of time

nice ty

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

few tests (audio files linked so you can check yourself if curious)

some general notes:
- rmb to cancel bpm movement didnt work for me

i remember implementing the rmb cancel so idk whats going on there

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- a keybind to insert bpm changes would be nice. maybe this exists already in nu-av but its not documented anywhere

ctrl-b will insert a bpm change, delta added it

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- it'd maybe be nice if one of the difficulty settings just tried to put the bpm changes on downbeats only to make ddream style whole beat adjustments more intuitive

i used to have hurt me plenty do something like that but i found it obnoxious when the bpm wasnt changing. i think i'd rather have something like (conceptually) invisible bpm changes on every measure line that appear when you mouse over and turn real when you drag

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- if there was a way to delete bpm changes in-place (by adjusting the prior bpm to preserve the positions of everything after) that would make adjusting sync on the higher difficulty settings easier sometimes (specifically for deleting oversensitive changes)

deleting bpms in place is a nice idea. i'll add it to the bpm drag so drag + del yanks it. it already kind of happens if you drag bpms past the next bpm

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- i was pleasantly surprised by being able to drag multiple bpms at once

i think the only reason i added that was for the same reason immersive videogame people add things to their games. its a natural interaction and it should work so why doesn't it

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- likewise for being able to tab select with autosync to only do specific parts

probably the most tedious and error prone thing to implement

in the normal case you just dump out new bpms and dont care about what else is there. so the entire approach on the talking-to-av side changes once you decide you need machinery to keep what is already there locked in place

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- not too rough generally seems the most reliable
- most common issue is it syncing to the 8ths

the three modes are different amounts of post processing on the raw beat tracking. so if not too rough is bad theyll all be bad. and uv can be so aggressive because it knows no changes it makes will make future bpms go off-sync

Windoze wrote: *

Wed Mar 04, 2026 5:32 pm

- i couldnt tell the difference between warp (holding alt) and just dragging the beat normally

warp takes the beat number the bpm you're dragging is currently on and moves that beat around. without it it inserts/removes beats to keep the tempo consistent. basically if you really want to nail the measure structure of a song then it will stop fighting you


the down beats and 8ths thing is like... SOTA deep learning beat tracking is starting to saturate on beat tracking test sets so they don't really have the 8th out problem, which makes sense cause the character of listening to the beat tick on beat and an 8th out is pretty distinct. so if you can hear it deep learning can learn to hear it too. downbeats though they are nowhere near saturating. and if you ask a non-musician where the downbeat is on any given track i don't think they would score well on such a test either. and among musicians they'd only score well on genres somewhat close to what theyre familiar with. so much contextual & musical knowledge is baked into determing the down beat, so much that actually i think deep learning is bottle necked on getting decent training data for the problem rather than the difficulty of extracting it from audio per se

anyway the beat tracker i made just operates on the onsets you can view in the waveform view now and it doesnt know anything else about what the audio is doing. if you isolate 30 seconds of a track theres a dumb trick i can do to more-consistently get the 8th right but i dont really know how to do the trick inline in a larger track. the trick is just to sync multiple times at different offsets and see what one produces a beat grid that best aligns to the onsets. but if you drop the offset part thats conceptually thats the whole algorithm in a nutshell. hmm i may have an idea

grapheme wrote: *

Wed Mar 04, 2026 7:30 pm

anyway the beat tracker i made just operates on the onsets you can view in the waveform view now and it doesnt know anything else about what the audio is doing. if you isolate 30 seconds of a track theres a dumb trick i can do to more-consistently get the 8th right but i dont really know how to do the trick inline in a larger track. the trick is just to sync multiple times at different offsets and see what one produces a beat grid that best aligns to the onsets. but if you drop the offset part thats conceptually thats the whole algorithm in a nutshell. hmm i may have an idea

sometimes the beat grid not aligning with the onsets is desirable. i tested this track and most of the detected onsets are on the offbeat 8th hi-hats (atm the autosync on not too rough does a great job at syncing this btw, it only really messes up one beat that's easily fixed)

yeah it's all set up to look for that kind of thing. but it only works because the assumption is you have some earlier position to be based on that isn't off sync

the zen example to think of is perfect onset detection for a pulse train. say you have 12 onsets. then is that 3 measures of 4 or 4 measures of 3? is each pulse a beat or an 8th or a 16th? so you have this aliasing problem. its the same as the original bpm detection always giving the true bpm and another guess 1.5x out. mine also has the 1.5x problem but you can detect it somewhat reliably if the onsets strengths follow the music stress by looking at the DFT of the onset times (when you only care about a couple frequencies you don't have to do a full FFT)

so the whole thing that makes it work is at that onsets are randomly dropped and the ones that make it through tend to be placed on beats, but you can't assume that. you just find a good starting point and cross your fingers

ok now doing some further experimenting, i think maybe it's more learning how to configure it properly using waveform settings.

i wonder if you could throw it into adjust sync with two tabs that let you use fieste's sync or graeme's sync, and i wish i could tell it to use a specific waveform instead of following my waveform, though that probably is expensive to compute lmao.