I don't know why I didn't switch from youtube-dl to yt-dlp earlier. It's way more reliable and doesn't throttle to kb/s anymore. Configuration is similar, you can basically use it as a drop-in replacement.
My custom webapp frontend to download video directly to my NAS and watch with Plex works way better and I proabably can remove the cancel button to kill slow downloads soon.
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!