Declouding: Spotify


I miss winamp, limewire, Purchasing CD’s from Sanity and ripping them down to mp3’s …

Sure spotify is convenient. It’s also a subscription I’ll be paying forever, with no guarantee the songs I like won’t disappear overnight when licensing deals expire.

Why am I paying for someone else to control my data? The only benefit I can see of Spotify is music discoverability … but I am willing to lose this if it means I control my audio library.

So … I built a pipeline to rip audio from YouTube, organize it properly, and manage it locally with cmus. Notes below!


The Idea

Simple enough (in theory)

  1. Rip audio from YouTube URLs → store in staging area
  2. Manually organize files into genre/artist/album/ structure
  3. Tag and move to final library with proper metadata
  4. Add to Cmus listen and enjoy!

So I thought two Python scripts handle this:

  • yt_ripper.py - Downloads audio from a list of URLs
  • metadata.py - Reads folder structure, applies ID3 tags, moves to library

A Makefile ties it together:

rip:        ./yt_ripper.py urls.txt
process:    ./mtadata.py
clean:      rm -rf ~/Downloads/music/*.{mp3,mp4,webm}

Simple workflow: add urls to urls.txt, run download, organize, process.

Should take 30 minutes to build, right?

Not even close!

!! A disclaimer before you continue.

Ripping music from YouTube violates TOS and deprives artists of potential revenue.

However, I’m not avoiding payment out of cheapness … I’m rejecting a fundamentally precarious relationship with media access. I believe in principled digital autonomy, and the continuing enshittification of online platforms has only strengthened this conviction.

Following this guide could get your YouTube account banned. I acknowledge and accept the risks. Also check your local jurisdiction regarding copyright law. Some places are stricter than others.


Problem 1: YouTube Hates You

My first version was naive:

def download_audio(url, output_dir):
    options = {
        'format': 'bestaudio/best',
        'postprocessors': [{'key': 'FFmpegExtractAudio', 'preferredcodec': 'mp3'}],
        'outtmpl': f'{output_dir}/%(title)s.%(ext)s',
    }
    
    with yt_dlp.YoutubeDL(options) as ydl:
        ydl.download([url])

Worked great for 2-3 videos. Then:

ERROR: Sign in to confirm you're not a bot.

YouTube rate-limited me immediately when trying to download 50+ URLs.

Solution: Slow down and authenticate

Added sleep intervals between downloads:

'sleep_interval': 5,
'max_sleep_interval': 10,

But that wasn’t enough. YouTube’s bot detection is aggressive. I needed to authenticate using browser cookies.

The cookie nightmare:

Initially tried 'cookiesfrombrowser': ('firefox',) - yt-dlp should auto-extract cookies. Didn’t work. Firefox uses encrypted cookie storage on Linux, requiring the keyring library. That also failed with “unsupported keyring” errors.

Final solution: Export cookies manually using a Firefox extension, save as cookies.txt, and point to it:

cookies_file = Path.cwd() / 'cookies.txt'
options = {
    'cookiefile': str(cookies_file),
    # ... rest of config
}

Works perfectly. YouTube thinks I’m just browsing logged in.

!! NOTE !! Cookies contain auth tokens. Don’t upload them to git or share them. They should never leave your personal device.


Problem 2: Format Availability

Even with cookies, every video failed:

ERROR: Requested format is not available.

Running yt-dlp --list-formats revealed the issue:

WARNING: Signature solving failed
WARNING: n challenge solving failed
WARNING: Only images are available for download

What I think is happening … is YouTube obfuscates video URLs with JavaScript challenges. yt-dlp needs a JS runtime (Node.js) to decode them. I had Node.js installed, but yt-dlp wasn’t using it properly.

Solution: Use alternate player clients

YouTube has multiple API endpoints for different clients (web, android, iOS, etc.). The Android client bypasses signature checks entirely:

'extractor_args': {
    'youtube': {
        'player_client': ['android_music', 'android'],
        'player_skip': ['webpage', 'configs']
    }
}

This tells yt-dlp: “Skip the web player, use the Android API directly.” No signature solving required. Every video downloaded successfully after this change.


Problem 3: Chapter Splitting

YouTube full album videos are great; one URL, entire album. But I want individual tracks, not a 56-minute mp3.

Many albums have chapter markers in the video. yt-dlp can split by chapters:

'split_chapters': True,

But this created .webm files, not mp3s. The FFmpegExtractAudio postprocessor wasn’t converting them.

Why? Postprocessor order. yt-dlp splits the chapters first, but the audio extraction only runs on the full download, not the split files.

Solution: Manual conversion after download

def convert_webm_to_mp3(output_dir):
    webm_files = list(Path(output_dir).glob("*.webm"))
    
    for webm_file in webm_files:
        mp3_file = webm_file.with_suffix('.mp3')
        subprocess.run([
            'ffmpeg', '-i', str(webm_file),
            '-b:a', '192k', '-y', str(mp3_file)
        ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
        webm_file.unlink()

After all downloads finish, find every .webm, convert it to mp3, delete the original.

Bonus cleanup: Chapter splitting also creates the full album file. Don’t need it if I have all the tracks:

chapter_files = list(Path(output_dir).glob("* - 01 - *.mp3"))
if chapter_files:
    for chapter_file in chapter_files:
        base_name = str(chapter_file.name).split(' - 01 - ')[0] + '.mp3'
        full_album_file = Path(output_dir) / base_name
        if full_album_file.exists():
            full_album_file.unlink()

Detect if chapters were split (check for ” - 01 - ” pattern), then remove the full album file.

!! Note

After playing with this some, I found the chapter splits are not perfect. It came down the uploaders not perfectly placing chapters .. so I don’t have a solution yet. In some cases I am not splitting if the chapters are poorly placed and just accepting the longer .mp3 file.


Problem 4: Handling Metadata

CMUS organizes itself based on an Audio files metadata.

The second script reads folder structure and applies ID3 tags. Initial implementation was rigid:

if len(parts) < 4:
    return None  # Skip file

This required genre/artist/album/track.mp3 structure. But sometimes I have misc tracks where I only care about the artist, not the album. Rigid structure meant those files got skipped.

Solution: Flexible depth handling

def extract_metadata_from_path(filepath, staging_root):
    relative = filepath.relative_to(staging_root)
    parts = relative.parts
    
    if len(parts) < 2:
        return None  # Need at least artist/file.mp3
    
    metadata = {'title': clean_title(parts[-1])}
    
    if len(parts) == 2:
        # artist/file.mp3
        metadata['artist'] = parts[0]
    elif len(parts) == 3:
        # genre/artist/file.mp3
        metadata['genre'] = parts[0]
        metadata['artist'] = parts[1]
    elif len(parts) >= 4:
        # genre/artist/album/file.mp3
        metadata['genre'] = parts[0]
        metadata['artist'] = parts[1]
        metadata['album'] = parts[2]
    
    return metadata

A little verbose … but now it adapts to whatever structure I give it. Full albums get full metadata, loose tracks get artist only.

Title cleaning:

YouTube titles are verbose: "Linkin Park - Hybrid Theory [Full Album] - 01 - Papercut.mp3"

Strip everything before the track number:

def clean_title(filename):
    name = filename.replace('.mp3', '')
    match = re.search(r'\d{2}\s*-\s*(.+)$', name)
    if match:
        return match.group(1).strip()
    return name

Result: "01 - Papercut.mp3" with title tag = “Papercut”


Problem 5: Track Numbers

cmus sorts by track number. Without proper track tags, albums play in alphabetical order instead of track order.

Solution: Extract from filename

def extract_track_number(filename):
    match = re.search(r'\b(\d{2})\s*-', filename)
    return match.group(1) if match else None

Looks for the “NN - ” pattern in filenames. If found, add it to metadata:

track_number = extract_track_number(filename)
if track_number:
    metadata['tracknumber'] = track_number

Now albums play in correct order.


After several (many) hours everything was going strong then suddenly …

ERROR: Sign in to confirm you're not a bot

WHAT! I thought I solved this already …

After some more googling … I learnt that YouTube rotates cookies frequently on open browser tabs as a security measure. Cookies exported from regular browsing sessions expire within hours.

The official solution (from yt-dlp docs): Export cookies from a private/incognito window that YouTube never rotates.

Process:

  1. Open incognito window
  2. Log into YouTube
  3. Navigate to https://www.youtube.com/robots.txt (same tab!)
  4. Export cookies with browser extension
  5. Close incognito window immediately

But even with properly exported cookies, using them with certain player clients triggered JavaScript signature challenges that required additional setup.

So… we wrap it in a try except and resort to cookies only when a direct DL fails first. Most videos don’t need authentication. Try downloading without cookies first, only use them on failure.

def download_audio(url, output_dir):
    # Try without cookies first
    options_no_cookies = {
        'format': 'bestaudio/best',
        'postprocessors': [{'key': 'FFmpegSplitChapters'}],
        'split_chapters': True,
        'sleep_interval': 8,
        'max_sleep_interval': 15,
    }
    
    try:
        with yt_dlp.YoutubeDL(options_no_cookies) as ydl:
            ydl.download([url])
            return
    except Exception as e:
        # Only use cookies if authentication required
        if 'Sign in' in str(e) or 'age' in str(e).lower():
            cookies_file = Path.cwd() / 'cookies.txt'
            options_with_cookies = {
                **options_no_cookies,
                'cookiefile': str(cookies_file),
            }
            with yt_dlp.YoutubeDL(options_with_cookies) as ydl:
                ydl.download([url])
        else:
            raise

This approach:

  • Avoids cookie rotation issues for most videos
  • Only authenticates when necessary
  • Bypasses JavaScript signature challenges
  • Relies on yt-dlp’s built-in rate limiting (8-15 second sleeps)

Additional learning: Be selective with playlist URLs. yt_dlp can handle them fine, but I had some “mix” playlists in my first run that expanded to 250+ songs and got me bot flagged.


The Final System

yt_ripper.py (condensed):

def download_audio(url, output_dir):
    # Try without cookies first (works for most videos)
    options_no_cookies = {
        'format': 'bestaudio/best',
        'postprocessors': [{'key': 'FFmpegSplitChapters'}],
        'split_chapters': True,     # Don't always use this.
        'outtmpl': {
            'default': f'{output_dir}/%(title)s.%(ext)s',
            'chapter': f'{output_dir}/%(title)s - %(section_number)02d - %(section_title)s.%(ext)s'
        },
        'sleep_interval': 8,
        'max_sleep_interval': 15,
    }
    
    try:
        with yt_dlp.YoutubeDL(options_no_cookies) as ydl:
            ydl.download([url])
            return
    except Exception as e:
        # Fallback to cookies only if needed
        if 'Sign in' in str(e):
            cookies_file = Path.cwd() / 'cookies.txt'
            options_with_cookies = {
                **options_no_cookies,
                'cookiefile': str(cookies_file),
            }
            with yt_dlp.YoutubeDL(options_with_cookies) as ydl:
                ydl.download([url])
        else:
            raise

# After all downloads:
convert_webm_to_mp3(output_dir)

metadata.py (condensed):

def process_file(filepath):
    metadata = extract_metadata_from_path(filepath, staging_dir)
    apply_metadata(filepath, metadata)
    move_to_library(filepath, metadata, library_dir)

Workflow:

  1. Add URLs to urls.txt
  2. make rip - Downloads everything to ~/Downloads/music/
  3. Manually organize into genre/artist/album/ folders
  4. make process - Tags and moves to ~/Music/
  5. :update-cache in cmus

Reflections

What worked:

  • Two-script separation. Download and metadata are separate concerns.
  • Manual organization step. Gives me control over structure before committing.
  • Makefile automation. Simple commands with make.
  • Terminal-based workflow. Everything runs from command line.
  • No-cookies-first approach. Avoids authentication complexity for public videos.
  • yt-dlp’s built-in rate limiting. 8-15 second sleeps prevent flagging.

What I learned:

  • YouTube’s bot detection is aggressive. Start simple (no cookies), only authenticate when required.
  • Cookie rotation. Export from incognito at /robots.txt, close window immediately.
  • yt-dlp is feature-rich but YouTube constantly changes APIs. Need to keep the lib updated.
  • Playlist URLs are dangerous. 250+ videos from one URL = instant rate limiting.
  • Flexible metadata extraction is better than rigid structure requirements.
  • ffmpeg is very powerful and I would like to learn more of it.

The result::

I now own my music library. And I can give a big middle finger to spotify and youtube … No more monthly fees, licensing bullshit, no songs disappearing, and no 1 minute ad breaks.

Granted, this is a terminal music player that exists only on my desktop. So there are extra steps needed to add a level of convenience (shared library, media player on graphene phone). But that’s not what this blog post is about.

While this was a bit of a time investment (~12 hours of debugging) … for me this is a far better experience than the one that is offered by youtube and spotify.

Plus, cmus is fast, keyboard-driven, and it means one less electron app eating up my RAM.

Ideas that are tabled for the moment:

  • Better error recovery (some videos still fail)
  • Automatic genre detection (maybe)
  • Integration with MusicBrainz for better metadata

Works for now! I already have 100 tracks organized, tagged, and playing in cmus. Spotify can keep its algorithmic recommendations. I’ll keep my files.


The tools: yt-dlp, ffmpeg, mutagen, Python, cmus, and an unhealthy disrespect for cloud services.