Download playlist with youtube-dl. Saving all data including the archive file into the uploader's folder for each video

Question

This can't be done just with arguments as it says here: Set download-archive path with args

I want to make two similar scripts. One that downloads the videos and archive file in the uploader's folder for each video. And another script that also downloads the metadata.

I want to check the archive before downloading.

I want to be able to use the function multiple times in parallel putting them on the background.

The only way I can think of getting the uploader is to download the metadata with the output %(uploader)s.

Then create a folder with the uploader's name, remove the file, and download again the files with the correct name from the created folder.

I have a few functions defined already that may help with this.

This is what I've tried but it isn't working

# Video Playlist saving archive file to uploader's folder
ytp() {
  # Dl opts=(
    ${opts[@]}
    --skownload metadata
    --skip-download
    --write-info-json
  )
  # Get ten random characters
  local rand=$( cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 10 | head -n 1 )
  LOG_NAME=$( get_log_name "$(${paste[@]})" )

  $yt_dl ${opts[@]} --output "%(uploader|Anon-%(id)s)s.%(ext)s" "$(${paste[@]})" >> "/tmp/$LOG_NAME" 2>&1

  for f in *.info.json; do
    local name="${f%.*}"
    if [[ $name == Anon* ]]; then
      name="Anon"
    fi
    mkdir -p "$name"
    rm "$f"
    cd "$name" || exit

    # Download media
    ytv

    cd .. || exit
  done
}

Answer 1

If you use a sh/bash/zsh-type shell you could do something like this

touch temp_archive.txt; yt-dlp --write-info-json --download-archive temp_archive.txt -o "%(uploader)s/%(title)s-[%(id)s].%(ext)s" --exec after_video:'cat temp_archive.txt >> "%(uploader)s/archive.txt"; printf "" > temp_archive.txt' "URL"

In a function:

opts=(
  --continue
  --no-overwrites
  --no-post-overwrites
  --verbose
  --restrict-filenames
  --retry-sleep fragment:exp=2:64
)

get_log_name() {
  TIMESTAMP=$( date +%y%m%d%H%M%S )
  NAME=${1//[^[:alnum:].-]/_}
  echo "yt-dlp_${TIMESTAMP}_${NAME}.log"
}

deal_with_long_filename_playlist() {
  LOG_NAME=$( get_log_name "$(${paste[@]})" )
  if ! $yt_dl ${opts[@]} --output "%(uploader|Anon)s/%(upload_date>%Y-%m-%d|)s%(upload_date& |)s%(title)s-%(id)s.%(ext)s" "$(${paste[@]})" >> "/tmp/$LOG_NAME" 2>&1; then
    $yt_dl ${opts[@]} --output "%(uploader|Anon)s/%(upload_date>%Y-%m-%d|)s%(upload_date& |)s%(webpage_url_domain)s-%(id)s.%(ext)s" "$(${paste[@]})" >> "/tmp/$LOG_NAME" 2>&1
  fi
  rm temp_archive.txt
}

# Video Playlist saving archive file to uploader's folder
ytvp() {
  local opts=(
    ${opts[@]}
    --format '(bv+(wa[abr>=64]/ba))/b'
    --format-sort res:720,tbr~2000
    --download-archive temp_archive.txt 
    --exec after_video:'cat temp_archive.txt >> "%(uploader)s/archive.txt"; printf "" > temp_archive.txt'
  )
  deal_with_long_filename_playlist &
}

# Video playlist with metadata
ytvpm() {
  local opts=(
    ${opts[@]}
    --format '(bv+(wa[abr>=64]/ba))/b'
    --format-sort res:720,tbr~2000
    --download-archive temp_archive.txt 
    --exec after_video:'cat temp_archive.txt >> "%(uploader)s/archive.txt"; printf "" > temp_archive.txt'
    --write-info-json
    --write-comments
    --write-thumbnail
  )
  deal_with_long_filename_playlist &
}