0

I'm trying to store a number that is part of a file name as a variable so that I can change in and rename the file for a bunch of files at once with a loop. The file names are trimmed-barcode*.1.fastq.gz where * is a variable two- or three-digit number that I would like to save as num. This will allow me to then multiply $num by the $number variable I have previously defined and to rename each file with the multiplied number (but for now I'm just testing this out with echo). How can Istore the two- or three- digit number from the filename?

for infile in trimmed-barcode*.1.fastq.gz
  do num= #how do I get this number
  num2=$(($num * $number))
  echo "trimmed-barcode${num2}.1.fastq.gz"
  done
2
  • Literally trimmed-barcode at the beginning? Jul 13, 2023 at 20:33
  • Unless you are multiplying by a sufficiently large number, you could encounter name collisions during your process. e.g. your $num is 10 and $number is 20 where a file 200 already exists. How do you plan to handle that scenario?
    – bxm
    Jul 14, 2023 at 4:49

4 Answers 4

1

If you use a bash script you can extract the two digits from the string. For example

var="abcdefg"
echo "${var:3:2}"    # Outputs "de"

Armed with that we can adjust your script like this

#!/bin/bash
for infile in trimmed-barcode??.1.fastq.gz
do
    num="${infile:15:2}"    # Two characters starting at position 15
    result=$(( ${num#0} * number ))
    echo "trimmed-barcode$result.1.fastq.gz"
done

While multiplying $num and $number we have to strip any leading zero from the value so that bash doesn't try to treat it as octal. That's what ${num#0} does.


You've amended your question to ask for two or three digit numbers. This changes the implementation significantly.

  1. You can repeat the loop, but this time matching three digits:

    for infile in trimmed-barcode??.1.fastq.gz
    do
        # Code as above for two digit extracts
    done
    
    for infile in trimmed-barcode???.1.fastq.gz
    do
        # Modify code as above to process three digit values
    done
    
  2. You can change the code to remove the invariant parts, assuming that whatever's left must be the value of interest:

    for infile in trimmed-barcode*.1.fastq.gz
    do
        num=${infile#trimmed-barcode}    # Strip leading text
        num=${num%.1.fastq.gz}           # Strip trailing text
        result=$(( ${num#0} * number ))
        echo "trimmed-barcode$result.1.fastq.gz"
    done
    
0
1
for file in trimmed-barcode*.1.fastq.gz; do
    x=${file#trimmed-barcode}
    num=${x%.1.fastq.gz}
    echo "$file: $num"
done

Or in Bash you could use a regex match to pick the desired part of the filename:

for file in trimmed-barcode??.1.fastq.gz; do
    [[ $file =~ trimmed-barcode(.*)\.1\.fastq\.gz ]]
    num=${BASH_REMATCH[1]};
    echo "$file: $num"
done

Or if you want to trim it, just pick the two characters before the first dot with [[ $file =~ (..)\. ]].

0

To store the two-digit number from the filename as a variable in a loop, you can use file name expansion and string manipulation in bash. Here's an example:

for infile in trimmed-barcode*.1.fastq.gz
do
  num="${infile:14:2}"
  num2=$((num * number))
  echo "trimmed-barcode${num2}.1.fastq.gz"
done

In this code, we use the ${infile:14:2} string expansion to extract the two-digit number from the filename starting at index 14. We then multiply that number (num) by the $number variable and store the result in num2. Finally, we echo the resulting file name with the modified number.

Remember to adjust the index (14 in this case) based on the actual position of the number in your file name.

1
  • I think you're off by one. Consider infile=trimmed-barcode27.1.fastq.gz; echo "${infile:14:2}", which returns e2 rather than 27 Jul 14, 2023 at 8:18
0

The following uses the perl rename utility:

Note: perl rename is also known as file-rename, perl-rename, or prename, depending on your distribution. It is not to be confused with the rename utility from util-linux which has completely different and incompatible capabilities and command-line options. Perl rename allows you to use any arbitrarily complex perl code to rename files, but is most often used to do simple sed-like s/search/replace/ operations on filenames.

First create some filenames to test with:

for i in 11 234 56 789 ; do touch "trimmed-barcode$i.1.fastq.gz" ; done

Then rename them:

export number=2
rename -n 's/^(trimmed-barcode)(\d{2,3})(\.1\.fastq\.gz)$/$1 . $2 * $ENV{number} . $3/e' trimmed*

sample output:

rename(trimmed-barcode11.1.fastq.gz, trimmed-barcode22.1.fastq.gz)
rename(trimmed-barcode234.1.fastq.gz, trimmed-barcode468.1.fastq.gz)
rename(trimmed-barcode56.1.fastq.gz, trimmed-barcode112.1.fastq.gz)
rename(trimmed-barcode789.1.fastq.gz, trimmed-barcode1578.1.fastq.gz)

The -n option makes this a dry-run, so it only shows what it would do. To actually rename the files, either remove the -n or replace it with -v for versbose output.

The $number variable has to be exported to the environment so that the rename script can access it through the %ENV hash. Alternatively, you could assign the variable on the same command line (e.g. number=2 rename -n '...' trimmed* (which has the effect of exporting the variable temporarily for that command) or hard-code it into the script.

To make it work with any number of digits, i.e. one or more, before the .1.fastq.gz, change the \d{2,3} to \d+.

To make it work with any filename prefix before the digits (not just "trimmed-barcode") and any extension after the first ., change it to:

rename -n 's/(.*?)(\d{2,3})(\..*)/$1 . $2 * $ENV{number} . $3/e' trimmed*

The script takes advantage of perl's ability to use a /e regex modifier to execute perl code in the right-hand-side (replacement) portion of the s/// substitution operator. In this case, . is perl's string concatenation operator, so the code concatenates the first capture group ($1) with the second capture group ($2) multiplied by your $number environment variable, and the third capture group ($3).

BTW, perl rename will not overwrite existing files unless you force it to with the -f / --force option.

Filenames that don't match the regular expression on the left-hand-side (search) portion of the substitution operator will not be renamed or modified in any way.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .