5

I have a folder with 1000 files; all characters before mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp refer to individual's ID (for example NA21117,NA21119,NA21126,..)

NA21117.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp
NA21119.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp
NA21126.mapped.ILLUMINA.bwa.GIH.low_coverage.20121211.bam_dp
NA21127.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp
NA21137.mapped.ILLUMINA.bwa.GIH.low_coverage.20120522.bam_dp
NA21142.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp
NA21143.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp

Each file has only one row:

cat NA21143.mapped.ILLUMINA.bwa.GIH.low_coverage.20130415.bam_dp
1   115258827   10

for each of these files, I want to paste the individual ID to the content of file and get an out put like:

1   115258827   10 NA21143

Is there anyway to do it?

4 Answers 4

4

plain bash

for file in *.bam_dp; do 
    contents=$(< "$file")
    echo "$contents ${file%%.*}" > "$file"
done

for multi-line files, still can be accomplished with plain bash

for file in *.bam_dp; do 
    mapfile -t contents < "$file"
    printf "%s\n" "${contents[@]/%/ ${file%%.*}}" > "$file"
done

notes:

  • the mapfile command reads the file into an array of lines.
  • the ${var/pattern/string} parameter expansion does a search-and-replace on the variable value. (documented in the manual)
    • if pattern starts with % the pattern is anchored at the end of the string. Here, I'm matching the empty pattern at the end of the string.
    • the variable can be an array expansions, in which case the replacement occurs for each array element.

Frankly, this approach is too clever, and I'd go for something more obvious.

2
  • yes this works. I have additional question. I have some other files in another folder which have more than one row! I want to do exactly the sam but paste the individual ID to each row. Do you know how can I do this? Thanks a lot
    – Anna1364
    May 2, 2018 at 20:48
  • In that case, go for Ole's or codeforester's solutions, which will operate on every line in the files. May 2, 2018 at 21:03
2

Use a loop:

#!/bin/bash

shopt -s nullglob
for file in ???????.mapped.*bam_dp; do
  [[ -f "$file" ]] || continue
  id=${file%%.*}              # grab the ID from file name
  sed -i "s/$/ $id/" "$file"  # modify the file in-place
done
3
  • Nice solution overall! It would be better to set shopt -s nullglob than to check for file existence with [[ -f "$file" ]]. May 2, 2018 at 23:54
  • Should be noted that this assumes a 7-character ID. This might not necessarily be the case.
    – Bob
    May 3, 2018 at 3:05
  • @DavidFoerster: Thanks for pointing out nullglob. I modified the answer. [[ -f "$file" ]] is meant for skipping directories, if any. May 3, 2018 at 3:10
2

Remove .* from $ARGV then append \t $ARGV to the file:

perl -i -pe '$ARGV=~s/\..*//; s/$/\t$ARGV/;' NA*

Glenn's solution is most likely faster to run:

perl -i -lpe '$_ .= " " . substr($ARGV,0,index($ARGV,"."))' NA*

though if each file is only a single line, most of the time will be seeking on the drive.

2
  • You might consider $_ .= "\t$ARGV" instead of s/// May 2, 2018 at 19:53
  • 1
    or even perl -i -lpe '$_ .= " " . substr($ARGV,0,index($ARGV,"."))' NA* May 2, 2018 at 19:56
1

awk

This method is compatible with the GNU (Linux) and BSD (Mac) versions of awk.

awk '{ id=FILENAME ; sub(/\..*/,"",id) ; print $0 "\t" id }' *.bam_dp
  • id=FILENAME ; sub(/\..*/,"",id)
    Store the first part of each *.bam_dp filename (everything before the first .) as id.
  • print $0 "\t" id
    Print each file's contents, then a tab character, then the record's id.

This will print a list with lines as in your example:

1   115258827   10 NA21143

The original files will not be modified. You can save this output by, for example, adding > file.txt to the end of the command.

1
  • print $0, id will use the default field separator and is (arguably) nicer to read. May 2, 2018 at 21:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .