7

I managed to write the following script:

#!/bin/bash

#files list
file1=/tmp/1wall_long.txt
file2=/tmp/1wall_test1.txt
file3=/tmp/1wall_test2.txt
file4=/tmp/1wall_test3.txt
file5=/tmp/3mt_long.txt
file6=/tmp/3mt_OpenSpace_test1.txt
file7=/tmp/3mt_OpenSpace_test2.txt
file8=/tmp/3mt_OpenSpace_test3.txt
file9=/tmp/3rooms_test1.txt
file10=/tmp/3rooms_test2.txt
file11=/tmp/3rooms_test3.txt
file12=/tmp/20mt_OpenSpace_test1.txt
file13=/tmp/20mt_OpenSpace_test2.txt
file14=/tmp/20mt_OpenSpace_test3.txt

#script for 1wall_long file
if [ ! -e "$file1" ]; then #check if the file exist
    echo "File 1wall_long.txt does not exist" #if not exist print echo output
else
    sed -i -e 's/- /-/g' $file1 #remove space on the first 10 values
    awk '{print $7}' $file1 > /tmp/1wall_long_S.txt #print the column number 7 and copy the output in a file
    rm $file1 #remove old file
fi

The script is repeated for all files described in the variable (basically I have the same script repeated 14 times with different variables) Is there a better way to do it and what is the best practice in these situations ?

  • 1
    This should probably be migrated to codereview.stackexchange.com – Ethan Bierlein Oct 12 '15 at 12:17
  • 1
    @EthanBierlein no, it shouldn't. We never migrate o topic questions unless the OP requests it. Bash scripting is very much on topic here, so this question is welcome to stay. – terdon Oct 13 '15 at 9:39
6

Personally, I would avoid hardcoding the file names. That is rarely a good idea and it is usually better to have the option of passing target files as arguments. Additionally, you are modifying the file in place and then deleting the original. That's not efficient, just modify the file on the fly and print the 7th column without having to write it to disk. For example:

#!/usr/bin/env bash

## Iterate over the file names given
for file in "$@"; do
    ## Get the output file's name. The ${file%.*} is
    ## the file's anme without its extension.
    outfile="${file%.*}"_S.txt
    ## If the file exists
    if [ -e "$file" ]; then
    ## remove the spaces and print the 7th column
    sed 's/- /-/g' "$file" | awk '{print $7}' > "$outfile" &&
        ## Delete the original but only if the step
        ## above was successful (that's what the && does)/
        rm "$file" 
    else
    ## If the file doesn't exist, print an error message
    echo "The file $file does not exist!"
    fi
done

Then, you can run the script like this:

foo.sh /tmp/1wall_long.txt /tmp/1wall_test1.txt /tmp/1wall_test2.txt /tmp/1wall_test3.txt /tmp/20mt_OpenSpace_test1.txt /tmp/20mt_OpenSpace_test2.txt /tmp/20mt_OpenSpace_test3.txt /tmp/3mt_long.txt /tmp/3mt_OpenSpace_test1.txt /tmp/3mt_OpenSpace_test2.txt /tmp/3mt_OpenSpace_test3.txt /tmp/3rooms_test1.txt /tmp/3rooms_test2.txt /tmp/3rooms_test3.txt 

If you do want to have the names hard coded, just use an array as suggested by @choroba:

#!/usr/bin/env bash

files=(/tmp/1wall_long.txt /tmp/1wall_test1.txt /tmp/1wall_test2.txt /tmp/1wall_test3.txt /tmp/20mt_OpenSpace_test1.txt /tmp/20mt_OpenSpace_test2.txt /tmp/20mt_OpenSpace_test3.txt /tmp/3mt_long.txt /tmp/3mt_OpenSpace_test1.txt /tmp/3mt_OpenSpace_test2.txt /tmp/3mt_OpenSpace_test3.txt /tmp/3rooms_test1.txt /tmp/3rooms_test2.txt /tmp/3rooms_test3.txt )


## Iterate over the file names given
for file in "${files[@]}"; do
    ## Get the output file's name. The ${file%.*} is
    ## the file's anme without its extension.
    outfile="${file%.*}"_S.txt
    ## If the file exists
    if [ -e "$file" ]; then
    ## remove the spaces and print the 7th column
    sed 's/- /-/g' "$file" | awk '{print $7}' > "$outfile" &&
        ## Delete the original but only if the step
        ## above was successful (that's what the && does)/
        rm "$file" 
    else
    ## If the file doesn't exist, print an error message
    echo "The file $file does not exist!"
    fi
done
  • Hi, thank for your reply. It sounds similar to reply given by choroba. Is it right? – Federi Oct 12 '15 at 11:58
  • The second one is, yes. The main difference is that I avoid creating a file only to delete it later. Your sed -i just adds extra work. Since you will delete the file anyway, there's no reason to use -i which will only add a needless write operation to the disk. My first suggestion is different: instead of writing the file names in the script, pass them as arguments. That way, you can easily change the files it runs on. – terdon Oct 12 '15 at 12:00
  • Sorry, last one. What is the meaning of % in "${file%.*}" – Federi Oct 12 '15 at 12:10
  • 1
    @Federi, ${var%pattern} returns the value of $var with pattern removed from the end. So ${file%.*} will remove the last dot and everything after that. (Note: foo.bar.baz will become foo.bar. If you use %%, it becomes foo.) – nyuszika7h Oct 13 '15 at 9:37
8

loopless

first use a function

function sevenc
{


if [ ! -e "$1" ]; then #check if the file exist
    echo "File $1 does not exist" #if not exist print echo output
else
    sed -i -e 's/- /-/g' "$1" #remove space on the first 10 values
    awk '{print $7}' "$1" > /tmp/$(basename $1.txt)_S.txt #print the column number 7 and copy the output in a file
    rm "$1"  #remove old file
fi
}
  • when the shell recognize a function, it will pass argument (if any to $1 $2 ... and so on).
  • by the way

's/- /-/g' "$1" #remove space on the first 10 values

NO, it turn all space- to - on the line, be there 1, 4, 10 or 255.

then no need for more var

sevenc /tmp/1wall_long.txt
sevenc /tmp/1wall_test1.txt
sevenc /tmp/1wall_test2.txt
sevenc /tmp/1wall_test3.txt
sevenc /tmp/3mt_long.txt
sevenc /tmp/3mt_OpenSpace_test1.txt
sevenc /tmp/3mt_OpenSpace_test2.txt
sevenc /tmp/3mt_OpenSpace_test3.txt
sevenc /tmp/3rooms_test1.txt
sevenc /tmp/3rooms_test2.txt
sevenc /tmp/3rooms_test3.txt
sevenc /tmp/20mt_OpenSpace_test1.txt
sevenc /tmp/20mt_OpenSpace_test2.txt
sevenc /tmp/20mt_OpenSpace_test3.txt

(provided you have no more use of fileXX var).

loopless (sol. 2)

should you want to pass more argument, and using Terdon's optimisation try

function eight
{

file=$1
destdir=${2-/tmp} # use second arg if defined, else /tmp
exten=${3-S} 

if [ ! -e "$file" ]; then #check if the file exist
    echo "File $file does not exist" #if not exist print echo output
else
    sed  -e 's/- /-/g' "$file" \
    awk '{print $7}' "$1" > /"$destdir"/$(basename $1.txt)_"$exten".txt #print the column number 7 and copy the output in a file
    rm "$file"  #remove old file
fi
}

to be called with

eight /tmp/1wall_test3.txt /my/projec/dir T ## will use /my/project/dir as dit, T as extension
eight /tmp/1wall_test1.txt /my/project ## will use /my/project as dir
eignt /tmp/1wall_test2.txt ## will use default value

those function can be defined in .bashrc and be use interactively.

with loop

while read f
do
if [ ! -e "$f" ]; then #check if the file exist
    echo "File $1 does not exist" #if not exist print echo output
else
    sed -i -e 's/- /-/g' "$f" #remove space on the first 10 values
    awk '{print $7}' "$f" > "/tmp/$(basename $f .txt)_S.txt" #print the column number 7 and copy the output in a file
    rm "$f"  #remove old file
fi
done <<EOF
/tmp/1wall_long.txt
/tmp/1wall_test1.txt
/tmp/1wall_test2.txt
/tmp/1wall_test3.txt
/tmp/3mt_long.txt
/tmp/3mt_OpenSpace_test1.txt
/tmp/3mt_OpenSpace_test2.txt
/tmp/3mt_OpenSpace_test3.txt
/tmp/3rooms_test1.txt
/tmp/3rooms_test2.txt
/tmp/3rooms_test3.txt
/tmp/20mt_OpenSpace_test1.txt
/tmp/20mt_OpenSpace_test2.txt
/tmp/20mt_OpenSpace_test3.txt
EOF
  • Thanks for your reply. I have some question "looples" option: 1.do I have to define the variable $1? 2.The name sevenc I guess is an example and I can use which name I prefer 3.How can I change the basename for each output? – Federi Oct 12 '15 at 11:52
  • see edit for sol 2. – Archemar Oct 12 '15 at 12:20
6

You can use an array of files and loop over it with for:

#!/bin/bash

files=(/tmp/1wall_long.txt
       /tmp/1wall_test1.txt
       /tmp/1wall_test2.txt
       /tmp/1wall_test3.txt
       /tmp/3mt_long.txt
       /tmp/3mt_OpenSpace_test1.txt
       /tmp/3mt_OpenSpace_test2.txt
       /tmp/3mt_OpenSpace_test3.txt
       /tmp/3rooms_test1.txt
       /tmp/3rooms_test2.txt
       /tmp/3rooms_test3.txt
       /tmp/20mt_OpenSpace_test1.txt
       /tmp/20mt_OpenSpace_test2.txt
       /tmp/20mt_OpenSpace_test3.txt )

for file in "${files[@]}" ; do
    if [ ! -e "$file" ]; then
        echo "File $file does not exist"
    else
        sed -i -e 's/- /-/g' "$file"
        # Use parameter expansion to create the new file name.
        newfile=${file%.txt}_S.txt
        awk '{print $7}' "$file" > "$newfile"
        rm "$file"
    fi
done
  • 1
    That sounds more familiar for my low level of knowledge :) Thank You Just a quick question, what @ means in ${files[@]} ? – Federi Oct 12 '15 at 11:55
  • 2
    ${files[@]} gives you the entire array, since ${files} only gives you the first element. – rexkogitans Oct 12 '15 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.