Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to iterate through the subdirectories of a directory and take two of the files, as arguments of an awk script. The script will compare the two files and generate other files.

I've this. But I need awk script to take as an argument files. ".*1.txt" and ".*2.txt"

for i in words/*/*1.txt words/*/*2.txt
do 
awk -f corpus_vs_flexion.awk "$i"
done

Something like:

awk -f corpus_vs_flexion.awk .*1.txt .*2.txt
# Taking them from each subdirectory in words/* 

Directory words/
               subdirectory Peter/
                                 whatever.txt
                                 whatever1.txt
                                 whatever.txt
                                 whatever.txt
                                 whatever2.txt
               subdirectory Blas/
                                 whatever1.txt
                                 whatever.txt
                                 whatever.txt
                                 whatever.txt
                                 whatever2.txt
               ........./
                                 .....
                                 ..
For each subdirectory loop: awk -f corpus_vs_flexion.awk whatever1.txt whatever2.txt
share|improve this question
    
So what are the pairs? words/Peter/whatever1.txt and words/Blas/whatever2.txt? words/Peter/whatever1.txt and words/Peter/whatever2.txt? Are the pairs always in the same directory? – terdon Feb 29 at 9:13
    
words/Peter/whatever1.txt and words/Peter/whatever2.txt are the pairs always in the same directory. For words/Peter/ awk -f corpus_vs_flexion.awk whatever1.txt whatever2.txt for words/Blas/ awk -f corpus_vs_flexion.awk whatever1.txt whatever2.txt – Firefly Feb 29 at 9:17
    
OK, please edit your question and clarify that there are always pairs in the directories. Your multiple /same/path/whatever.txt are confusing. Also, will the files always end in 1.txt and 2.txt? Or can we have 3.txt and 4.txt or anything else? – terdon Feb 29 at 9:23
    
If what you found solved your problem (and it looks like it does), add it as an answer and accept it if nobody comes up with another answer that you like better - it's perfectly acceptable to answer your own question on this site. Answers do not belong embedded in questions. – cas Feb 29 at 9:47
    
@cas You're right!! Done! – Firefly Feb 29 at 9:52
up vote 4 down vote accepted

Ok, with file names occurring in matching pairs, you could use the following:

for f in words/*/*1.txt ; do awk -f corpus_vs_flexion.awk "$f" "${f%1.txt}2.txt" ; done

The "${f%1.txt}2.txt" phrase says "use the filename of "$f" but remove the ending 1.txt and add ending 2.txt instead".

share|improve this answer
    
Files to process in each subdirectory. End in "1.txt" and "2.txt". Example: "blabla1.txt" or "blublu2.txt", plus there may be more files with other different names. Then I have to use a regular expression to process – Firefly Feb 29 at 8:45

I have been reading a bash book, and I found what I needed!

typ1_files=(words/*/*1.txt)
typ2_files=(words/*/*2.txt)

for ((i=0;i<=${#typ1_files[@]};i++)); do
   awk -f corpus_vs_flexion.awk "${typ1_files[i]}" "${typ2_files[i]}"
done
share|improve this answer
1  
i'm unsure if your answer or @Ralph's answer is better. Both assume that there will always be a matching 2.txt for each 1.txt. Your answer has the potential to get out of sync (with elements from typ1_files not corresponding to matching elements from typ2_files - i.e. referring to unrelated 2.txt files in another directory) and messing up the entire run from the point that synchronisation failed. Ralph's answer will fail only for the specific file(s) where a matching 2.txt doesn't exist, a far less disastrous failure mode - i've changed my mind, Ralph's answer is better as it's safer. – cas Feb 29 at 9:58
    
A simple safety net would be to add a line like [[ ${#typ1_files[@]} = ${#typ2_files[@]} ]] || exit so it will exit the program if the number of files in each array isn't the same. – terdon Feb 29 at 10:00
    
yep. OTOH, that would also fail in the same way if there were 2.txt files without matching 1.txt files, so that the count remained equal. probably not an issue in real life with this set of files, but it's always worthwhile being paranoid and thinking about what could go wrong and programming defensively to avoid it. – cas Feb 29 at 10:06

If you can be sure your file search is exhaustive and not matching other files, simplify by losing the for loop and just use a command substitution with find.

awk -f corpus_vs_flexion.awk $(find /path/to/your/dir -name "*.txt" -type f | tr '\n' ' ')
                                    ^^^^^^^^^^^^^^^^^ Put your dir here
share|improve this answer
    
I have to process, whatever 1.txt and whatever2.txt at the same time, in each loop. awk -f corpus_vs_flexion.awk whatever1.txt whatever2.txt for each subdirectory. – Firefly Feb 29 at 8:58

I like Ralph's answer. This might also work (untested)

find. -name '*[12].txt' -print0 | xargs -0 -n 2 awk '...'
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.