-2

I have multiple .txt files in my folder " Subj1 " arranged in following format

  regional_vol_GM_atlas1.txt
  regional_vol_GM_atlas1_prop.txt
  ....
  regional_vol_GM_atlas152.txt

each file consists of data in following format

667869 667869
580083 580083
316133 316133
11398.1 11398.1

i would like to export the data along with associated file name in text file to csv file.

I have written a code for that

#!/bin/bash

for x in regional_vol*.txt; do

   /bin/echo -n -e "$x\t"

done

paste regional_vol*.txt >> final_table.csv

which pastes the data in each file into csv file,the sample output image looks like this ,

Output image

but in the output i would like to obtain associated file name as a header for the data how could i change my code to obtain such output ?

This is the sample example of output i expect

Sample output

  • I don't understand your script. In the loop, you simply echo the filenames to STDOUT with a tab at the end, it really doesn't do anything! The paste command should be enough. – maulinglawns Oct 18 '16 at 12:41
  • @maulinglawns : exactly paste should be enough to copy the data in text file and paste it in CSV, but i would like to use the file name of each file as a header, right now it only echos file names how can i change it ? – DevanDevak Oct 18 '16 at 12:44
  • By using redirection perhaps? – maulinglawns Oct 18 '16 at 12:48
  • @maulinglawns , tried with redirection, none of it works – DevanDevak Oct 18 '16 at 13:38
0

A simple but not elegant solution will be add the file name in the first line of each file:

for x in $(ls regional_vol*.txt | sort -n); do
   echo ${x} > tmp-${x}
   cat ${x} >> tmp-${x}
done

paste tmp-regional_vol*.txt > final_table.csv
rm tmp-regional_vol*.txt
  • It fails unfortunately, i could see complete mismatch between headers and associated data in files – DevanDevak Oct 18 '16 at 12:50
  • It works fine here, can you send a screenshot or the final_table.csv? – a0f3dd13 Oct 18 '16 at 12:52
  • Hello ,unfortunately, i dont have enough points to send you that – DevanDevak Oct 18 '16 at 12:56
  • This is the screen shot imgur.com/YiUnoEY – DevanDevak Oct 18 '16 at 13:00
  • I have sent you a screen shot – DevanDevak Oct 18 '16 at 13:38
1

I think what you want is a simple redirection to the echo command, and most likely another echo to print a newline. Something like this:

files="regional_vol*.txt"
dst=final_table.csv
# remove the hash sign here to truncate the destination before writing
# > $dst    
for x in $files; do
   /bin/echo -n -e "$x\t" >> $dst
done
echo >> $dst
paste $files >> $dst

or with printf instead of the loop:

printf "%s\t" $files >> $dst
echo >> $dst
paste $files >> $dst

Both of those will print an extra tab at the end of the header line, but that shouldn't matter much.

  • it fails again, i need the header name on top of associated data, i have updated my question again please check – DevanDevak Oct 18 '16 at 12:52
  • I have updated my question again ,the out from your code looks like this imgur.com/NDiMRA1 – DevanDevak Oct 18 '16 at 13:06
0

(I am assuming that you have rotated the file, since each file is a column in the pictures.)

In your script, each file is added as a row in the CSV. Each row in the CSV should be a row in the spreadsheet.

Use SED and AWK to manipulate the file contents and build a string, "filename,field1,field2,field3,...". Echo that string to append to your CSV.

Then do the same rotation you've done on the pictured spreadsheet when you open it. Or keep it in the usual, "each row is an entry", format. Actually this is recommended, because the filename should be a field, not a header. In other words, the field with the header name of filename should contain the filename for each entry in your CSV.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.