Transpose paramter values from a file and Generate parameter files from it

Question

There's a file containing a table with the parameter names as the first line, values in the following lines: How to perform the following using awk, sed or any shell scripting utility:

• generate a config file named fileXX for each line,
• transpose each line column values to rows (lines) in each generated config file,
• in each file concatenate the first line which represent the parameter names to the transposed columns values to appear like parameter name-value pairs.

Source file:

Column1     Column2      Column3  
Row1_Col1   Row1_Col2   Row1_Col3  
Row2_Col1   Row2_Col2   Row2_Col3  
Row3_Col1   Row3_Col2   Row3_Col3  

OUtPut files:

• File1

  Column1=Row1_Col1  
  Column2=Row1_Col2  
  Column2=Row1_Col3  
  

• File2

  Column1=Row2_Col1  
  Column2=Row2_Col2  
  Column2=Row2_Col3  
  

• File3

  Column1=Row3_Col1  
  Column2=Row3_Col2  
  Column2=Row3_Col3  
  

Answer 1

You can use GNU datamash (a new-ish command-line program), which has built-in transpose command with strict input validation.

Example:

$ cat in.txt
Column1     Column2     Column3
Row1_Col1   Row1_Col2   Row1_Col3
Row2_Col1   Row2_Col2   Row2_Col3
Row3_Col1   Row3_Col2   Row3_Col3

$ datamash transpose < in.txt
Column1 Row1_Col1   Row2_Col1   Row3_Col1
Column2 Row1_Col2   Row2_Col2   Row3_Col2
Column3 Row1_Col3   Row2_Col3   Row3_Col3

Then split by columns with cut:

$ datamash transpose < in.txt | cut -f1,3 | tr '\t' '='
Column1=Row2_Col1
Column2=Row2_Col2
Column3=Row2_Col3

To create the multiple files, you can use this:

for i in 2 3 4 ; do
   datamash transpose < in.txt | cut -f1,$i | tr '\t' '=' > file$i.txt
done

GNU Datamash is available here: http://www.gnu.org/s/datamash , and packages are available in several gnu/linux distributions (disclaimer: I'm datamash's developer).

Answer 2

You can do it with awk:

{
  # extract files names form first line
  if (1 == NR) {
    num_fields = NF       
    for(i = 1; i <= num_fields ; i++)
    {
      # get line header
      header[i] = $i
      # create file name
      file[i] = "file-"i
    }
  }
  else 
  {
    # extract data if not first line
    for (i = 1; i <= num_fields; i++)
    {
      print header[i] ":" $i > file[i]
    }          
  }
}

And invoke it with:

awk -f script.awk file_to_process

Answer 3

src=/the/source/file
set -- `head -1 $src`;
col1=$1 ; col2=$2 ; col3=$3; i=0;
grep -v "`head -1 $src`" $src | while read c1 c2 c3
  do
    echo -e "$col1=$c1\n$col2=$c2\n$col3=$c3" > file-$((i=i+1)).out ; 
  done

test

$ cat $src
Column1     Column2      Column3  
Row1_Col1   Row1_Col2   Row1_Col3  
Row2_Col1   Row2_Col2   Row2_Col3  
Row3_Col1   Row3_Col2   Row3_Col3
$ set -- `head -1 $src`; i=0; col1=$1 ; col2=$2 ; col3=$3; i=0; grep -v "`head -1 $src`" $src | while read c1 c2 c3 ; do echo -e "$col1=$c1\n$col2=$c2\n$col3=$c3\n" > file-$((i=i+1)).out ; done
$ cat file-1.out 
Column1=Row1_Col1
Column2=Row1_Col2
Column3=Row1_Col3
$ cat file-2.out 
Column1=Row2_Col1
Column2=Row2_Col2
Column3=Row2_Col3
$ cat file-3.out 
Column1=Row3_Col1
Column2=Row3_Col2
Column3=Row3_Col3
$