1

I have a 3D tensor of dimension MxNxD data object. And this data object is decomposed and stored in different files.

Initially, each slice/matrix of dimension MxN (first two dimension) is saved in a separate file. Each line contains one row of the matrix. Elements in a row are separated by space. So in total, there are D such files.

Now, I'd like to transpose the 3D tensor so that the last 2 dimensions are in the same files. For example, I'd like M files that each contain one NxD slice.

Example

file0:

000 010 020 030
100 110 120 130
200 210 220 230

file1:

001 011 021 031
101 111 121 131
201 211 221 231

should be transposed to

out1:

000 010 020 030
001 011 021 031

out2:

100 110 120 130
101 111 121 131

out3:

200 210 220 230
201 211 221 231

My questions are:

  • how to do this efficiently in shell script or Python
  • is there any efficient file format to do this?
  • 1
    Please edit your question and add i) an example of your input files and ii) the output you would like to see from them. We can't help you parse data you are not showing us. – terdon Mar 22 '17 at 11:03
  • Hi @xiaohan Is this the correct example for your question? – Philippos May 4 '17 at 9:56
1

Solution in TXR Lisp:

(let* ((data [mapcar file-get-lines (glob "file*")]))
  (mapdo (op file-put-lines `out@2` @1) (transpose data) (range 1)))

Run:

$ ls
file0  file1  trans.tl
$ txr trans.tl
$ ls
file0  file1  out1  out2  out3  trans.tl
$ cat out1
000 010 020 030
001 011 021 031
$ cat out2
100 110 120 130
101 111 121 131
$ cat out3
200 210 220 230
201 211 221 231

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.