2

I have two huge files and number of lines in both of the file is not same.

file1

61346877
41724134
85406965
59647779
25199749
86213
45417131
41905714
19415458
1828594
56543876
70603415

File2

212
1231
324234
213

Desired Output

61346877,212
41724134,1231
85406965,324234
59647779,213
25199749,212
86213,1231
45417131,324234
41905714,213
19415458,212
1828594,1231
56543876,324234
70603415,213
Improve this question

3 Answers 3

Reset to default
4

bash:

size1=$( wc -l < file1 )
size2=$( wc -l < file2 )
i=0
while (( i < size1 )); do
    cat file2
    (( i += size2 ))
done | paste -d, file1 -  | head -n $size1

I pipe into head in case the size of file1 is not an even multiple of file2

output

61346877,212
41724134,1231
85406965,324234
59647779,213
25199749,212
86213,1231
45417131,324234
41905714,213
19415458,212
1828594,1231
56543876,324234
70603415,213

A 3-line awk program

awk -v OFS=, '
    # read the smaller file into memory
    NR == FNR {size2++; file2[FNR] = $0; next} 

    # store the last line of the array as the zero-th element
    FNR == 1 && NR > 1 {file2[0] = file2[size2]} 

    # print the current line of file1 and the corresponding file2 line
    {print $0, file2[FNR % size2]}
' file2 file1
Improve this answer
8
  • You can join two last awk actions (lines) into one: awk -v OFS=, 'NR==FNR{size2++; file2[FNR]=$0; next}{file2[0]=file2[size2]; print $0, file2[FNR%size2]}' file2 file1
    – jimmij
    Aug 11, 2015 at 19:08
  • The files are "huge", so it might be better to use awk's getline f2 <file2 to read both files in parallel. If you can get awk to start from the beginning of a file again, when it hits the end...
    – Peter Cordes
    Aug 11, 2015 at 19:17
  • @jimmij, sure, but you don't really need to do that assignment for every line in file1
    – glenn jackman
    Aug 12, 2015 at 0:33
  • @PeterCordes: define "huge": is file2 measured in GB?
    – glenn jackman
    Aug 12, 2015 at 0:34
  • No clue, have to ask @anurag, since he's the one that wrote that his files are "huge". But yes, it's basically fine unless it start paging out your GUI desktop. And potentially faster to sequential-read one file and then the other, rather than generating seeks back and forth.
    – Peter Cordes
    Aug 12, 2015 at 0:44
2

What you have there is file2 repeating once all the lines in the file2 have been read. Here's an awk/sed solution that takes number of lines in file2 as variable, and prints lines in file2 according to counter, which is being reset each time we go over the total number of lines in the file

$ awk -v file2lines=$(wc -l file2 | cut -f1 -d' ') 'BEGIN{count=1} {if(count > file2lines) count=1; printf $1",";system("sed -n -e "count"p file2");count++}' file1

61346877,212
41724134,1231
85406965,324234
59647779,213
25199749,212
86213,1231
45417131,324234
41905714,213
19415458,212
1828594,1231
56543876,324234
70603415,213
Improve this answer
3
  • You're invoking sed once for every line in file1 -- this will be really slow.
    – glenn jackman
    Aug 12, 2015 at 0:35
  • @glennjackman admitedly, if the files are large, that will get slow. With what OP's given, time command reports 0.00user 0.00system 0:00.11elapsed 3%CPU (0avgtext+0avgdata 3496maxresident)k 0inputs+0outputs (0major+2480minor)pagefaults 0swaps
    – Sergiy Kolodyazhnyy
    Aug 12, 2015 at 0:38
  • A little bit complicated, but it works ;)
    – A.B.
    Aug 12, 2015 at 6:40
2

Simple using awk ;)

awk 'FNR==NR{a[i++]=$0; max=i; next} {if ((NR % max) == 0) {i=max-1} else {i=(NR%max) - 1}; printf "%s,%s\n",$0,a[i]}' file2 file1

Example output

61346877,212
41724134,1231
85406965,324234
59647779,213
25199749,212
86213,1231
45417131,324234
41905714,213
19415458,212
1828594,1231
56543876,324234
70603415,213
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .