-1

I have a long list of file names that all end with .mp4xyz

12334,dogimage.mp4001
12335,dogimage.mp4002
12336,dogimage.mp4003
12347,cats.mp4001
12348,cats.mp4002

I need to convert the output into this format combining all images that share the same name. In this example I am only using dogimage.mp4 and cats.mp4 but there will be thousands of keywords.

a:3:{i:0;s:4:"12334";i:1;s:4:"12335";i:2;s:4:"12336";}
a:2:{i:0;s:4:"12347";i:1;s:4:"12348";}

The strings below represent:

a:3 & a:2 = The total amount of images

i:0 = The image count for that keyword

  • This seems like a task that would be better suited for a freelance developer. – Jesse_b May 24 at 18:51
  • Is i to be derived from the xyz part of the extension, or from the order within the file? Or can these be assumed to be the same? Is the s:4: a fixed string? – steeldriver May 24 at 18:57
  • No i:0 should be mp4001, i:1 should be mp4002, i:2 mp4003 and so on. s:4: is a fixed string yes. – Teddy291 May 24 at 19:01
  • Sorry yes that is correct it should be the xyz part mp4001 being first and so on. Not the order of the files in the list, they may be jumbled. – Teddy291 May 24 at 19:10
  • That very much looks like the output of php's serialize(), so it should be s:5 here not s:4 as those strings are 5 bytes long, not 4. – Stéphane Chazelas May 25 at 10:43
2

gawk:

BEGIN{

    # split by , or .mp4
    FS=",|\\.mp4"

    # sort array by numeric value
    PROCINFO["sorted_in"] = "@val_num_asc"
}

# store count in a, store others as key, value pair in d[keyword]
{ a[$2]++; d[$2][$3]= $1 }

END{
    for( keyword in d ){
        printf( "a:%d:{", a[keyword] )
        for( i in d[keyword] )
            printf( "i:%d;s:4:\"%d\";", i-1, d[keyword][i] )
        printf( "}\n" )
    }
}

update

To keep keyword order:

BEGIN {
    # split by , or .mp4
    FS=",|\\.mp4"

    # sort array by numeric value
    PROCINFO["sorted_in"] = "@val_num_asc"
}

{
    # insert next ordered number into loop_order upon new keyword
    if(!($2 in d))
        loop_order[k++] = $2

    # store count in a, store others as key, value pair in d[$2]
    a[$2]++; d[$2][$3]= $1
}

END{
    for (j = 0; j < length(loop_order); ++j) {
        keyword = loop_order[j]
        printf( "a:%d:{", a[keyword] )
        for( i in d[keyword] )
            printf( "i:%d;s:4:\"%d\";", i-1, d[keyword][i] )
        printf( "}\n" )
    }
}
  • This works however I need them to be in the order of filenames read from our list. – Teddy291 May 25 at 14:37
  • @Teddy291 see update. – dedowsdi May 25 at 23:47
2

That's the output format of php's serialize(), so:

php -r '
  while ($line = fgets(STDIN)) {
    $n = strtok($line, ",");
    $image = strtok(".\n");
    $a[$image][] = $n;
  }
  foreach ($a as $v) echo serialize($v) . "\n";' < file.list

On your input, that gives:

a:3:{i:0;s:5:"12334";i:1;s:5:"12335";i:2;s:5:"12336";}
a:2:{i:0;s:5:"12347";i:1;s:5:"12348";}

(note the s:5 instead of s:4 as those strings are 5 bytes long, not 4).

To sort the list of arrays based on the keyword (lexically), insert of ksort($a); before the foreach loop, then the array for cats will be shown before the array for dogimage because cats comes before dogimage lexically.

Or maybe serialise the whole associative array so you don't lose the information about which array corresponds to which keyword like:

php -r '
  while ($line = fgets(STDIN)) {
    $n = strtok($line, ",");
    $image = strtok(".\n");
    $a[$image][] = $n;
  }
  echo serialize($a) . "\n";' < file.list

Which gives:

a:2:{s:8:"dogimage";a:3:{i:0;s:5:"12334";i:1;s:5:"12335";i:2;s:5:"12336";}s:4:"cats";a:2:{i:0;s:5:"12347";i:1;s:5:"12348";}}
  • This did not work as there are files called "cats2.mp4xyz" these should be separate. – Teddy291 May 25 at 11:23
  • @Teddy291, separate from what? Do you mean they're called cats2.mp4xyz as opposed to 123,cats.mp4xyz? – Stéphane Chazelas May 25 at 11:30
  • cats.mp4xyz & cats2.mp4xyz combined into one. They should bring back two results as they are different but similar keywords. – Teddy291 May 25 at 11:31
  • @Teddy291, they do for me with that code. – Stéphane Chazelas May 25 at 11:31
  • My apologies it does seem to be working but for some reason it's skipping 6 of the images. I am trying to find which are being skipped. – Teddy291 May 25 at 11:40
1

Here's one way to do it - using a hash-of-hashes in Perl:

perl -F, -lne '
  ($k,$i) = split(/\.mp4/, $F[1]); 
  $s{$k}->{$i-1} = $F[0]
  }{
  foreach $k (keys %s) {
    $hr = $s{$k};
    printf "a:%d:{", keys %$hr;
    foreach $i (sort {$a <=> $b} keys %$hr) {
      printf "i:%d;s:4:\"%s\";", $i, $hr->{$i}; 
    }
    printf "}\n";
  }  
' file
a:1:{i:0;s:4:"12334";i:1;s:4:"12335";i:2;s:4:"12336";}
a:0:{i:0;s:4:"12347";i:1;s:4:"12348";}

The "outer" hash %s is keyed on the keyword, while the inner anonymous hashes are keyed on the numeric value of the suffix (minus one) and have values taken from the first comma separated field of the line.

Note that the outer hash is unsorted so the order of output lines is not guaranteed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.