1

I'm trying to move away from AppleScripts for workflows at my job, and create something simpler that can run in the background instead. For this task, I am given 35-40 files (5 renditions of 7 or 8 files at different qualities) every night, and I need to extract a part of the filename.

As an example, here's what one (abbreviated) batch of those files might look:

Each file has 5 renditions

    ab_12_345_01_dest_xxxxxxxxxx_640x360_1000.jpg
    ab_12_345_01_dest_xxxxxxxxxx_768x432_3000.jpg
    ab_12_345_01_dest_xxxxxxxxxx_960x540_5000.jpg
    ab_12_345_01_dest_xxxxxxxxxx_1280x720_7000.jpg
    ab_12_345_01_dest_xxxxxxxxxx_1920x1080_9000.jpg

And the files are all named as such (using the highest rendition, I'll get to why in a second):

    ab_12_345_01_dest_xxxxxxxxxx_1920x1080_9000.jpg
    ab_12_345_02_dest_yyyyyyyyyy_1920x1080_9000.jpg
    ab_12_345_03_dest_zzzzzzzzzz_1920x1080_9000.jpg
    ab_12_345_part1_aaaaaaaaaa_1920x1080_9000.jpg
    ab_12_345_part2_bbbbbbbbbb_1920x1080_9000.jpg
    ab_12_345_part3_special_cccccccccc_1920x1080_9000.jpg
    ab_12_345_part4_dddddddddd_1920x1080_9000.jpg
    ab_12_345_04_dest_special_eeeeeeeeee_1920x1080_9000.jpg

So my goal is to use the 9000 part of the filename to grep only the highest rendition of each (it takes the longest to copy over, so if it's there, the rest of the files are there too), and then extract everything up to the second to last _. So far, I've been able to do part one, but not part 2.

When I do this, I am able to get a list of only the highest rendition:

    $ ls | grep 9000
    ab_12_345_01_dest_xxxxxxxxxx_1920x1080_9000.jpg
    ab_12_345_02_dest_yyyyyyyyyy_1920x1080_9000.jpg
    ab_12_345_03_dest_zzzzzzzzzz_1920x1080_9000.jpg
    ab_12_345_part1_aaaaaaaaaa_1920x1080_9000.jpg
    ab_12_345_part2_bbbbbbbbbb_1920x1080_9000.jpg
    ab_12_345_part3_special_cccccccccc_1920x1080_9000.jpg
    ab_12_345_part4_dddddddddd_1920x1080_9000.jpg
    ab_12_345_04_dest_special_eeeeeeeeee_1920x1080_9000.jpg

Then I tried ls | grep 9000 | perl -pe '/^.+(?=_.+_.+)/mg thinking I'd get the following (based on what every online RegEx tester, and specifically, Perl RegEx testers I could find said would work):

    $ ls | grep 9000 | perl -pe '/^.+(?=_.+_.+)/mg`
    ab_12_345_01_dest_xxxxxxxxxx
    ab_12_345_02_dest_yyyyyyyyyy
    ab_12_345_03_dest_zzzzzzzzzz
    ab_12_345_part1_aaaaaaaaaa
    ab_12_345_part2_bbbbbbbbbb
    ab_12_345_part3_special_cccccccccc
    ab_12_345_part4_dddddddddd
    ab_12_345_04_dest_special_eeeeeeeeee

However, I got the same result as if I never even piped to perl. I originally tried implementing this with awk, but the command I was typing was getting pretty length and I figured RegEx might be the way to go. However, I need the positive lookahead in order to get it to stop matching at the second to last _ (rather than a _ counted from the beginning of the string), and awk was preserving the final __ when I set {$NL=$(NL-1)=""; print $0}.

  • 1
    What is that expression supposed to be doing? You're matching something... and then what? You're not doing anything with the match. – muru Oct 18 at 4:25
  • Eventually, it's going to print to a text file on my desktop. I haven't gotten that far yet, I'm building it piece by piece. I need to get the strings to cut down correctly, then I need to sort them so that the 4 "part" titles go to the top of the list, and finally, print that to a text file on my desktop. – Alex Torma Oct 18 at 4:45
  • So... what you want is s/^.+(?=_.+_.+)/$1/? – muru Oct 18 at 4:46
  • @muru's recommendation only gave me the "_1920x1080_9000.jpg" part of the string – Alex Torma Oct 18 at 4:48
  • @AlexTorma well, at least now you know how to remove parts of the string. Now add brackets or use $& accordingly. – muru Oct 18 at 4:53
1

With your perl command, you always print the line because you have -p option. The match part doesn't do anything.

You want -n and print the matching part:

ls -1 *9000.jpg \
| perl -lne 'print $1 if /^(.+)(?=_.+_.+)/'

As filenames may have newlines, you should modify this to read zero-delimited filenames although, in your case that may not be needed:

printf '%s\0' *9000.jpg \
| perl -lne 'INIT{ $/ = "\0"}; print $1 if /^(.+)(?=_.+_.+)/'

Alternatively, read the file names in a for-loop, then you can use shell only paramater expansion:

for f in *9000.jpg; do printf '%s\n' "${f%_*_*}"; done

This may be better suited for your task. (=> "Don't use line-based text-editing tools on filenames." @Kusalananda)

  • And why not perl ... *9000.jpg directly? Give the globlist directly? Either your last one, or some partial_f_name.pl *9000.jpg >9.txt call. I think the usual "Dont use..." rule does not apply here, because the info really is in the filenames this time. Bad luck/design. Nobody is grepping "9000". – rastafile Oct 18 at 9:43
  • perl ... *9000.jpg will read the contents of the files, not the filenames. – pLumo Oct 18 at 10:48
  • so I used ... and an alternative. I know, and I feel that exactly is the problem here. The sooner we get rid of the connection to contents, the easier. Is it not possible to loop through the arguments, which are first just names? In your printf loop you are ready to access file content of $f, but that is not needed here. Insted of every file...every line (*, then <>), it is just every file...only the name as string. You combine it well in that alternative, but what if it surpasses possibilities of $(f...) ? back to start! – rastafile Oct 18 at 11:02
  • I have no idea what you are talking about. – pLumo Oct 18 at 11:06
  • 1
    @pLumo, that first sentence should read "you always print the line regardless of if it matches your regex", because that's what -p does, it prints $_ after running the script. And the match doesn't do anything. Also, instead of /.../; print $1, I'd use print $1 if /.../, because otherwise it'll print something on each line, and $1 can contain values from the previous match (if the current line didn't match). – ilkkachu Oct 20 at 8:07
0

There is no need to pipe from ls to grep to filter your file list you can just

ls *9000.jpg

Also with your grep it will pick out any files that happen to have 9000 elsewhere in the name.

There is no issue with your regex, only with the perl. Use grep and you get what you want

ls *9000.jpg | grep -Po "^.+(?=_.+_.+)"

An alternative way of doing this could be

find . -iname "*9000.jpg" -exec sh -c 'basename ${1%_*_*}' sh {} \;

the find does the same as ls

The expansion ${1%_*_*} strips the characters from the 2nd last _ until the end of the string and basename removes the file path which find includes in its results.

The construct

-exec sh -c `blah blah` sh {} \;

is well worth learning to use with find and @Kusalananda has a good post here

-exec just tells find to do 'blah blah' with its output, \; means do the 'blah blah' with each result individually, sh -c 'put some script in here' is what you want to do with the results and finally sh {} passes the output from find back to the script defined in sh -c

  • So with your adventurous find "one liner": it does the job, or it needs processing? Because if yes, then better work on ls *9000.jpg's output to a file from the start. No matter if it was you who downvoted me: I feel confirmed, see my answer if you haven't yet. – rastafile Oct 18 at 8:12
  • The linked Q is about needing find. Here you need only ls with glob *. – rastafile Oct 18 at 8:16
  • @rastafile I didn't downvote anyone.... – bu5hman Oct 18 at 23:08
  • thanks for answering. I thought there was an additional processing step involved. That is why I insisted on a file, and why I ask you if your solution is finished or not. Sorry. – rastafile Oct 18 at 23:30
  • This was my solution, and somehow it is true: "Start with ls -f *9000.jpg >resol9000.txt, then you can work on these lines like a normal human being." Rest was my confusion. – rastafile Oct 18 at 23:32
-4

"...given 35-40 files every night, and I need to extract a part of the filename."

You say "40 files". But it is the file's names you want. Normally, when you "get a file", one means the contents. 99% percent of the commands work on the contents, not the name. Like you would write cp $badname-goodcontent newname. Or compare mv A B to ln -s A B. Confusing -- suddenly files and their names start to dissolve, semantically.

You need a listing of your daily "batch" to work on from the start. Best would be to have a list produced, and not file names. Do these files serve a (other) purpose at all? (I see it is jpgs -- )

Start with ls -f *9000.jpg >resol9000.txt, then you can work on these lines like a normal human being.

(I dont have a solution, but I think this change of strategy is what you need)

(looks like you know perl well -- use it on that temp file now)

(as you say: you need positive lookahead. So you need a file)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.