1

It is possible to ignore some of the fields in an regex expression. The issue is that I have an output of a sorted files from an FTP server. The problem is that FTP does not list the year from the files that are 6 months or newer. So for example if I want to sort this 020319 and 100518 and I want to list the latest by date it will sort first 100518, and that's not good.

FTP_FILES_LIST is a file with a bunch of files from an "ls" command from an FTP Site. I use "grep" to get only the files I'm interested in.

A="AT_20_10_REL_ARCA_"

more FTP_FILES_LIST | grep "$A[0-9][0-9][0-9][0-9][0-9][0-9].txt" | sort -k 9

-r-xr-xr-x    1 14       2000     34037013 Jan 17 00:45 AT_20_10_REL_ARCA_011719.txt
-r-xr-xr-x    1 14       2000     34036101 Jan 18 11:13 AT_20_10_REL_ARCA_011819.txt
-r-xr-xr-x    1 14       2000     34036564 Jan 25 01:09 AT_20_10_REL_ARCA_012519.txt
-r-xr-xr-x    1 14       2000     34041306 Feb 03 21:42 AT_20_10_REL_ARCA_020319.txt
-r-xr-xr-x    1 14       2000     34099207 Feb 08 03:15 AT_20_10_REL_ARCA_020819.txt
-r-xr-xr-x    1 14       2000     34099827 Feb 11 02:55 AT_20_10_REL_ARCA_021119.txt
-r-xr-xr-x    1 14       2000     34010091 Oct 05 00:42 AT_20_10_REL_ARCA_100518.txt
-r-xr-xr-x    1 14       2000     34025780 Nov 26 02:55 AT_20_10_REL_ARCA_112618.txt
-r-xr-xr-x    1 14       2000     34037370 Dec 19 22:10 AT_20_10_REL_ARCA_121918.txt

Using the "sed" doesn't sort as it should not sort as it should be. Here is the output:

more FTP_FILES_LIST | grep $A[0-9][0-9][0-9][0-9][0-9][0-9] | sed -E 's/^(..)(..)(..)/\3\1\2/' | sort | sed -E 's/^(..)(..)(..)/\2\3\1/'

-r-xr-xr-x    1 14       2000     34010091 Oct 05 00:42 AT_20_10_REL_ARCA_100518.txt
-r-xr-xr-x    1 14       2000     34025780 Nov 26 02:55 AT_20_10_REL_ARCA_112618.txt
-r-xr-xr-x    1 14       2000     34036101 Jan 18 11:13 AT_20_10_REL_ARCA_011819.txt
-r-xr-xr-x    1 14       2000     34036564 Jan 25 01:09 AT_20_10_REL_ARCA_012519.txt
-r-xr-xr-x    1 14       2000     34037013 Jan 17 00:45 AT_20_10_REL_ARCA_011719.txt
-r-xr-xr-x    1 14       2000     34037370 Dec 19 22:10 AT_20_10_REL_ARCA_121918.txt -> Wrong sort!
-r-xr-xr-x    1 14       2000     34041306 Feb 03 21:42 AT_20_10_REL_ARCA_020319.txt
-r-xr-xr-x    1 14       2000     34099207 Feb 08 03:15 AT_20_10_REL_ARCA_020819.txt
-r-xr-xr-x    1 14       2000     34099827 Feb 11 02:55 AT_20_10_REL_ARCA_021119.txt

It is possible to group the dates in pairs with sed and/or regex? I have six [0-9]; one per each date digit. What about if its possibe to us regex or sed to sort those in pairs? like for example 100518; to sort 10 then 05 then 18.

Using more FTP_FILES_LIST | grep "$A[0-9][0-9][0-9][0-9][0-9][9].txt" | sort -k 9 is a workaround but I'm losing all files from 2018. I added a picture of the output because the format changes here! enter image description here

closed as unclear what you're asking by roaima, JRFerguson, Thomas, Archemar, X Tian Feb 13 at 11:41

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • None of your output lists matches the grep commands you're using. (For example, using the first grep in your question against the set of files you've shown returns no results.) Please go back and edit your question so that they match. – roaima Feb 8 at 13:46
  • 2
    It looks to me like you just want to sort by the 5th character of the 1st (only) field on i.e. sort -k1.5 - I don't see how regular expressions need to be involved here at all – steeldriver Feb 8 at 14:07
  • 1
    your use of -k9 makes it sound like you're parsing the output of an ftp ls command; is that true? – Jeff Schaller Feb 8 at 14:29
  • Post updated; please now its more understandable and can received a good answer. Thanks to all – Javier Gonzalez Feb 11 at 18:07
  • @JavierGonzalez, I've updated my answer given the actual filenames; give it a shot. – Jeff Schaller Feb 11 at 18:44
0

Based on the examples I assume the file names are in the format MMDDYY.txt where MM is month, DD is day and YY is year.

You could rearrange the characters for sorting using sed

sed 's/\(....\)\(..\)/\2\1/'| sort | sed 's/\(..\)\(....\)/\2\1/'

or if your sed supports option -E for extended regular expressions

sed -E 's/(....)(..)/\2\1/'| sort | sed -E 's/(..)(....)/\2\1/'

The first sed will exchange the first group of 4 characters (MMDD) and the second group of 2 characters (YY). Whatever may follow (.txt) will be unchanged. This will change 100518.txt to 181005.txt etc. After sorting the characters are exchanged back.

This script assumes the list of filenames is already filtered to contain only names of the correct format. Otherwise the match patterns should be more complicated to match only 6 numbers followed by .txt.

sort without option will put the highest date last, use sort -r to reverse the order

  • If you sed -E then you don't need to escape the () – bu5hman Feb 8 at 14:55
  • ls *.txt | sed -E 's/^(..)(..)(..)/\3\1\2/' | sort | sed -E 's/^(..)(..)(..)/\2\3\1/' – bu5hman Feb 8 at 15:09
  • @bu5hman You are right, the pattern for .txt can be left out. I think the anchor ^ is not necessary because a series of grouped . will match anything from the start of the line. – Bodo Feb 8 at 15:15
0

If you're already assuming that the timestamps are in field 9, then you can tell sort to pick apart the three date fields:

sort -k9.24,9.25n -k9.20,9.21n -k9.22,9.23n FTP_FILES_LIST

Here's a sample run over your sample input using GNU sort's --debug option; you can see the series of dashed lines indicating the sort keys that sort used in succession to determine the ordering:

-r-xr-xr-x    1 14       2000     34010091 Oct 05 00:42 AT_20_10_REL_ARCA_100518.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34025780 Nov 26 02:55 AT_20_10_REL_ARCA_112618.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34037370 Dec 19 22:10 AT_20_10_REL_ARCA_121918.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34037013 Jan 17 00:45 AT_20_10_REL_ARCA_011719.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34036101 Jan 18 11:13 AT_20_10_REL_ARCA_011819.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34036564 Jan 25 01:09 AT_20_10_REL_ARCA_012519.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34041306 Feb 03 21:42 AT_20_10_REL_ARCA_020319.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34099207 Feb 08 03:15 AT_20_10_REL_ARCA_020819.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________
-r-xr-xr-x    1 14       2000     34099827 Feb 11 02:55 AT_20_10_REL_ARCA_021119.txt
                                                                              __
                                                                          __
                                                                            __
____________________________________________________________________________________

Doing the sorting this way is fragile in that it relies on the filenames to start in the 9th whitespace-delimited field. An alternative would be to use some reliance on the filename pattern itself, perhaps the appearance of the timestamp in the 6th underscore-delimited:

sort -t_ -k6.5,6.6n -k6.1,6.2n -k6.3n,6.4n FTP_FILES_LIST
  • Yes jeff. Its a single file with a list from all files from an ftp – Javier Gonzalez Feb 10 at 3:03
  • Hi Jeff. I edited my post; maybe can help better to answer the question. Sadly the "sed" didnt sort as expected. Read my new updated post please. Maybe can help – Javier Gonzalez Feb 11 at 18:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.