2

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google).

So, I have a file with data that looks like this (irrelevant data omitted):

... ... 2014 Jan 01 ... ... ...
... ... 2014 Jan 02 ... ... ...
(...)
... ... 2014 Mar 01 ... ... ...
(...)

We can assume the data is sorted by date. So, one thing I want to do is to capture the lines that fit a range of months. For example, if I have a range of February to April, I want to get all of the lines with Feb, Mar, and Apr in the month column.

To de-specify the problem and abstract my request a bit, I really just want to get the first line the matches a certain pattern and capture all of the lines from then on until I reach the last occurrence of another pattern (and all of the lines that match a certain pattern are grouped together).

I've found solutions to problems that are somewhat similar, and they use either grep, sed, or awk. I'm not sure which would be best in this case.

This is all being done in a bash script, so I've thought about having an array with all of the month "codes" in it [Jan, Feb, Mar, ...], and somehow checking whether I'm within the range I want, but I feel like there's probably a more elegant solution to this problem.

Edit: @jasonwryan Well, I'm actually supplying the script a range, so './script.sh --month "Jan, Apr"' sorts the data file by month, then should be able to grep the arbitrary range (from Jan to Apr in this case). So, awk '/Feb|Apr|May/' works for that case, but if I want Feb-Nov, that'd be different (awk '/Feb|Mar|Apr|May|Jun|Jul|Aug|Sept|Nov/' file). So that would be tough to automate based on an arbitrary range. Also, there should be a line for every month at least once (and probably each day, too). Forgot to specify that.

  • awk '/Feb|Apr|May/' file (assuming there is a date for every record--your question isn't clear about that)... – jasonwryan Feb 8 '15 at 4:24
  • One file per year, or would you ever need to search for Nov-Feb for example? – Kent Feb 8 '15 at 4:42
  • apologies for the unnecessary bump to this (and my other) posts, I ended up doing some edits for clarity. – grish Aug 20 '15 at 17:11
  • @grish - unless I'm missing something, to get the Feb-Apr lines, simply run sed -n '/Feb/,/May/{/May/!p}' infile Since your file is sorted by date, this will print from the first line matching Feb up to but not including the first line matching May (which means it prints up to the last line matching Apr). Similarly, for Feb-Nov you'd do sed -n '/Feb/,/Dec/{/Dec/!p}' infile. Let me know if I misunderstood your question... – don_crissti Aug 21 '15 at 19:40
1
mrng(){ sed "$(set -f;unset IFS rng l;n=' 
';[ -n "$ZSH_VERSION" ] && emulate sh                
prng()  for m do rng=${r%%"$m"*}${r##*"$m"} _l=$((_l+1))
                 printf "\n\n%s\n/$pat/{\n\t:$l.$_l\n\tn" $m
                 printf "\n\n%s\n\t/$pat/b$l.$_l.0" $rng
                 printf "\n\tb$l.$_l\n\t:$l.$_l.0\n}"
        done                                        
pat=$(  printf %s "${1:-%m}$n"| sed -n 's/%/&&/g;l'|
        sed ":n$n\$!N;s/\\\\\n//;tn${n}s/\$$//"); shift
r=$(    locale -c LC_TIME|sed '4!d;y/;/ /')       
for m do case   $m      in      (-)     rng=$r  ;;
        (-*)    rng=${r%%"${m#-}"*}${m#-}       ;;
        (*-)    rng=${m%-}${r##*"${m%-}"}       ;;     
        (*-*)   rng=${m%%-*}${r##*"${m%%-*}"}   ;        
                case    $rng    in      (*${m##*-}*)     
                rng=${rng%%"${m##*-}"*}${m##*-}  ;;(*)   
                rng=$rng\ ${r%%"${m##*-}"*}${m##*-};;esac
        ;;esac; : $((l+=1))                                                
        prng    ${rng:="$m"}; unset rng                                    
done|   sed "   1d;s/.*\(...\)\(\n\)\(.*[^%]\(%%\)*\)%m/\2\1\2\3\1/
                /./!{N;N$n};/\n/D"
);d"       
}

This is a shell function - you'll either need to adapt it into a shell-script if that's how you plan to call it, or to evaluate it in the current shell. You can call it like:

mrng "$pat" Jan-Mar Jun Sep-Nov <infile

It accepts open-ended ranges as well, such as - to mean all or Mar- to mean from March through December. And arguments don't have to be ranges - as above, Jun is fine.

In fact, though, it doesn't really interpret the month names at all - it collects those from the locale utility (which is a dependency) and works with whatever the current locale says the 3-char month names are.

It can do wrap-around ranges, and, in truth, almost all of them actually do wrap-around, or, maybe it is better to say, accumulate, anyway.

It expects its first argument to be a sed compatible BRE pattern - with the exception that wherever you would expect to encounter the month name, you should use %m instead. You can insert several %ms as well - which you might do if you only wanted to match lines like Mar...Mar - Jun...Jun. Probably that is not incredibly useful, but maybe...?

Though it looks complicated, more than half of that is devoted to arg-parsing - the sed is relatively simple, after all. For example, if I do:

mrng %m Dec-Jan

...it will generate a sed script that looks like:

/Dec/{
        :1.1
        n
        /Jan/b1.1.0
        /Feb/b1.1.0
        /Mar/b1.1.0
        /Apr/b1.1.0
        /May/b1.1.0
        /Jun/b1.1.0
        /Jul/b1.1.0
        /Aug/b1.1.0
        /Sep/b1.1.0
        /Oct/b1.1.0
        /Nov/b1.1.0
        b1.1
        :1.1.0
}
/Jan/{
        :1.2
        n
        /Feb/b1.2.0
        /Mar/b1.2.0
        /Apr/b1.2.0
        /May/b1.2.0
        /Jun/b1.2.0
        /Jul/b1.2.0
        /Aug/b1.2.0
        /Sep/b1.2.0
        /Oct/b1.2.0
        /Nov/b1.2.0
        /Dec/b1.2.0
        b1.2
        :1.2.0
};d

...which winds up being a lot of code, but most of which is not evaluated most of the time. On a typical line, it will check if it matches Dec, and, if not, if it matches Jan, and, if not, delete it from output.

But once it does match one of those patterns, it will begin a simple branch loop. So, given the above example, if a line matches Dec then it will be printed and overwritten with the next input line. If the new line matches any month but Dec, sed branches to the :1.1.0 label - which means that line has yet to be evaluated against Jan - where it will get similar treatment - but will not be evaluated against any month before Dec. If it does not match any month but Dec, then sed branches up to the :1.1 label, which gets the line printed and the next pulled in and etc.

If I did - instead, then it would gen a function similar to the above - each bearing its own unique : label - for every month within the range. This means that the command-line arguments have cumulative effect. Some examples:

printf %s\\n 'not a month' May 'not a month' 'also not a month' Apr |
m_rng %m Apr May

The above prints:

May
not a month
also not a month

Because May comes before Apr in input, but after Apr on the command-line. However, it is a pretty rough heuristic. The input is processed in the order of the command-line args, but as soon as a full-cycle is made the processing starts over again, so...

printf %s\\n 'not a month' May 'not a month' 'also not a month' Jun Apr |
m_rng %m Apr May

...prints...

May
not a month
also not a month
Apr

Because the cycle breaks at Jun, the line is deleted, and processing starts again from the top w/ the next input line - Apr.

Anyway, for your pattern you should use:

mrng '^\([^ ]\{1,\}  *\)\{3\}%m' [month args]
  • I can't quite get this to work, is there something missing possibly? I'm replacing 'infile' with the name of my file, and tried using the #!/bin/bash and #!/bin/sh shebangs. It just sits there for a minute or two when I run it, so it does seem to be reading the file, but there's no output. – grish Feb 12 '15 at 3:56
  • @grish - thanks. It doesn't actually work is why. Though it just sitting there doesn't make any sense. I did write one that does work, but I guess I forgot to update it. This ends the range at the first occurrence of, for example, Mar in a Jan-Mar range. I'll edit it in a few and put the fixed one in. – mikeserv Feb 12 '15 at 4:01
  • ah, glad you figured it out. I'd like to have done it on my own, but I have to work quickly and figuring out stuff with bash/shell functions/etc. always takes me awhile – grish Feb 12 '15 at 6:39
  • are you able to post the update code now? – grish Feb 13 '15 at 22:05
  • @grish - yeah, I'm doing it now. – mikeserv Feb 15 '15 at 7:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.