Consider the input file
1,10/22/2017,Scheduled
2,10/23/2017,Confimred
1,10/24/2017,NA
1,10/29/2017,Scheduled
3,11/1/2017,Scheduled
1,11/2/2017,Scheduled
How do I filter date(within range) in 2nd column by providing date range as input?
Add a comment
|
2 Answers
This snippet:
# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
pl " Input data file $FILE:"
head data1
# start="10/29/2017" end="11/2/2017"
START="10/29/2017"
END="11/2/2017"
pl " Results, from $START through $END:"
dateutils.dgrep -i "%m/%d/%Y" ">=$START" '&&' "<=$END" < data1
pl " Unsorted file, data2:"
head data2
pl " Results, from $START through $END, randomly organized file:"
dateutils.dgrep -i "%m/%d/%Y" ">=$START" '&&' "<=$END" < data2
produces:
-----
Input data file :
1,10/22/2017,Scheduled
2,10/23/2017,Confimred
1,10/24/2017,NA
1,10/29/2017,Scheduled
3,11/1/2017,Scheduled
1,11/2/2017,Scheduled
-----
Results, from 10/29/2017 through 11/2/2017:
1,10/29/2017,Scheduled
3,11/1/2017,Scheduled
1,11/2/2017,Scheduled
-----
Unsorted file, data2:
1,10/22/2017,Scheduled
1,10/24/2017,NA
1,10/29/2017,Scheduled
1,11/2/2017,Scheduled
2,10/23/2017,Confimred
3,11/1/2017,Scheduled
-----
Results, from 10/29/2017 through 11/2/2017, randomly organized file:
1,10/29/2017,Scheduled
1,11/2/2017,Scheduled
3,11/1/2017,Scheduled
on a system like:
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution : Debian 8.9 (jessie)
bash GNU bash 4.3.30
Because the comparison is arithmetic on date-formatted data, the data can be in any order. The final result can be sorted if desired -- see sort, msort, dsort. The dateutils codes are available in many repositories, and in OSX (via brew).
Some details for dateutils.dgrep:
dateutils.dgrep Grep standard input for lines that match EXPRESSION. (man)
Path : /usr/bin/dateutils.dgrep
Package : dateutils
Home : http://www.fresse.org/dateutils
Version : 0.3.1
Type : ELF64-bitLSBsharedobject,x86-64,version1(S ...)
Help : probably available with -h,--help
Home : https://github.com/hroptatyr/dateutils (doc)
Best wishes ... cheers, drl
-
Thanks to anonymous editor for catching my spelling errors. A downvote for each spelling error seems a touch harsh -- when I took English at William & Mary (In the Navy), that was the penalty for misspelling, but we were warned about it, and it WAS an English class. I have not seen any policies about that here. I looked on Meta for some guidance, and I didn't see a consensus for the issue of spelling, except when it was likely to cause confusion. I would have rather had a heads-up and tried to fix it myself. I usually do the spell check, and clearly omitted it here, perhaps on the final edit.– drlDec 23, 2017 at 13:06
Use awk and call shell date command Using getline from a Pipe:
awk -v start="$start" -v end="$end" -F, '
BEGIN{srt="date -d"start" +%s"; srt|getline start; close(srt);
ed="date -d"end" +%s"; ed|getline end; close(ed) }
{ bkp=$0; epoch="date -d"$2" +%s";epoch |getline $2;close(epoch)};
($2>=start && $2<=end){print bkp}' infile
For the below input:
1,10/22/2017,Scheduled
1,10/24/2017,NA
1,10/24/2017,NA,NA
1,10/29/2017,Scheduled
3,11/1/2017,Scheduled
1,11/2/2017,NA
5,9/30/2017,Confirmed
6,10/1/2017,Scheduled
With start='10/24/2017' and end='11/1/2017', the result is:
1,10/24/2017,NA
1,10/24/2017,NA,NA
1,10/29/2017,Scheduled
3,11/1/2017,Scheduled
-
-
2-1, this assumes Awk can compare dates reliably. Awk does not have any date-specific comparisons. This code will be extremely fragile and will break (for example) when the dates span multiple years.– WildcardOct 23, 2017 at 20:39
-
awk -v start="$start" -v end="$end" -F, '$2>=start && $2<=end' filename works fine by giving range of dates but if start="10/25/2017" and end="10/25/2017" and if the input file has only one entry with date 10/25/2017 then output is not obtained Oct 24, 2017 at 19:28
-
@Wildcard Updated my answer, to does support date as date in awk calling shell date.– αғsнιηOct 25, 2017 at 13:35