2

Situation

I have awk command reads results from mywebsite.log current command

awk 'BEGIN{ curr_d=strftime("%d/%b/%Y", systime()) }
$3~"^\\["curr_d{ cc[$8]++ } 
END{ for(i in cc) print i":"cc[i] }' /var/log/mywebsite.log > /home/mywebsite/www/countries.txt

Is extract only country code and counts country codes then exported to countries.txt as below for same day records only getting the day from linux.

IN:35
US:123
QA:3
RU:23
MA:23

Desired Output:

I would like to get only most visited 20 IP address and count how many times these ip addresses visited same day. The day match with linux date (today) then save to countries2.txt

168.235.194.192-US:10
12.34.65.34-RU:5
223.223.129.225-IN:4
34.21.57.1-QA:2
212.252.112.45-MA:2

/var/log/mywebsite.log

        123.223.129.225 - [25/Nov/2017:02:01:59 +1100] GET /pdf/aloha-micronutrients.html HTTP/1.1 US
        64.223.129.225 - [25/Nov/2017:02:01:59 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        53.223.129.225 - [25/Nov/2017:02:01:59 +1100] GET /dmcaemail.png HTTP/1.1 IN
        223.223.129.225 - [26/Nov/2017:02:01:59 +1100] GET /pdf/role-of-micronutrients.html HTTP/1.1 IN
        223.223.129.225 - [26/Nov/2017:02:01:59 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 IN
        223.223.129.225 - [26/Nov/2017:02:01:59 +1100] GET /dmcaemail.png HTTP/1.1 IN
        223.223.129.225 - [26/Nov/2017:02:01:59 +1100] GET /logo.jpg HTTP/1.1 IN
        66.249.92.158 - [26/Nov/2017:02:02:00 +1100] GET /pdf/role-of-micronutrients.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:02:07 +1100] GET /pdf/gastroscopy.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:02:08 +1100] GET /pdf/astringents.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:02:08 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:02:08 +1100] GET /logo.jpg HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:02:18 +1100] GET /Why-study-literature--pdf.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:02:39 +1100] GET /index.php/about/ HTTP/1.1 US
        37.211.15.120 - [26/Nov/2017:02:02:41 +1100] GET /The-Honey-Bee--Florida-State-University--pdf.html HTTP/1.1 QA
        157.55.39.169 - [26/Nov/2017:02:02:47 +1100] GET /pdf/emt-refresher-class.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:02:49 +1100] GET /urea-urei-urease-01-M-urea---pdf.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:03:11 +1100] GET /pdf/typhoon-presentation.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:03:13 +1100] GET /pdf/astringents.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:03:13 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:03:18 +1100] GET /Trade-Secrets--JurisDiction--pdf.html HTTP/1.1 US
        196.190.64.27 - [26/Nov/2017:02:03:18 +1100] GET /pdf/antibiotics-residues.html HTTP/1.1 ET
        168.235.194.192 - [26/Nov/2017:02:03:33 +1100] GET /?s=astringents+and+emetics HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:03:34 +1100] GET /pdf/astringents-and-emetics.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:03:34 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:03:37 +1100] GET /pdf/crossing-red-sea.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:03:47 +1100] GET /Wilderness-Manager-Survey--pdf.html HTTP/1.1 US
        71.61.49.221 - [26/Nov/2017:02:03:47 +1100] GET /pdf/sbar.html HTTP/1.1 US
        71.61.49.221 - [26/Nov/2017:02:03:47 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        71.61.49.221 - [26/Nov/2017:02:03:47 +1100] GET /logo.jpg HTTP/1.1 US
        71.61.49.221 - [26/Nov/2017:02:03:48 +1100] GET /dmcaemail.png HTTP/1.1 US
        196.190.64.27 - [26/Nov/2017:02:03:55 +1100] GET //antibiotics-residues-web-Guidelines-for-Antibiotic-Residue-Avoidance--pdf.html HTTP/1.1 ET
        168.235.194.192 - [26/Nov/2017:02:03:56 +1100] GET /pdf/astringents.html HTTP/1.1 US
        168.235.194.192 - [26/Nov/2017:02:03:56 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        196.190.64.27 - [26/Nov/2017:02:04:00 +1100] GET /antibiotics-residues-web-Guidelines-for-Antibiotic-Residue-Avoidance--pdf.html HTTP/1.1 ET
        66.249.92.129 - [26/Nov/2017:02:04:03 +1100] GET /antibiotics-residues-web-Guidelines-for-Antibiotic-Residue-Avoidance--pdf.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:04:07 +1100] GET /pdf/hose-drill-fire-fighting-training.html HTTP/1.1 US
        209.126.120.38 - [26/Nov/2017:02:04:12 +1100] GET /pdf/sbar.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:04:18 +1100] GET /WAS-THE-ALAMO-A-DEFEAT-OR-A-VICTORY-OR-A-DEFEAT--pdf.html HTTP/1.1 US
        141.8.143.151 - [26/Nov/2017:02:04:18 +1100] GET /Quality-Tools--PCMAC--pdf.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:04:37 +1100] GET /pdf/environmental-impacts-of-pesticides-effects.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:04:48 +1100] GET /Fly-Eagle-Fly-PowerPoint-2--Jefferson-County-Schools-TN--pdf.html HTTP/1.1 US
        168.235.195.45 - [26/Nov/2017:02:04:59 +1100] GET /pdf/story-boarding.html HTTP/1.1 US
        168.235.195.45 - [26/Nov/2017:02:04:59 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        117.211.126.164 - [26/Nov/2017:02:06:05 +1100] GET /pdf/food-preservation-methods-.html HTTP/1.1 IN
        117.211.126.164 - [26/Nov/2017:02:06:06 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 IN
        117.211.126.164 - [26/Nov/2017:02:06:06 +1100] GET /logo.jpg HTTP/1.1 IN
        157.55.39.156 - [26/Nov/2017:02:06:06 +1100] GET /pdf/bearing-materials.html HTTP/1.1 US
        117.211.126.164 - [26/Nov/2017:02:06:14 +1100] GET /dmcaemail.png HTTP/1.1 IN
        157.55.39.228 - [26/Nov/2017:02:06:18 +1100] GET /Programme-6-Water-Resources-Management-Development------pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:06:36 +1100] GET /pdf/endocrine-disruption.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:06:48 +1100] GET /Pre-Algebra--pdf.html HTTP/1.1 US
        196.86.85.113 - [26/Nov/2017:02:07:06 +1100] GET /pdf/applied-linguistics.html HTTP/1.1 MA
        157.55.39.156 - [26/Nov/2017:02:07:06 +1100] GET /Nurse-to-Patient-Ratio--pdf.html HTTP/1.1 US
        196.86.85.113 - [26/Nov/2017:02:07:08 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 MA
        196.86.85.113 - [26/Nov/2017:02:07:09 +1100] GET /logo.jpg HTTP/1.1 MA
        196.86.85.113 - [26/Nov/2017:02:07:09 +1100] GET /dmcaemail.png HTTP/1.1 MA
        66.249.92.159 - [26/Nov/2017:02:07:11 +1100] GET /pdf/applied-linguistics.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:07:14 +1100] GET /pdf/on-thypoid.html HTTP/1.1 US
        196.86.85.113 - [26/Nov/2017:02:07:16 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 MA
        157.55.39.228 - [26/Nov/2017:02:07:18 +1100] GET /PowerPoint-Template-GuideStone-Corporate-Presentations--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:07:38 +1100] GET /pdf/Embracing-Generational-Differences--pdf.html HTTP/1.1 US
        196.86.85.113 - [26/Nov/2017:02:07:41 +1100] GET //Applied-Linguistics--Ohio-State-University--pdf.html HTTP/1.1 MA
        196.86.85.113 - [26/Nov/2017:02:07:42 +1100] GET /Applied-Linguistics--Ohio-State-University--pdf.html HTTP/1.1 MA
        168.235.195.239 - [26/Nov/2017:02:07:42 +1100] GET /pdf/prokinetics-antiemetics.html HTTP/1.1 US
        168.235.195.239 - [26/Nov/2017:02:07:42 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        103.231.160.187 - [26/Nov/2017:02:07:43 +1100] GET /pdf/electrostatics-gauss-s-law-and-its-applications.html HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:07:44 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:07:44 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:07:44 +1100] GET /dmcaemail.png HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:07:44 +1100] GET /logo.jpg HTTP/1.1 BD
        66.249.92.158 - [26/Nov/2017:02:07:45 +1100] GET /pdf/electrostatics-gauss-s-law-and-its-applications.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:07:47 +1100] GET /pdf/roll-of-thunder-hear-my-cry.html HTTP/1.1 US
        103.231.160.187 - [26/Nov/2017:02:08:03 +1100] GET //electrostatics-gauss-s-law-and-its-applications-web--Principles-with-Applications-7--pdf.html HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:08:03 +1100] GET /electrostatics-gauss-s-law-and-its-applications-web--Principles-with-Applications-7--pdf.html HTTP/1.1 BD
        103.231.160.187 - [26/Nov/2017:02:08:03 +1100] GET /download.jpg HTTP/1.1 BD
        66.249.92.158 - [26/Nov/2017:02:08:04 +1100] GET /electrostatics-gauss-s-law-and-its-applications-web--Principles-with-Applications-7--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:08:06 +1100] GET /pdf/ricci-maternity.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:08:18 +1100] GET /Pathology--pcmac--pdf.html HTTP/1.1 US
        106.202.142.57 - [26/Nov/2017:02:08:23 +1100] GET /pdf/structure-of-atom-for-class-11th.html HTTP/1.1 IN
        106.202.142.57 - [26/Nov/2017:02:08:23 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 IN
        106.202.142.57 - [26/Nov/2017:02:08:24 +1100] GET /logo.jpg HTTP/1.1 IN
        106.202.142.57 - [26/Nov/2017:02:08:24 +1100] GET /dmcaemail.png HTTP/1.1 IN
        66.249.92.159 - [26/Nov/2017:02:08:25 +1100] GET /pdf/structure-of-atom-for-class-11th.html HTTP/1.1 US
        168.235.195.239 - [26/Nov/2017:02:08:27 +1100] GET /pdf/astringents.html HTTP/1.1 US
        168.235.195.239 - [26/Nov/2017:02:08:27 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        75.188.138.63 - [26/Nov/2017:02:08:28 +1100] GET /pdf/ethics-in-corrections.html HTTP/1.1 US
        75.188.138.63 - [26/Nov/2017:02:08:28 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        75.188.138.63 - [26/Nov/2017:02:08:28 +1100] GET /logo.jpg HTTP/1.1 US
        75.188.138.63 - [26/Nov/2017:02:08:28 +1100] GET /dmcaemail.png HTTP/1.1 US
        106.202.142.57 - [26/Nov/2017:02:08:28 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 IN
        66.249.92.158 - [26/Nov/2017:02:08:29 +1100] GET /pdf/ethics-in-corrections.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:09:17 +1100] GET /Part-III--Modes-of-Reproduction--Arkansas-State-University--pdf.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:09:37 +1100] GET /pdf/composites.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:09:47 +1100] GET /COMBINING-ECONOMICS-WITH-NETWORK-ENGINEERING--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:12:47 +1100] GET /pdf/materials-requirements-planning--pdf.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:12:48 +1100] GET /pdf/sectional-anatomy-brain.html HTTP/1.1 US
        66.249.66.94 - [26/Nov/2017:02:12:52 +1100] GET /pdf/bat-pdf.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:13:01 +1100] GET /HANDWASHING-Timeless-Wisdom--A-Healthy-Habit--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:13:07 +1100] GET /Common-Diseases-of-Small-Animals--Northwest-ISD--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:13:17 +1100] GET /Nomadic-Empires-and-Eurasian-Integration--pdf.html HTTP/1.1 US
        2.31.250.240 - [26/Nov/2017:02:13:45 +1100] GET /pdf/pink-floyd-power-point.html HTTP/1.1 GB
        2.31.250.240 - [26/Nov/2017:02:13:45 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 GB
        2.31.250.240 - [26/Nov/2017:02:13:45 +1100] GET /logo.jpg HTTP/1.1 GB
        2.31.250.240 - [26/Nov/2017:02:13:45 +1100] GET /dmcaemail.png HTTP/1.1 GB
        2.31.250.240 - [26/Nov/2017:02:13:46 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 GB
        157.55.39.169 - [26/Nov/2017:02:13:48 +1100] GET /pdf/firefighter-safty-and-health.html HTTP/1.1 US
        66.249.92.129 - [26/Nov/2017:02:13:49 +1100] GET /pdf/pink-floyd-power-point.html HTTP/1.1 US
        34.207.174.103 - [26/Nov/2017:02:13:55 +1100] GET /pdf/pink-floyd-power-point.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:14:10 +1100] GET /pdf/fire-team-formations.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:14:11 +1100] GET /pdf/socialization-process.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:14:17 +1100] GET /Erythrocyte-Sedimentation-Rare--port-health-gov-ir--pdf.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:14:26 +1100] GET /FAST-Exam--lectures-shanyar-com--pdf.html HTTP/1.1 US
        95.233.1.98 - [26/Nov/2017:02:14:31 +1100] GET /wp-login.php HTTP/1.1 IT
        95.233.1.98 - [26/Nov/2017:02:14:33 +1100] GET / HTTP/1.1 IT
        71.61.49.221 - [26/Nov/2017:02:14:37 +1100] GET /pdf/sbar.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:14:37 +1100] GET /American-Heart-Association--Skidmore-College--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:14:47 +1100] GET /Truth-in-Justice--stingraybobs-weebly-com--pdf.html HTTP/1.1 US
        66.249.66.65 - [26/Nov/2017:02:14:59 +1100] GET /logo.jpg HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:15:08 +1100] GET /Animal-VirusHost-Interactions--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:15:17 +1100] GET /pdf/leadership-in-nursing-pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:15:47 +1100] GET /Presentation-title--Teachinglinks-net--pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:15:49 +1100] GET /pdf/nozzles-pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:15:49 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:15:51 +1100] GET /pdf/fatty-acids-beta-oxidation.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:15:54 +1100] GET /pdf/cantilever-beam-analysis.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:16:02 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:16:08 +1100] GET /pdf/anesthesia-for-thyroid-surgery.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:16:10 +1100] GET //nozzles-opwglobal-Why-Are-These-Nozzles-Found--pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:16:11 +1100] GET /nozzles-opwglobal-Why-Are-These-Nozzles-Found--pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:16:11 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:16:17 +1100] GET /pdf/english-pronunciation-exercises.html HTTP/1.1 US
        157.55.39.68 - [26/Nov/2017:02:16:27 +1100] GET /robots.txt HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:16:36 +1100] GET /Orthopaedic-special-tests-for-the-shoulder--Stort-Physio--pdf.html HTTP/1.1 US
        157.49.7.217 - [26/Nov/2017:02:16:39 +1100] GET /pdf/typhoid-vaccine.html HTTP/1.1 IN
        157.49.7.217 - [26/Nov/2017:02:16:39 +1100] GET /pdf/typhoid-vaccine.html HTTP/1.1 IN
        157.49.7.217 - [26/Nov/2017:02:16:40 +1100] GET /logo.jpg HTTP/1.1 IN
        157.49.7.217 - [26/Nov/2017:02:16:40 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 IN
        157.49.7.217 - [26/Nov/2017:02:16:40 +1100] GET /dmcaemail.png HTTP/1.1 IN
        51.15.44.92 - [26/Nov/2017:02:16:40 +1100] GET /Rocks-and-Minerals--Edgewood-High-School--pdf.html HTTP/1.1 GB
        157.49.7.217 - [26/Nov/2017:02:16:42 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 IN
        66.249.92.159 - [26/Nov/2017:02:16:43 +1100] GET /pdf/typhoid-vaccine.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:16:47 +1100] GET /PROYECTOS-DE-AHORRO-DE-ENERGIA-ELECTRICA--Luis-Alberto-Arcos------pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:17:03 +1100] GET //nozzles-bae-A-Summary-of-Studies-Comparing--pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:17:04 +1100] GET /nozzles-bae-A-Summary-of-Studies-Comparing--pdf.html HTTP/1.1 US
        168.235.198.112 - [26/Nov/2017:02:17:04 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:17:08 +1100] GET /pdf/test-construction.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:17:16 +1100] GET /The-Japan-Culture--Rice-University--pdf.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:17:17 +1100] GET /pdf/brodmans-areas.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:17:47 +1100] GET /Organisational-Buying-Behaviour--McGrawHill-Education--pdf.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:18:06 +1100] GET /pdf/organizational-behaviour-robbins-chapter-2.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:18:17 +1100] GET /pdf/fluid-management-in-children.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:18:19 +1100] GET /pdf/sociology-anthropology.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:18:37 +1100] GET /pdf/macro-minerals-pdf.html HTTP/1.1 US
        110.54.187.210 - [26/Nov/2017:02:18:39 +1100] GET /pdf/facts-and-opinion-lessons.html HTTP/1.1 PH
        110.54.187.210 - [26/Nov/2017:02:18:40 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 PH
        77.88.47.68 - [26/Nov/2017:02:18:40 +1100] GET /pdf/plant-nutrition.html HTTP/1.1 US
        110.54.187.210 - [26/Nov/2017:02:18:41 +1100] GET /logo.jpg HTTP/1.1 PH
        110.54.187.210 - [26/Nov/2017:02:18:41 +1100] GET /dmcaemail.png HTTP/1.1 PH
        110.54.187.210 - [26/Nov/2017:02:18:42 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 PH
        66.249.92.159 - [26/Nov/2017:02:18:44 +1100] GET /pdf/facts-and-opinion-lessons.html HTTP/1.1 US
        157.55.39.156 - [26/Nov/2017:02:18:47 +1100] GET /POSTPARTUM-DEPRESSION-BEYOND-THE-BLUES--Utah-Department-of------pdf.html HTTP/1.1 US
        67.111.46.50 - [26/Nov/2017:02:19:02 +1100] GET /pdf/GLASS-IONOMER-CEMENT--pdf.html HTTP/1.1 US
        67.111.46.50 - [26/Nov/2017:02:19:02 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        67.111.46.50 - [26/Nov/2017:02:19:02 +1100] GET /logo.jpg HTTP/1.1 US
        67.111.46.50 - [26/Nov/2017:02:19:02 +1100] GET /dmcaemail.png HTTP/1.1 US
        67.111.46.50 - [26/Nov/2017:02:19:04 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        66.249.92.158 - [26/Nov/2017:02:19:04 +1100] GET /pdf/GLASS-IONOMER-CEMENT--pdf.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:19:07 +1100] GET /pdf/pstn.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:19:17 +1100] GET /pdf/seed-dormancy-and-its-importance-pdf.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:19:43 +1100] GET /pdf/sccm-2012-r2.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:19:47 +1100] GET /pdf/laser-angioplasty.html HTTP/1.1 US
        8.37.232.108 - [26/Nov/2017:02:20:01 +1100] GET /pdf/leukocyte-disorders.html HTTP/1.1 US
        8.37.232.108 - [26/Nov/2017:02:20:01 +1100] GET /dmcaemail.png HTTP/1.1 US
        8.37.232.108 - [26/Nov/2017:02:20:01 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        8.37.232.108 - [26/Nov/2017:02:20:01 +1100] GET /logo.jpg HTTP/1.1 US
        8.37.232.108 - [26/Nov/2017:02:20:04 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:20:07 +1100] GET /artificial-intelligence-2016-cs-AI-or-Computational-Intelligence-There--pdf.html HTTP/1.1 US
        8.37.235.34 - [26/Nov/2017:02:20:07 +1100] GET /pdf/basic-electrical-substation.html HTTP/1.1 US
        8.37.235.34 - [26/Nov/2017:02:20:07 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        8.37.235.34 - [26/Nov/2017:02:20:07 +1100] GET /dmcaemail.png HTTP/1.1 US
        8.37.235.34 - [26/Nov/2017:02:20:07 +1100] GET /logo.jpg HTTP/1.1 US
        8.37.235.34 - [26/Nov/2017:02:20:09 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:20:17 +1100] GET /pdf/operating-brts.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:20:18 +1100] GET /pdf/mrcp-paces.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:20:47 +1100] GET /pdf/1-corinthians.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:20:53 +1100] GET /pdf/electric-shock-causes-symptoms-treatment.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:21:11 +1100] GET /pdf/mk.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:21:17 +1100] GET /pdf/have-got-exercises.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:21:31 +1100] GET /pdf/types-of-cement-pdf.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:21:44 +1100] GET /pdf/postmodernism-theory.html HTTP/1.1 US
        40.77.167.11 - [26/Nov/2017:02:21:47 +1100] GET /Basic-Traffic-Stops--Career-and-Technical-Education-cteunt--pdf.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:22:07 +1100] GET /JET-PROPULSION--sseaimes-files-wordpress-com--pdf.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:22:17 +1100] GET /pdf/teton-dam-failure-presentations.html HTTP/1.1 US
        68.180.229.87 - [26/Nov/2017:02:22:26 +1100] GET /tree-falling-smithsgradesix-Trees-and-Forests-%E2%80%93-Falling--pdf.html HTTP/1.1 US
        194.187.171.53 - [26/Nov/2017:02:22:30 +1100] GET /favicon.ico HTTP/1.1 FR
        207.46.13.186 - [26/Nov/2017:02:22:37 +1100] GET /pdf/accounting-and-finance-for-bankers-jaiib.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:22:47 +1100] GET /Part-1--Page--Health-Care-Conference-Administrators-Home--pdf.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:22:50 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:23:11 +1100] GET /pdf/service-excellence.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:23:17 +1100] GET /pdf/beautiful.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:23:37 +1100] GET /pdf/usps-retirement.html HTTP/1.1 US
        131.253.25.254 - [26/Nov/2017:02:23:41 +1100] GET /pdf/nursing-and-neuro-assessment.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:23:47 +1100] GET /Year-10-Summer-Exams--pdf.html HTTP/1.1 US
        85.195.116.66 - [26/Nov/2017:02:23:57 +1100] GET / HTTP/1.0 DE
        54.82.61.140 - [26/Nov/2017:02:27:23 +1100] GET /pdf/medical-surgical-nursing.html HTTP/1.1 US
        68.180.229.87 - [26/Nov/2017:02:27:24 +1100] GET /fasteners-cengage-Understand-the-theory-behind-torquetoyield--pdf.html HTTP/1.1 US
        67.141.91.182 - [26/Nov/2017:02:27:25 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:27:39 +1100] GET /pdf/management-of-patients-with-neurologic-trauma.html HTTP/1.1 US
        207.46.13.186 - [26/Nov/2017:02:27:47 +1100] GET /Earth-Science-Chapter-12--pdf.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:28:17 +1100] GET /Wrist-and-Hand--groupf2-yolasite-com--pdf.html HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:28:17 +1100] GET /pdf/seasonality-of-reproduction.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:28:18 +1100] GET /pdf/cell-phone-infrastructure.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:28:37 +1100] GET /pdf/nfpa-20.html HTTP/1.1 US
        89.3.118.149 - [26/Nov/2017:02:28:42 +1100] GET /pdf/recombinant-proteins.html HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:28:42 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:28:42 +1100] GET /logo.jpg HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:28:42 +1100] GET /dmcaemail.png HTTP/1.1 FR
        66.249.92.159 - [26/Nov/2017:02:28:44 +1100] GET /pdf/recombinant-proteins.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:28:47 +1100] GET /reakes-weebly-com--pdf.html HTTP/1.1 US
        89.3.118.149 - [26/Nov/2017:02:28:48 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 FR
        119.95.172.9 - [26/Nov/2017:02:28:49 +1100] GET /pdf/first-vita-plus.html HTTP/1.1 PH
        119.95.172.9 - [26/Nov/2017:02:28:50 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 PH
        119.95.172.9 - [26/Nov/2017:02:28:50 +1100] GET /logo.jpg HTTP/1.1 PH
        119.95.172.9 - [26/Nov/2017:02:28:50 +1100] GET /dmcaemail.png HTTP/1.1 PH
        34.230.8.39 - [26/Nov/2017:02:28:53 +1100] GET /pdf/recombinant-proteins.html HTTP/1.1 US
        68.180.229.87 - [26/Nov/2017:02:28:53 +1100] GET /N--pdf.html HTTP/1.1 US
        157.55.39.169 - [26/Nov/2017:02:29:07 +1100] GET /pdf/radiation-therapy.html HTTP/1.1 US
        66.249.92.129 - [26/Nov/2017:02:29:10 +1100] GET /pdf/first-vita-plus.html HTTP/1.1 US
        89.3.118.149 - [26/Nov/2017:02:29:17 +1100] GET /?s=recombinant+proteins HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:29:17 +1100] GET /pdf/recombinant-proteins.html HTTP/1.1 FR
        157.55.39.228 - [26/Nov/2017:02:29:17 +1100] GET /OSEP-Update-on-Identification-and-Correction-of-Noncompliance------pdf.html HTTP/1.1 US
        119.95.172.9 - [26/Nov/2017:02:29:25 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 PH
        39.33.87.181 - [26/Nov/2017:02:29:30 +1100] GET /pdf/carbonate-reservoirs.html HTTP/1.1 PK
        39.33.87.181 - [26/Nov/2017:02:29:31 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 PK
        39.33.87.181 - [26/Nov/2017:02:29:31 +1100] GET /logo.jpg HTTP/1.1 PK
        39.33.87.181 - [26/Nov/2017:02:29:31 +1100] GET /dmcaemail.png HTTP/1.1 PK
        66.249.92.159 - [26/Nov/2017:02:29:33 +1100] GET /pdf/carbonate-reservoirs.html HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:35 +1100] GET //The-Egyptian-Pyramids--Mt--Lebanon-School-District--pdf.html HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:36 +1100] GET /The-Egyptian-Pyramids--Mt--Lebanon-School-District--pdf.html HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:36 +1100] GET /wp-content/themes/vivekweb/style.css HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:36 +1100] GET /logo.jpg HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:37 +1100] GET /dmcaemail.png HTTP/1.1 US
        77.88.47.68 - [26/Nov/2017:02:29:38 +1100] GET /pdf/istishna.html HTTP/1.1 US
        40.77.167.138 - [26/Nov/2017:02:29:38 +1100] GET /pdf/digital-newbie.html HTTP/1.1 US
        134.35.90.43 - [26/Nov/2017:02:29:39 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 US
        89.3.118.149 - [26/Nov/2017:02:29:42 +1100] GET //Recombinant-DNA--pdf.html HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:29:43 +1100] GET /Recombinant-DNA--pdf.html HTTP/1.1 FR
        89.3.118.149 - [26/Nov/2017:02:29:44 +1100] GET /download.jpg HTTP/1.1 FR
        66.249.92.158 - [26/Nov/2017:02:29:44 +1100] GET /Recombinant-DNA--pdf.html HTTP/1.1 US
        54.226.130.166 - [26/Nov/2017:02:29:45 +1100] GET /Recombinant-DNA--pdf.html HTTP/1.1 US
        39.33.87.181 - [26/Nov/2017:02:29:45 +1100] GET //carbonate-reservoirs-carbonet-Seismic-is-but-one-part--pdf.html HTTP/1.1 PK
        39.33.87.181 - [26/Nov/2017:02:29:46 +1100] GET /carbonate-reservoirs-carbonet-Seismic-is-but-one-part--pdf.html HTTP/1.1 PK
        39.33.87.181 - [26/Nov/2017:02:29:46 +1100] GET /download.jpg HTTP/1.1 PK
        66.249.92.158 - [26/Nov/2017:02:29:47 +1100] GET /carbonate-reservoirs-carbonet-Seismic-is-but-one-part--pdf.html HTTP/1.1 US
        157.55.39.228 - [26/Nov/2017:02:29:47 +1100] GET /Classification-of-EEG-signals--pdf.html HTTP/1.1 US
        119.95.172.9 - [26/Nov/2017:02:29:49 +1100] GET /pdf/first-vita-plus.html HTTP/1.1 PH
        39.33.87.181 - [26/Nov/2017:02:29:56 +1100] GET /wp-content/themes/vivekweb/favicon.ico HTTP/1.1 PK
7
  • you have written count how many times these ip addresses visited same day - but that count is not figuring in your output. Update your expected output Nov 27, 2017 at 8:08
  • @RomanPerekhrest I updated expected output. I didn't count how many same ip address I have. I guess it will be around 5-10
    – danone
    Nov 27, 2017 at 8:12
  • @danone Are you looking for something like awk '{a[$NF]++ ;} END{for (i in a) print i, a[i]}' logfile ? Nov 27, 2017 at 8:29
  • so, what is 7 in the output line 223.223.129.225-IN:7? The IN code count OR the count of IP address 223.223.129.225 occurrences regarding to specific country code? Nov 27, 2017 at 8:47
  • @RomanPerekhrest 7 is count of ip address I looked at and count again this is the result 223.223.129.225-IN:4 168.235.194.192-US:10 correct count –
    – danone
    Nov 27, 2017 at 9:29

2 Answers 2

3

Extended GNU awk solution:

awk 'BEGIN{ 
         curr_d=strftime("%d/%b/%Y", systime());
         PROCINFO["sorted_in"]="@val_num_desc" 
     }
     $3~"^\\["curr_d{ a[$8][$1]++ } 
     END{ 
         for (cc in a) { 
             for (ip in a[cc]) { 
                 print ip"-"cc":"a[cc][ip]; break 
             }
             if (++c==20) break 
         } 
     }' /var/log/mywebsite.log > /home/mywebsite/www/countries2.txt

For your additional condition "I would like to sort count of ip address last digits which is after country prefix e.g IN:20 ":

remove the line if (++c==20) break and add | sort -t: -k2nr | head -n20 to the end of the command.

7
  • when I run this script it gives me all ip addresses listed in log files however what I would like this script does is to sort as count big to small count of ip address and only pull first 20 records If you look at example I sorted 196.249.65.222-TZ:20 157.39.204.147-ID:18 77.75.76.160-CZ:16 87.98.227.4-ES:11 196.190.64.15-ET:10 197.149.221.194-GN:5 66.249.93.84-EU:4 46.8.37.254-GR:4 212.25.107.145-IL:4 197.248.60.227-KE:4 14.139.191.225-IN:3 96.9.67.84-KH:3 105.158.149.195-MA:2 37.238.135.28-IQ:2 188.0.249.246-MD:2 188.210.148.140-IR:2 196.27.120.7-ZW:2 217.73.208.143-IT:1
    – danone
    Nov 27, 2017 at 11:33
  • @danone, use my update Nov 27, 2017 at 11:33
  • it is giving me 20 record yes but is not sorted from highest to lowest record I want to sort first highest 20 ip address record then cut first 20 did you noticed how its been sorted ? e.g 91.151.138.133-GE:18 196.249.65.222-TZ:16 157.39.204.147-ID:14 77.75.76.160-CZ:10 87.98.227.4-ES:8 196.190.64.15-ET:5 197.149.221.194-GN:4 66.249.93.84-EU:4 46.8.37.254-GR:4 212.25.107.145-IL:3 197.248.60.227-KE:3 14.139.191.225-IN:3 96.9.67.84-KH:3 105.158.149.195-MA:2 37.238.135.28-IQ:2 188.0.249.246-MD:2 188.210.148.140-IR:2 196.27.120.7-ZW:2 217.73.208.143-IT:2 116.44.86.58-KR:2
    – danone
    Nov 27, 2017 at 11:37
  • @danone, your current input is not representative to reflect your requirements. It works fine, tested on input fragment from the question (though that input does not even contain the current day) Nov 27, 2017 at 11:39
  • I sorted expected output please check again. I would like to sort count of ip address last digits which is after country prefix e.g IN:20 how can I do that ? I want only biggest 20 numbers IP-Country:19 IP-Country:17 IP-Country:15 IP-Country:13 IP-Country:10 IP-Country:8 IP-Country:8 IP-Country:5 IP-Country:5 etc. – danone 6 mins ago
    – danone
    Nov 27, 2017 at 11:55
0
#!/bin/bash
i=`date +%d/%b/%Y -d "1 days ago"`
awk -v i="$i" '$3~i {print $1,$NF}' /tmp/l.txt  >/tmp/ToTAL_VISITED_SITES
awk -v i="$i" '$3~i {print $1}' /tmp/l.txt| sort | uniq >/tmp/Sorted_Ips.txt
while read line1; do grep $line1 ToTAL_VISITED_SITES | uniq -c;done <Sorted_Ips.txt

/tmp/l.txt contains the input file. From above script provide individual sites visited count today. If you need top 20 sites Use below command:

sh scriptname | sort -nr| head -20

It worked fine for me.

3
  • 1
    I’ve downvoted your answer because you’ve continued to put prose in the code box. Please review the formatting options and corrections that I and others have made to your previous posts in order to make them more readable. Let me know if/when you have corrected this post and would be happy to undo or reverse my vote.
    – Jeff Schaller
    Nov 27, 2017 at 10:36
  • @JeffSchaller Please suggest to me to how to post in correct format. i am unable to do Nov 27, 2017 at 10:38
  • Look at your previous answers!
    – Jeff Schaller
    Nov 27, 2017 at 10:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.