0

I have many trajectories data. 500k. I try to get adresses of these coordinates. I use bash command in linux.

#!/bin/bash

filename="$1"

while  read -r line
do
uid=$(echo $line | cut -f1 -d,)
lat=$(echo $line | cut -f2 -d,)
long=$(echo $line | cut -f3 -d,)
timestamp=$(echo $line | cut -f4 -d,)
printf "$uid,$lat,$long,$timestamp|"
wget -O- -q "http://maps.google.com/maps/api/geocode/xml?latlng=$lat,$long&sensor=false"|\
      grep formatted_address|\
      head -n1|\
      cut -d\> -f2|\
      cut -d\< -f1
done < "$filename"

I can only get 2500 addresses. After 2500 coordinates, This command "wget" is not work. When I reset modem, I can get 2500 more addresses. Is there any way to get address from coordinates? Or is it possible to reset IP in bash command after getting 2500 address ? thanks.

My input file include 500k data like this 11395,39.9050283,32.6977617,2008-11-03T16:58:05Z my coordinates 39.9050283,32.6977617

My output file will be like this 11395,39.9050283,32.6977617,2008-11-03T16:58:05Z|11395,39.9051033,32.6984550,2008-11-03T16:58:06Z|Erler, Eskişehir Yolu No:120, 06790 Etimesgut/Ankara, Turkey

4
  • Ahmet, When modem is ON according the DSL service provider will assign the public IP. And when you try wget on google.map, google has a restriction by preventing connection after 2500 hits. Please contact your service provider to reset modem from bash command. It will be vary for each modem. What is your modem type?. – Muralibabud Jan 20 '16 at 8:26
  • airties rt 211.. Is there another way? For example use dns and change adress of dns ? – AhmetMelihBaşbuğ Jan 20 '16 at 8:57
  • i dont think we can reset from command line for arities tr 211, try split 2500 coordinate from 500k file and open 192.168.0.1 from browser login and toosl -> reset modem, use it. – Muralibabud Jan 20 '16 at 9:03
  • 1
    geopy.readthedocs.org/en/1.10.0 . works offline – jijinp Jan 20 '16 at 9:20
2

What's happening is that Google is throttling your API requests. There's a limit to how much data they let you get through a free service. Apparently, after 2500 requests, they cut you off. When you “reset modem”, you evidently get a different IP address from your ISP, so you can make another 2500 requests before you get blocked there.

You need to rethink your strategy. For 500k requests, you may need to pay for a service, or get the data from different sources such as OpenStreetMap. If you get the data from OpenStreetMap, consider that while it's free, they still won't appreciate you making too many requests in a short time. If you need so much data, maybe you should download some databases and do local queries.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.