Thursday, October 7, 2010

Bypass Query Over Limit For Geocoder: Google Maps API

Recently I needed to get Longitude/Latitudes of long list of cities.(more than thousand cities)
I used Geocoding API of google maps.  Using this API  , i sent individual requests to google geocoder passing in the name of individual city and storing back the longitude and latitude it returned.

Since i looped over the name of the cities , requests to geocoder were sent too quickly. and i got back the result 'QUERY OVER LIMIT' . Infact google had specified a query rate limit , you cannot send too many requests in quick succession of time. I am not aware what actual rate limit is used by google , but some hit and trial showed me that if number of requests was kept to 15/s then it was fine.

So i needed a way to sleep my loop after every 10 iterations (using 10/second rate ) . Now first thing that comes to your mind is 'hey use sleep function ' . Ah i wish i could but javascript supports no 'sleep' or 'wait' functions.

So what then? Use setTimeout("function name" , time in milliseconds) . it calls the 'function name' after 'time in milliseconds' expires.
counter=1; 
mySleepFunction= function ()
{
    var geocoder=new google.maps.Geocoder();
    var cityList=["NewYork","Tokyo","Beijing","London","Paris","Berlin","Warsaw","Helsinki","Stockholm",,"Lahore","Islamabad",,"Karachi","Faisalabad",,"Multan","Peshawar",,"Quetta","Gwadar"]; 
    for(;counter<= cityList.length ; counter++)
    {
        if(counter %10 == 0)
        {
              counter++;
              setTimeout("mySleepFunction()",5000);
              break;
        }
        if (geocoder)
        {
           geocoder.geocode({ 'address': cityList[counter-1] }, function (results, status)
           {
               if (status == google.maps.GeocoderStatus.OK)
               {
                  console.log(results[0].address_components[0].long_name+","+results[0].geometry.location.lat()+","+results[0].geometry.location.lng());
                  // Parse return results here , google sends back xml/json and you can extract longitude/latitude 
               }
               else
               {
                  //do whatever you like
               }
            });  
        }
    }
}

Here is the result of above script:


So after every ten iterations , function sets timer for calling itself after 5secs until whole list is processed. I had a list of around 1000 cities and it took me around 15mins to get all the results .



Beware though, Google won't allow more than 2500 requests from one PC/user/client/IP address(??) in 24hrs.

No comments:

Post a Comment