The advantages of asynchronous network requests (or any high latency requests) over synchronous is easy to explain with a little example. However as I’ll explain at the end, my little example hit a problem.
Doing the requests in a synchronous manner, one after the other, means you make the first one, wait for its response before moving on to the next and so on until all the requests are completed. The time taken is the sum of all the requests. If you did 20 requests where the longest took 1 second and the average took half a second then doing this synchronously would take 10 seconds.
Doing the same in an asynchronous manner you create a listener or a callback to handle the response then start the first request followed immediately by the second and so on without waiting for a response (as you have already created another piece of code to handle the responses). This should only take as long as the longest response, plus a little overhead, or about 1 second in our example above.
However if you just have a batch of HTTP based requests you want to run asynchronously, Kenneth Reitz, who wrote the excellent requests module I used in the Posting to Slack blog entry, has released grequests; basicly a monkey patch for requests to use the gevent module to make asynchronous calls.
You use the same get, post, put, head and delete request functions and returns the same response object. The difference the asynchronous way has an extra line to set up a tuple for handling the requests before calling grequests.map (or imap) to poll until all the requests are complete whereas the synchronous way just maps the get calls directly to the urls. I created a little program to demonstrate this and uploaded it to BitBucket. It makes 10 get requests asynchronously first and then synchronously and displays the timings. Putting the async first should eliminate any possibility requests being faster skewing the results.
So to the problem. The asynchronous calls on my Windows machine did not end up faster, if anything the average was slower. Confused, I tested the same code on a Linux box which produced the expected results; the async completing in a quarter of the time. At a guess it seems there may be a problem either with gevents or the greenlets module grequests depends upon for performance. I will do some more investigation and let you know.