GIT – A lesser known capability of ’s CURL extension is the ability to perform multiple non-blocking requests in parallel using the multi_curl_* function family. Say we had an array of URLs we wanted to retrieve, using standard CURL we would:

$responses = array();
foreach ($urls as $key=>$url) {
  $ch = curl_init();
  curl_setopt($ch,CURLOPT_URL,$url);
  curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
  $responses[$key] = curl_exec($ch);
}

These are blocking requests: the external requests will be executed serially one after each other and the total execution time will be the sum of the request durations.

This task can be completed significantly faster if you execute more than one request in parallel. The core concept is maintaining an extra entity: the request queue. After the CURL request definition, instead of executing it immediately you push it into a queue. This queue manages the request execution (in parallel) and you ask the queue for the current status of the requests (have they completed?).

$queue = curl_multi_init();
$map = array();

foreach ($urls as $key=>$url) {
  $ch = curl_init();
  curl_setopt($ch,CURLOPT_URL,$url);
  curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);
  curl_multi_add_handle($queue,$ch);
  $map[(string) $ch] = $key;
}

// if you have something else to do, do it 
// here while the requests are executing...

$responses = array();
do {
  $code = curl_multi_exec($queue,$still_executing);
  if ($code==CURLM_OK) { 
    // something ready
    while ($done=curl_multi_info_read($queue)) {
      $ch = $done['handle'];
      $key = $map[(string) $ch];
      $responses[$key] = curl_multi_getcontent($ch);
      curl_multi_remove_handle($queue,$ch);
    }
  } elseif ($code!=CURLM_CALL_MULTI_PERFORM) { 
    // something wrong
    throw new RuntimeException("multi_curl failure [$code]");
  }
} while ($still_executing);

Note how we first define our CURL requests and then add them all to the queue. We then wait on the queue, picking up requests as they finish. Instead of curl_exec blocking for us, we are do the blocking ourselves in the do ... while loop. Because we are blocking ourselves, we have flexibility: other tasks can be done while we wait for the requests to finish. For example we could perform expensive post-processing of a request while we were waiting for others to finish downloading.

Print Friendly

Comments

comments

Bài viết liên quan