Requests_Transport_cURL - Code Metrics - Inspection of "WIP - Fixes #283 : Fixes CPU busy loop when using..." - rmccue/Requests - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Pull Request — master (#284)

unknown

created 2019-10-23 20:54 UTC

Requests_Transport_cURL F

↳ Parent: Project

Complexity

Total Complexity

Size/Duplication

Total Lines	563
Duplicated Lines	1.6 %

Coupling/Cohesion

Components	1
Dependencies	4

Importance

Changes

Metric	Value
dl	9
loc	563
rs	2.4
c	0
b	0
f	0
wmc	75
lcom	1
cbo	4

11 Methods

Rating	Name	Duplication	Size	Complexity
A	__construct()	0	17	4
A	__destruct()	0	5	2
B	request()	3	56	10
D	request_multiple()	3	114	14
A	get_subrequest_handle()	3	17	3
F	setup_handle()	0	85	20
A	process_response()	0	27	4
A	stream_headers()	0	15	3
A	stream_body()	0	28	5
A	format_get()	0	22	4
A	test()	0	15	6

How to fix Duplicated Code Complexity

Duplicated Code

Duplicate code is one of the most pungent code smells. A rule that is often used is to re-structure code once it is duplicated in three or more places.

Common duplication problems, and corresponding solutions are:

If you have the same expression in different places: Extract expression to a method
If you have the same method in different sub-classes: Extract method, and pull up field to the parent class
If you have the same code in unrelated classes: Consider extracting the code to a new class, and injecting that class

Complex Class

Tip: Before tackling complexity, make sure that you eliminate any duplication first. This often can reduce the size of classes significantly.

Complex classes like Requests_Transport_cURL often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes. You can also have a look at the cohesion graph to spot any un-connected, or weakly-connected components.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

While breaking up the class, it is a good idea to analyze how other classes use Requests_Transport_cURL, and based on these observations, apply Extract Interface, too.

<?php
/**
 * cURL HTTP transport
 *
 * @package Requests
 * @subpackage Transport
 */

/**
 * cURL HTTP transport
 *
 * @package Requests
 * @subpackage Transport
 */
class Requests_Transport_cURL implements Requests_Transport {
	const CURL_7_10_5 = 0x070A05;
	const CURL_7_16_2 = 0x071002;

	/**
	 * Raw HTTP data
	 *
	 * @var string
	 */
	public $headers = '';

	/**
	 * Raw body data
	 *
	 * @var string
	 */
	public $response_data = '';

	/**
	 * Information on the current request
	 *
	 * @var array cURL information array, see {@see https://secure.php.net/curl_getinfo}
	 */
	public $info;

	/**
	 * Version string
	 *
	 * @var long
	 */
	public $version;

	/**
	 * cURL handle
	 *
	 * @var resource
	 */
	protected $handle;

	/**
	 * Hook dispatcher instance
	 *
	 * @var Requests_Hooks
	 */
	protected $hooks;

	/**
	 * Have we finished the headers yet?
	 *
	 * @var boolean
	 */
	protected $done_headers = false;

	/**
	 * If streaming to a file, keep the file pointer
	 *
	 * @var resource
	 */
	protected $stream_handle;

	/**
	 * How many bytes are in the response body?
	 *
	 * @var int
	 */
	protected $response_bytes;

	/**
	 * What's the maximum number of bytes we should keep?
	 *
	 * @var int|bool Byte count, or false if no limit.
	 */
	protected $response_byte_limit;

	/**
	 * Constructor
	 */
	public function __construct() {
		$curl = curl_version();
		$this->version = $curl['version_number'];
		$this->handle = curl_init();

		curl_setopt($this->handle, CURLOPT_HEADER, false);
		curl_setopt($this->handle, CURLOPT_RETURNTRANSFER, 1);
		if ($this->version >= self::CURL_7_10_5) {
			curl_setopt($this->handle, CURLOPT_ENCODING, '');
		}
		if (defined('CURLOPT_PROTOCOLS')) {
			curl_setopt($this->handle, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
		}
		if (defined('CURLOPT_REDIR_PROTOCOLS')) {
			curl_setopt($this->handle, CURLOPT_REDIR_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
		}
	}

	/**
	 * Destructor
	 */
	public function __destruct() {
		if (is_resource($this->handle)) {
			curl_close($this->handle);
		}
	}

	/**
	 * Perform a request
	 *
	 * @throws Requests_Exception On a cURL error (`curlerror`)
	 *
	 * @param string $url URL to request
	 * @param array $headers Associative array of request headers
	 * @param string|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
	 * @param array $options Request options, see {@see Requests::response()} for documentation
	 * @return string Raw HTTP result
	 */
	public function request($url, $headers = array(), $data = array(), $options = array()) {
		$this->hooks = $options['hooks'];

		$this->setup_handle($url, $headers, $data, $options);

		$options['hooks']->dispatch('curl.before_send', array(&$this->handle));

		if ($options['filename'] !== false) {
			$this->stream_handle = fopen($options['filename'], 'wb');
		}

		$this->response_data = '';
		$this->response_bytes = 0;
		$this->response_byte_limit = false;
		if ($options['max_bytes'] !== false) {
			$this->response_byte_limit = $options['max_bytes'];
		}

		if (isset($options['verify'])) {
			if ($options['verify'] === false) {
				curl_setopt($this->handle, CURLOPT_SSL_VERIFYHOST, 0);
				curl_setopt($this->handle, CURLOPT_SSL_VERIFYPEER, 0);
			}
			elseif (is_string($options['verify'])) {
				curl_setopt($this->handle, CURLOPT_CAINFO, $options['verify']);
			}
		}

		if (isset($options['verifyname']) && $options['verifyname'] === false) {
			curl_setopt($this->handle, CURLOPT_SSL_VERIFYHOST, 0);
		}

		curl_exec($this->handle);
		$response = $this->response_data;

		$options['hooks']->dispatch('curl.after_send', array());

		if (curl_errno($this->handle) === 23 || curl_errno($this->handle) === 61) {
			// Reset encoding and try again
			curl_setopt($this->handle, CURLOPT_ENCODING, 'none');

			$this->response_data = '';
			$this->response_bytes = 0;
			curl_exec($this->handle);
			$response = $this->response_data;
		}

		$this->process_response($response, $options);

		// Need to remove the $this reference from the curl handle.
		// Otherwise Requests_Transport_cURL wont be garbage collected and the curl_close() will never be called.
		curl_setopt($this->handle, CURLOPT_HEADERFUNCTION, null);
		curl_setopt($this->handle, CURLOPT_WRITEFUNCTION, null);

		return $this->headers;
	}

	/**
	 * Send multiple requests simultaneously
	 *
	 * @param array $requests Request data
	 * @param array $options Global options
	 * @return array Array of Requests_Response objects (may contain Requests_Exception or string responses as well)
	 */
	public function request_multiple($requests, $options) {
		// If you're not requesting, we can't get any responses ¯\_(ツ)_/¯
		if (empty($requests)) {
			return array();
		}

		$multihandle = curl_multi_init();
		$subrequests = array();
		$subhandles = array();

		$class = get_class($this);
		foreach ($requests as $id => $request) {
			$subrequests[$id] = new $class();
			$subhandles[$id] = $subrequests[$id]->get_subrequest_handle($request['url'], $request['headers'], $request['data'], $request['options']);
			$request['options']['hooks']->dispatch('curl.before_multi_add', array(&$subhandles[$id]));
			curl_multi_add_handle($multihandle, $subhandles[$id]);
		}

		$completed = 0;
		$responses = array();

		$request['options']['hooks']->dispatch('curl.before_multi_exec', array(&$multihandle));
foreach ($a as $b) {
}

// $b is defined here only if $a has elements, for example if $a is array()
// then $b would not be defined here. To avoid that, we recommend to set a
// default value for $b.


// Better
$b = 0; // or whatever default makes sense in your context
foreach ($a as $b) {
}

// $b is now guaranteed to be defined here.

		$made_progress = true;
		$all_requests_start = microtime(true);
		$microseconds_taken_by_last_curl_operation = 0;
		do {
			$active = false;

			if (!$made_progress) {
				// For a small fraction of slow requests, curl_multi_select will busy loop and return immediately with no indication of error (it returns 0 immediately with/without setting curl_multi_errno).
				// (https://github.com/rmccue/Requests/pull/284)
				// To mitigate this, add our own sleep if curl returned but no requests completed (failing or succeeding)
				// - The amount of time to sleep is the smaller of 1ms or 10% of total time spent waiting for curl requests
				//   (10% was arbitrarily chosen to slowing down requests that would normally take less than 1 millisecond)
				// - The amount of time that was already spent in curl_multi_exec+curl_multi_select in the previous request is subtracted from that.
				//   (E.g. if curl_multi_select already slept for 1 milliseconds, curl_multi_select might be operating normally)
				$elapsed_microseconds = (microtime(true) - $all_requests_start) * 1000000;
				$microseconds_to_sleep = min($elapsed_microseconds / 10, 1000) - $microseconds_taken_by_last_curl_operation;
				if ($microseconds_to_sleep >= 1) {
					usleep($microseconds_to_sleep);
				}
			}
			$operation_start = microtime(true);
			$is_first_multi_exec = true;
			do {
				if (!$is_first_multi_exec) {
					// Sleep for 1 millisecond to avoid a busy loop that would chew CPU
					// See ORA2PG-498
					usleep(1000);
				}
				$status = curl_multi_exec($multihandle, $active);
				$is_first_multi_exec = false;
			}
			while ($status === CURLM_CALL_MULTI_PERFORM);

			// curl_multi_select will sleep for at most 50 milliseconds before returning.
			// Note that in php 7.1.23+, curl_multi_select can return immediately but return 0 with no error in some edge cases.
			$select_result = curl_multi_select($multihandle, 0.05);
			// A return value of 0 means that nothing interesting happened.
			// A return value of -1 means that either (1) nothing interesting happened or (2) there was an error.
			// A return value of 1 or more means that interesting things happened to that many connections.
			$made_progress = $select_result > 0;

			$to_process = array();

			// Read the information as needed
			while ($done = curl_multi_info_read($multihandle)) {
				$key = array_search($done['handle'], $subhandles, true);
				if (!isset($to_process[$key])) {
					$to_process[$key] = $done;
				}
			}

			// Parse the finished requests before we start getting the new ones
			foreach ($to_process as $key => $done) {
				$made_progress = true;  // Set this just in case progress was made despite curl_multi_select saying otherwise.
				$options = $requests[$key]['options'];
				if (CURLE_OK !== $done['result']) {
					//get error string for handle.
					$reason = curl_error($done['handle']);
					$exception = new Requests_Exception_Transport_cURL(
									$reason,
									Requests_Exception_Transport_cURL::EASY,
									$done['handle'],
									$done['result']
								);
					$responses[$key] = $exception;
					$options['hooks']->dispatch('transport.internal.parse_error', array(&$responses[$key], $requests[$key]));
				}
				else {
					$responses[$key] = $subrequests[$key]->process_response($subrequests[$key]->response_data, $options);

					$options['hooks']->dispatch('transport.internal.parse_response', array(&$responses[$key], $requests[$key]));
				}

				curl_multi_remove_handle($multihandle, $done['handle']);
				curl_close($done['handle']);

				if (!is_string($responses[$key])) {
					$options['hooks']->dispatch('multiple.request.complete', array(&$responses[$key], $key));
				}
				$completed++;
			}
			$microseconds_taken_by_last_curl_operation = (microtime(true) - $operation_start) * 100000;
		}
		while ($active || $completed < count($subrequests));

		$request['options']['hooks']->dispatch('curl.after_multi_exec', array(&$multihandle));

		curl_multi_close($multihandle);

		return $responses;
	}

	/**
	 * Get the cURL handle for use in a multi-request
	 *
	 * @param string $url URL to request
	 * @param array $headers Associative array of request headers
	 * @param string|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
	 * @param array $options Request options, see {@see Requests::response()} for documentation
	 * @return resource Subrequest's cURL handle
	 */
	public function &get_subrequest_handle($url, $headers, $data, $options) {
		$this->setup_handle($url, $headers, $data, $options);

		if ($options['filename'] !== false) {
			$this->stream_handle = fopen($options['filename'], 'wb');
		}

		$this->response_data = '';
		$this->response_bytes = 0;
		$this->response_byte_limit = false;
		if ($options['max_bytes'] !== false) {
			$this->response_byte_limit = $options['max_bytes'];
		}
		$this->hooks = $options['hooks'];

		return $this->handle;
	}

	/**
	 * Setup the cURL handle for the given data
	 *
	 * @param string $url URL to request
	 * @param array $headers Associative array of request headers
	 * @param string|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
	 * @param array $options Request options, see {@see Requests::response()} for documentation
	 */
	protected function setup_handle($url, $headers, $data, $options) {
		$options['hooks']->dispatch('curl.before_request', array(&$this->handle));

		// Force closing the connection for old versions of cURL (<7.22).
		if ( ! isset( $headers['Connection'] ) ) {
			$headers['Connection'] = 'close';
		}

		$headers = Requests::flatten($headers);

		if (!empty($data)) {
			$data_format = $options['data_format'];

			if ($data_format === 'query') {
				$url = self::format_get($url, $data);

				$data = '';
			}
			elseif (!is_string($data)) {
				$data = http_build_query($data, null, '&');
			}
		}

		switch ($options['type']) {
			case Requests::POST:
				curl_setopt($this->handle, CURLOPT_POST, true);
				curl_setopt($this->handle, CURLOPT_POSTFIELDS, $data);
				break;
			case Requests::HEAD:
				curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
				curl_setopt($this->handle, CURLOPT_NOBODY, true);
				break;
			case Requests::TRACE:
				curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
				break;
			case Requests::PATCH:
			case Requests::PUT:
			case Requests::DELETE:
			case Requests::OPTIONS:
			default:
				curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
				if (!empty($data)) {
					curl_setopt($this->handle, CURLOPT_POSTFIELDS, $data);
				}
		}

		// cURL requires a minimum timeout of 1 second when using the system
		// DNS resolver, as it uses `alarm()`, which is second resolution only.
		// There's no way to detect which DNS resolver is being used from our
		// end, so we need to round up regardless of the supplied timeout.
		//
		// https://github.com/curl/curl/blob/4f45240bc84a9aa648c8f7243be7b79e9f9323a5/lib/hostip.c#L606-L609
		$timeout = max($options['timeout'], 1);

		if (is_int($timeout) || $this->version < self::CURL_7_16_2) {
			curl_setopt($this->handle, CURLOPT_TIMEOUT, ceil($timeout));
		}
		else {
			curl_setopt($this->handle, CURLOPT_TIMEOUT_MS, round($timeout * 1000));
		}

		if (is_int($options['connect_timeout']) || $this->version < self::CURL_7_16_2) {
			curl_setopt($this->handle, CURLOPT_CONNECTTIMEOUT, ceil($options['connect_timeout']));
		}
		else {
			curl_setopt($this->handle, CURLOPT_CONNECTTIMEOUT_MS, round($options['connect_timeout'] * 1000));
		}
		curl_setopt($this->handle, CURLOPT_URL, $url);
		curl_setopt($this->handle, CURLOPT_REFERER, $url);
		curl_setopt($this->handle, CURLOPT_USERAGENT, $options['useragent']);
		if (!empty($headers)) {
			curl_setopt($this->handle, CURLOPT_HTTPHEADER, $headers);
		}
		if ($options['protocol_version'] === 1.1) {
			curl_setopt($this->handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
		}
		else {
			curl_setopt($this->handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
		}

		if (true === $options['blocking']) {
			curl_setopt($this->handle, CURLOPT_HEADERFUNCTION, array(&$this, 'stream_headers'));
			curl_setopt($this->handle, CURLOPT_WRITEFUNCTION, array(&$this, 'stream_body'));
			curl_setopt($this->handle, CURLOPT_BUFFERSIZE, Requests::BUFFER_SIZE);
		}
	}

	/**
	 * Process a response
	 *
	 * @param string $response Response data from the body
	 * @param array $options Request options
	 * @return string HTTP response data including headers
	 */
	public function process_response($response, $options) {
		if ($options['blocking'] === false) {
			$fake_headers = '';
			$options['hooks']->dispatch('curl.after_request', array(&$fake_headers));
			return false;
		}
		if ($options['filename'] !== false) {
			fclose($this->stream_handle);
			$this->headers = trim($this->headers);
		}
		else {
			$this->headers .= $response;
		}

		if (curl_errno($this->handle)) {
			$error = sprintf(
				'cURL error %s: %s',
				curl_errno($this->handle),
				curl_error($this->handle)
			);
			throw new Requests_Exception($error, 'curlerror', $this->handle);
		}
		$this->info = curl_getinfo($this->handle);

		$options['hooks']->dispatch('curl.after_request', array(&$this->headers, &$this->info));
		return $this->headers;
	}

	/**
	 * Collect the headers as they are received
	 *
	 * @param resource $handle cURL resource
	 * @param string $headers Header string
	 * @return integer Length of provided header
	 */
	public function stream_headers($handle, $headers) {
		// Why do we do this? cURL will send both the final response and any
		// interim responses, such as a 100 Continue. We don't need that.
		// (We may want to keep this somewhere just in case)
		if ($this->done_headers) {
			$this->headers = '';
			$this->done_headers = false;
		}
		$this->headers .= $headers;

		if ($headers === "\r\n") {
			$this->done_headers = true;
		}
		return strlen($headers);
	}

	/**
	 * Collect data as it's received
	 *
	 * @since 1.6.1
	 *
	 * @param resource $handle cURL resource
	 * @param string $data Body data
	 * @return integer Length of provided data
	 */
	public function stream_body($handle, $data) {
		$this->hooks->dispatch('request.progress', array($data, $this->response_bytes, $this->response_byte_limit));
		$data_length = strlen($data);

		// Are we limiting the response size?
		if ($this->response_byte_limit) {
			if ($this->response_bytes === $this->response_byte_limit) {
				// Already at maximum, move on
				return $data_length;
			}

			if (($this->response_bytes + $data_length) > $this->response_byte_limit) {
				// Limit the length
				$limited_length = ($this->response_byte_limit - $this->response_bytes);
				$data = substr($data, 0, $limited_length);
			}
		}

		if ($this->stream_handle) {
			fwrite($this->stream_handle, $data);
		}
		else {
			$this->response_data .= $data;
		}

		$this->response_bytes += strlen($data);
		return $data_length;
	}

	/**
	 * Format a URL given GET data
	 *
	 * @param string $url
	 * @param array|object $data Data to build query using, see {@see https://secure.php.net/http_build_query}
	 * @return string URL with data
	 */
	protected static function format_get($url, $data) {
		if (!empty($data)) {
			$url_parts = parse_url($url);
			if (empty($url_parts['query'])) {
				$query = $url_parts['query'] = '';
			}
			else {
				$query = $url_parts['query'];
			}

			$query .= '&' . http_build_query($data, null, '&');
			$query = trim($query, '&');

			if (empty($url_parts['query'])) {
				$url .= '?' . $query;
			}
			else {
				$url = str_replace($url_parts['query'], $query, $url);
			}
		}
		return $url;
	}

	/**
	 * Whether this transport is valid
	 *
	 * @codeCoverageIgnore
	 * @return boolean True if the transport is valid, false otherwise.
	 */
	public static function test($capabilities = array()) {
		if (!function_exists('curl_init') || !function_exists('curl_exec')) {
			return false;
		}

		// If needed, check that our installed curl version supports SSL
		if (isset($capabilities['ssl']) && $capabilities['ssl']) {
			$curl_version = curl_version();
			if (!(CURL_VERSION_SSL & $curl_version['features'])) {
				return false;
			}
		}

		return true;
	}
}


1		<?php
2		/**
3		* cURL HTTP transport
4		*
5		* @package Requests
6		* @subpackage Transport
7		*/
8
9		/**
10		* cURL HTTP transport
11		*
12		* @package Requests
13		* @subpackage Transport
14		*/
15		class Requests_Transport_cURL implements Requests_Transport {
16		const CURL_7_10_5 = 0x070A05;
17		const CURL_7_16_2 = 0x071002;
18
19		/**
20		* Raw HTTP data
21		*
22		* @var string
23		*/
24		public $headers = '';
25
26		/**
27		* Raw body data
28		*
29		* @var string
30		*/
31		public $response_data = '';
32
33		/**
34		* Information on the current request
35		*
36		* @var array cURL information array, see {@see https://secure.php.net/curl_getinfo}
37		*/
38		public $info;
39
40		/**
41		* Version string
42		*
43		* @var long
44		*/
45		public $version;
46
47		/**
48		* cURL handle
49		*
50		* @var resource
51		*/
52		protected $handle;
53
54		/**
55		* Hook dispatcher instance
56		*
57		* @var Requests_Hooks
58		*/
59		protected $hooks;
60
61		/**
62		* Have we finished the headers yet?
63		*
64		* @var boolean
65		*/
66		protected $done_headers = false;
67
68		/**
69		* If streaming to a file, keep the file pointer
70		*
71		* @var resource
72		*/
73		protected $stream_handle;
74
75		/**
76		* How many bytes are in the response body?
77		*
78		* @var int
79		*/
80		protected $response_bytes;
81
82		/**
83		* What's the maximum number of bytes we should keep?
84		*
85		* @var int\|bool Byte count, or false if no limit.
86		*/
87		protected $response_byte_limit;
88
89		/**
90		* Constructor
91		*/
92		public function __construct() {
93		$curl = curl_version();
94		$this->version = $curl['version_number'];
95		$this->handle = curl_init();
96
97		curl_setopt($this->handle, CURLOPT_HEADER, false);
98		curl_setopt($this->handle, CURLOPT_RETURNTRANSFER, 1);
99		if ($this->version >= self::CURL_7_10_5) {
100		curl_setopt($this->handle, CURLOPT_ENCODING, '');
101		}
102		if (defined('CURLOPT_PROTOCOLS')) {
103		curl_setopt($this->handle, CURLOPT_PROTOCOLS, CURLPROTO_HTTP \| CURLPROTO_HTTPS);
104		}
105		if (defined('CURLOPT_REDIR_PROTOCOLS')) {
106		curl_setopt($this->handle, CURLOPT_REDIR_PROTOCOLS, CURLPROTO_HTTP \| CURLPROTO_HTTPS);
107		}
108		}
109
110		/**
111		* Destructor
112		*/
113		public function __destruct() {
114		if (is_resource($this->handle)) {
115		curl_close($this->handle);
116		}
117		}
118
119		/**
120		* Perform a request
121		*
122		* @throws Requests_Exception On a cURL error (`curlerror`)
123		*
124		* @param string $url URL to request
125		* @param array $headers Associative array of request headers
126		* @param string\|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
127		* @param array $options Request options, see {@see Requests::response()} for documentation
128		* @return string Raw HTTP result
129		*/
130		public function request($url, $headers = array(), $data = array(), $options = array()) {
131		$this->hooks = $options['hooks'];
132
133		$this->setup_handle($url, $headers, $data, $options);
134
135		$options['hooks']->dispatch('curl.before_send', array(&$this->handle));
136
137	View Code Duplication	if ($options['filename'] !== false) {
138		$this->stream_handle = fopen($options['filename'], 'wb');
139		}
140
141		$this->response_data = '';
142		$this->response_bytes = 0;
143		$this->response_byte_limit = false;
144		if ($options['max_bytes'] !== false) {
145		$this->response_byte_limit = $options['max_bytes'];
146		}
147
148		if (isset($options['verify'])) {
149		if ($options['verify'] === false) {
150		curl_setopt($this->handle, CURLOPT_SSL_VERIFYHOST, 0);
151		curl_setopt($this->handle, CURLOPT_SSL_VERIFYPEER, 0);
152		}
153		elseif (is_string($options['verify'])) {
154		curl_setopt($this->handle, CURLOPT_CAINFO, $options['verify']);
155		}
156		}
157
158		if (isset($options['verifyname']) && $options['verifyname'] === false) {
159		curl_setopt($this->handle, CURLOPT_SSL_VERIFYHOST, 0);
160		}
161
162		curl_exec($this->handle);
163		$response = $this->response_data;
164
165		$options['hooks']->dispatch('curl.after_send', array());
166
167		if (curl_errno($this->handle) === 23 \|\| curl_errno($this->handle) === 61) {
168		// Reset encoding and try again
169		curl_setopt($this->handle, CURLOPT_ENCODING, 'none');
170
171		$this->response_data = '';
172		$this->response_bytes = 0;
173		curl_exec($this->handle);
174		$response = $this->response_data;
175		}
176
177		$this->process_response($response, $options);
178
179		// Need to remove the $this reference from the curl handle.
180		// Otherwise Requests_Transport_cURL wont be garbage collected and the curl_close() will never be called.
181		curl_setopt($this->handle, CURLOPT_HEADERFUNCTION, null);
182		curl_setopt($this->handle, CURLOPT_WRITEFUNCTION, null);
183
184		return $this->headers;
185		}
186
187		/**
188		* Send multiple requests simultaneously
189		*
190		* @param array $requests Request data
191		* @param array $options Global options
192		* @return array Array of Requests_Response objects (may contain Requests_Exception or string responses as well)
193		*/
194		public function request_multiple($requests, $options) {
195		// If you're not requesting, we can't get any responses ¯\_(ツ)_/¯
196		if (empty($requests)) {
197		return array();
198		}
199
200		$multihandle = curl_multi_init();
201		$subrequests = array();
202		$subhandles = array();
203
204		$class = get_class($this);
205		foreach ($requests as $id => $request) {
206		$subrequests[$id] = new $class();
207		$subhandles[$id] = $subrequests[$id]->get_subrequest_handle($request['url'], $request['headers'], $request['data'], $request['options']);
208		$request['options']['hooks']->dispatch('curl.before_multi_add', array(&$subhandles[$id]));
209		curl_multi_add_handle($multihandle, $subhandles[$id]);
210		}
211
212		$completed = 0;
213		$responses = array();
214
215		$request['options']['hooks']->dispatch('curl.before_multi_exec', array(&$multihandle));
		0 ignored issues – show Bug introduced 2016-02-09 23:07 UTC by Report Bug Copy Issue Report The variable `$request` seems to be defined by a `foreach` iteration on line `205`. Are you sure the iterator is never empty, otherwise this variable is not defined? It seems like you are relying on a variable being defined by an iteration: foreach ($a as $b) { } // $b is defined here only if $a has elements, for example if $a is array() // then $b would not be defined here. To avoid that, we recommend to set a // default value for $b. // Better $b = 0; // or whatever default makes sense in your context foreach ($a as $b) { } // $b is now guaranteed to be defined here. Loading history...
216
217		$made_progress = true;
218		$all_requests_start = microtime(true);
219		$microseconds_taken_by_last_curl_operation = 0;
220		do {
221		$active = false;
222
223		if (!$made_progress) {
224		// For a small fraction of slow requests, curl_multi_select will busy loop and return immediately with no indication of error (it returns 0 immediately with/without setting curl_multi_errno).
225		// (https://github.com/rmccue/Requests/pull/284)
226		// To mitigate this, add our own sleep if curl returned but no requests completed (failing or succeeding)
227		// - The amount of time to sleep is the smaller of 1ms or 10% of total time spent waiting for curl requests
228		// (10% was arbitrarily chosen to slowing down requests that would normally take less than 1 millisecond)
229		// - The amount of time that was already spent in curl_multi_exec+curl_multi_select in the previous request is subtracted from that.
230		// (E.g. if curl_multi_select already slept for 1 milliseconds, curl_multi_select might be operating normally)
231		$elapsed_microseconds = (microtime(true) - $all_requests_start) * 1000000;
232		$microseconds_to_sleep = min($elapsed_microseconds / 10, 1000) - $microseconds_taken_by_last_curl_operation;
233		if ($microseconds_to_sleep >= 1) {
234		usleep($microseconds_to_sleep);
235		}
236		}
237		$operation_start = microtime(true);
238		$is_first_multi_exec = true;
239		do {
240		if (!$is_first_multi_exec) {
241		// Sleep for 1 millisecond to avoid a busy loop that would chew CPU
242		// See ORA2PG-498
243		usleep(1000);
244		}
245		$status = curl_multi_exec($multihandle, $active);
246		$is_first_multi_exec = false;
247		}
248		while ($status === CURLM_CALL_MULTI_PERFORM);
249
250		// curl_multi_select will sleep for at most 50 milliseconds before returning.
251		// Note that in php 7.1.23+, curl_multi_select can return immediately but return 0 with no error in some edge cases.
252		$select_result = curl_multi_select($multihandle, 0.05);
253		// A return value of 0 means that nothing interesting happened.
254		// A return value of -1 means that either (1) nothing interesting happened or (2) there was an error.
255		// A return value of 1 or more means that interesting things happened to that many connections.
256		$made_progress = $select_result > 0;
257
258		$to_process = array();
259
260		// Read the information as needed
261		while ($done = curl_multi_info_read($multihandle)) {
262		$key = array_search($done['handle'], $subhandles, true);
263		if (!isset($to_process[$key])) {
264		$to_process[$key] = $done;
265		}
266		}
267
268		// Parse the finished requests before we start getting the new ones
269		foreach ($to_process as $key => $done) {
270		$made_progress = true; // Set this just in case progress was made despite curl_multi_select saying otherwise.
271		$options = $requests[$key]['options'];
272		if (CURLE_OK !== $done['result']) {
273		//get error string for handle.
274		$reason = curl_error($done['handle']);
275		$exception = new Requests_Exception_Transport_cURL(
276		$reason,
277		Requests_Exception_Transport_cURL::EASY,
278		$done['handle'],
279		$done['result']
280		);
281		$responses[$key] = $exception;
282		$options['hooks']->dispatch('transport.internal.parse_error', array(&$responses[$key], $requests[$key]));
283		}
284		else {
285		$responses[$key] = $subrequests[$key]->process_response($subrequests[$key]->response_data, $options);
286
287		$options['hooks']->dispatch('transport.internal.parse_response', array(&$responses[$key], $requests[$key]));
288		}
289
290		curl_multi_remove_handle($multihandle, $done['handle']);
291		curl_close($done['handle']);
292
293	View Code Duplication	if (!is_string($responses[$key])) {
294		$options['hooks']->dispatch('multiple.request.complete', array(&$responses[$key], $key));
295		}
296		$completed++;
297		}
298		$microseconds_taken_by_last_curl_operation = (microtime(true) - $operation_start) * 100000;
299		}
300		while ($active \|\| $completed < count($subrequests));
301
302		$request['options']['hooks']->dispatch('curl.after_multi_exec', array(&$multihandle));
303
304		curl_multi_close($multihandle);
305
306		return $responses;
307		}
308
309		/**
310		* Get the cURL handle for use in a multi-request
311		*
312		* @param string $url URL to request
313		* @param array $headers Associative array of request headers
314		* @param string\|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
315		* @param array $options Request options, see {@see Requests::response()} for documentation
316		* @return resource Subrequest's cURL handle
317		*/
318		public function &get_subrequest_handle($url, $headers, $data, $options) {
319		$this->setup_handle($url, $headers, $data, $options);
320
321	View Code Duplication	if ($options['filename'] !== false) {
322		$this->stream_handle = fopen($options['filename'], 'wb');
323		}
324
325		$this->response_data = '';
326		$this->response_bytes = 0;
327		$this->response_byte_limit = false;
328		if ($options['max_bytes'] !== false) {
329		$this->response_byte_limit = $options['max_bytes'];
330		}
331		$this->hooks = $options['hooks'];
332
333		return $this->handle;
334		}
335
336		/**
337		* Setup the cURL handle for the given data
338		*
339		* @param string $url URL to request
340		* @param array $headers Associative array of request headers
341		* @param string\|array $data Data to send either as the POST body, or as parameters in the URL for a GET/HEAD
342		* @param array $options Request options, see {@see Requests::response()} for documentation
343		*/
344		protected function setup_handle($url, $headers, $data, $options) {
345		$options['hooks']->dispatch('curl.before_request', array(&$this->handle));
346
347		// Force closing the connection for old versions of cURL (<7.22).
348		if ( ! isset( $headers['Connection'] ) ) {
349		$headers['Connection'] = 'close';
350		}
351
352		$headers = Requests::flatten($headers);
353
354		if (!empty($data)) {
355		$data_format = $options['data_format'];
356
357		if ($data_format === 'query') {
358		$url = self::format_get($url, $data);
		0 ignored issues – show Bug introduced 2016-02-09 23:07 UTC by Report Bug Copy Issue Report It seems like `$data` defined by parameter `$data` on line 344 can also be of type `string`; however, `Requests_Transport_cURL::format_get()` does only seem to accept `array\|object`, maybe add an additional type check? This check looks at variables that have been passed in as parameters and are passed out again to other methods. If the outgoing method call has stricter type requirements than the method itself, an issue is raised. An additional type check may prevent trouble. Loading history...
359		$data = '';
360		}
361		elseif (!is_string($data)) {
362		$data = http_build_query($data, null, '&');
363		}
364		}
365
366		switch ($options['type']) {
367		case Requests::POST:
368		curl_setopt($this->handle, CURLOPT_POST, true);
369		curl_setopt($this->handle, CURLOPT_POSTFIELDS, $data);
370		break;
371		case Requests::HEAD:
372		curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
373		curl_setopt($this->handle, CURLOPT_NOBODY, true);
374		break;
375		case Requests::TRACE:
376		curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
377		break;
378		case Requests::PATCH:
379		case Requests::PUT:
380		case Requests::DELETE:
381		case Requests::OPTIONS:
382		default:
383		curl_setopt($this->handle, CURLOPT_CUSTOMREQUEST, $options['type']);
384		if (!empty($data)) {
385		curl_setopt($this->handle, CURLOPT_POSTFIELDS, $data);
386		}
387		}
388
389		// cURL requires a minimum timeout of 1 second when using the system
390		// DNS resolver, as it uses `alarm()`, which is second resolution only.
391		// There's no way to detect which DNS resolver is being used from our
392		// end, so we need to round up regardless of the supplied timeout.
393		//
394		// https://github.com/curl/curl/blob/4f45240bc84a9aa648c8f7243be7b79e9f9323a5/lib/hostip.c#L606-L609
395		$timeout = max($options['timeout'], 1);
396
397		if (is_int($timeout) \|\| $this->version < self::CURL_7_16_2) {
398		curl_setopt($this->handle, CURLOPT_TIMEOUT, ceil($timeout));
399		}
400		else {
401		curl_setopt($this->handle, CURLOPT_TIMEOUT_MS, round($timeout * 1000));
402		}
403
404		if (is_int($options['connect_timeout']) \|\| $this->version < self::CURL_7_16_2) {
405		curl_setopt($this->handle, CURLOPT_CONNECTTIMEOUT, ceil($options['connect_timeout']));
406		}
407		else {
408		curl_setopt($this->handle, CURLOPT_CONNECTTIMEOUT_MS, round($options['connect_timeout'] * 1000));
409		}
410		curl_setopt($this->handle, CURLOPT_URL, $url);
411		curl_setopt($this->handle, CURLOPT_REFERER, $url);
412		curl_setopt($this->handle, CURLOPT_USERAGENT, $options['useragent']);
413		if (!empty($headers)) {
414		curl_setopt($this->handle, CURLOPT_HTTPHEADER, $headers);
415		}
416		if ($options['protocol_version'] === 1.1) {
417		curl_setopt($this->handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
418		}
419		else {
420		curl_setopt($this->handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
421		}
422
423		if (true === $options['blocking']) {
424		curl_setopt($this->handle, CURLOPT_HEADERFUNCTION, array(&$this, 'stream_headers'));
425		curl_setopt($this->handle, CURLOPT_WRITEFUNCTION, array(&$this, 'stream_body'));
426		curl_setopt($this->handle, CURLOPT_BUFFERSIZE, Requests::BUFFER_SIZE);
427		}
428		}
429
430		/**
431		* Process a response
432		*
433		* @param string $response Response data from the body
434		* @param array $options Request options
435		* @return string HTTP response data including headers
436		*/
437		public function process_response($response, $options) {
438		if ($options['blocking'] === false) {
439		$fake_headers = '';
440		$options['hooks']->dispatch('curl.after_request', array(&$fake_headers));
441		return false;
442		}
443		if ($options['filename'] !== false) {
444		fclose($this->stream_handle);
445		$this->headers = trim($this->headers);
446		}
447		else {
448		$this->headers .= $response;
449		}
450
451		if (curl_errno($this->handle)) {
452		$error = sprintf(
453		'cURL error %s: %s',
454		curl_errno($this->handle),
455		curl_error($this->handle)
456		);
457		throw new Requests_Exception($error, 'curlerror', $this->handle);
458		}
459		$this->info = curl_getinfo($this->handle);
460
461		$options['hooks']->dispatch('curl.after_request', array(&$this->headers, &$this->info));
462		return $this->headers;
463		}
464
465		/**
466		* Collect the headers as they are received
467		*
468		* @param resource $handle cURL resource
469		* @param string $headers Header string
470		* @return integer Length of provided header
471		*/
472		public function stream_headers($handle, $headers) {
473		// Why do we do this? cURL will send both the final response and any
474		// interim responses, such as a 100 Continue. We don't need that.
475		// (We may want to keep this somewhere just in case)
476		if ($this->done_headers) {
477		$this->headers = '';
478		$this->done_headers = false;
479		}
480		$this->headers .= $headers;
481
482		if ($headers === "\r\n") {
483		$this->done_headers = true;
484		}
485		return strlen($headers);
486		}
487
488		/**
489		* Collect data as it's received
490		*
491		* @since 1.6.1
492		*
493		* @param resource $handle cURL resource
494		* @param string $data Body data
495		* @return integer Length of provided data
496		*/
497		public function stream_body($handle, $data) {
498		$this->hooks->dispatch('request.progress', array($data, $this->response_bytes, $this->response_byte_limit));
499		$data_length = strlen($data);
500
501		// Are we limiting the response size?
502		if ($this->response_byte_limit) {
503		if ($this->response_bytes === $this->response_byte_limit) {
504		// Already at maximum, move on
505		return $data_length;
506		}
507
508		if (($this->response_bytes + $data_length) > $this->response_byte_limit) {
509		// Limit the length
510		$limited_length = ($this->response_byte_limit - $this->response_bytes);
511		$data = substr($data, 0, $limited_length);
512		}
513		}
514
515		if ($this->stream_handle) {
516		fwrite($this->stream_handle, $data);
517		}
518		else {
519		$this->response_data .= $data;
520		}
521
522		$this->response_bytes += strlen($data);
523		return $data_length;
524		}
525
526		/**
527		* Format a URL given GET data
528		*
529		* @param string $url
530		* @param array\|object $data Data to build query using, see {@see https://secure.php.net/http_build_query}
531		* @return string URL with data
532		*/
533		protected static function format_get($url, $data) {
534		if (!empty($data)) {
535		$url_parts = parse_url($url);
536		if (empty($url_parts['query'])) {
537		$query = $url_parts['query'] = '';
538		}
539		else {
540		$query = $url_parts['query'];
541		}
542
543		$query .= '&' . http_build_query($data, null, '&');
544		$query = trim($query, '&');
545
546		if (empty($url_parts['query'])) {
547		$url .= '?' . $query;
548		}
549		else {
550		$url = str_replace($url_parts['query'], $query, $url);
551		}
552		}
553		return $url;
554		}
555
556		/**
557		* Whether this transport is valid
558		*
559		* @codeCoverageIgnore
560		* @return boolean True if the transport is valid, false otherwise.
561		*/
562		public static function test($capabilities = array()) {
563		if (!function_exists('curl_init') \|\| !function_exists('curl_exec')) {
564		return false;
565		}
566
567		// If needed, check that our installed curl version supports SSL
568		if (isset($capabilities['ssl']) && $capabilities['ssl']) {
569		$curl_version = curl_version();
570		if (!(CURL_VERSION_SSL & $curl_version['features'])) {
571		return false;
572		}
573		}
574
575		return true;
576		}
577		}
578

rmccue / Requests