Completed
Push — renovate/gridicons-3.x ( c004c1...f8ccd4 )
by
unknown
284:06 queued 275:32
created

Queue::fetch_items()   A

Complexity

Conditions 3
Paths 4

Size

Total Lines 16

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
cc 3
nc 4
nop 1
dl 0
loc 16
rs 9.7333
c 0
b 0
f 0
1
<?php
2
3
namespace Automattic\Jetpack\Sync;
4
5
use Automattic\Jetpack\Sync\Defaults;
6
7
/**
8
 * A persistent queue that can be flushed in increments of N items,
9
 * and which blocks reads until checked-out buffers are checked in or
10
 * closed. This uses raw SQL for two reasons: speed, and not triggering
11
 * tons of added_option callbacks.
12
 */
13
class Queue {
14
	public $id;
15
	private $row_iterator;
16
17
	function __construct( $id ) {
18
		$this->id           = str_replace( '-', '_', $id ); // necessary to ensure we don't have ID collisions in the SQL
19
		$this->row_iterator = 0;
20
		$this->random_int   = mt_rand( 1, 1000000 );
0 ignored issues
show
Bug introduced by
The property random_int does not exist. Did you maybe forget to declare it?

In PHP it is possible to write to properties without declaring them. For example, the following is perfectly valid PHP code:

class MyClass { }

$x = new MyClass();
$x->foo = true;

Generally, it is a good practice to explictly declare properties to avoid accidental typos and provide IDE auto-completion:

class MyClass {
    public $foo;
}

$x = new MyClass();
$x->foo = true;
Loading history...
21
	}
22
23
	function add( $item ) {
24
		global $wpdb;
25
		$added = false;
26
		// this basically tries to add the option until enough time has elapsed that
27
		// it has a unique (microtime-based) option key
28
		while ( ! $added ) {
29
			$rows_added = $wpdb->query(
30
				$wpdb->prepare(
31
					"INSERT INTO $wpdb->options (option_name, option_value, autoload) VALUES (%s, %s,%s)",
32
					$this->get_next_data_row_option_name(),
33
					serialize( $item ),
34
					'no'
35
				)
36
			);
37
			$added      = ( 0 !== $rows_added );
38
		}
39
	}
40
41
	// Attempts to insert all the items in a single SQL query. May be subject to query size limits!
42
	function add_all( $items ) {
43
		global $wpdb;
44
		$base_option_name = $this->get_next_data_row_option_name();
45
46
		$query = "INSERT INTO $wpdb->options (option_name, option_value, autoload) VALUES ";
47
48
		$rows = array();
49
50
		for ( $i = 0; $i < count( $items ); $i += 1 ) {
0 ignored issues
show
Performance Best Practice introduced by
It seems like you are calling the size function count() as part of the test condition. You might want to compute the size beforehand, and not on each iteration.

If the size of the collection does not change during the iteration, it is generally a good practice to compute it beforehand, and not on each iteration:

for ($i=0; $i<count($array); $i++) { // calls count() on each iteration
}

// Better
for ($i=0, $c=count($array); $i<$c; $i++) { // calls count() just once
}
Loading history...
51
			$option_name  = esc_sql( $base_option_name . '-' . $i );
52
			$option_value = esc_sql( serialize( $items[ $i ] ) );
53
			$rows[]       = "('$option_name', '$option_value', 'no')";
54
		}
55
56
		$rows_added = $wpdb->query( $query . join( ',', $rows ) );
57
58
		if ( count( $items ) === $rows_added ) {
59
			return new \WP_Error( 'row_count_mismatch', "The number of rows inserted didn't match the size of the input array" );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'row_count_mismatch'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
60
		}
61
	}
62
63
	// Peek at the front-most item on the queue without checking it out
64
	function peek( $count = 1 ) {
65
		$items = $this->fetch_items( $count );
66
		if ( $items ) {
67
			return Utils::get_item_values( $items );
68
		}
69
70
		return array();
71
	}
72
73
	// lag is the difference in time between the age of the oldest item
74
	// (aka first or frontmost item) and the current time
75
	function lag( $now = null ) {
76
		global $wpdb;
77
78
		$first_item_name = $wpdb->get_var(
79
			$wpdb->prepare(
80
				"SELECT option_name FROM $wpdb->options WHERE option_name LIKE %s ORDER BY option_name ASC LIMIT 1",
81
				"jpsq_{$this->id}-%"
82
			)
83
		);
84
85
		if ( ! $first_item_name ) {
86
			return 0;
87
		}
88
89
		if ( null === $now ) {
90
			$now = microtime( true );
91
		}
92
93
		// break apart the item name to get the timestamp
94
		$matches = null;
95
		if ( preg_match( '/^jpsq_' . $this->id . '-(\d+\.\d+)-/', $first_item_name, $matches ) ) {
96
			return $now - floatval( $matches[1] );
97
		} else {
98
			return 0;
99
		}
100
	}
101
102
	function reset() {
103
		global $wpdb;
104
		$this->delete_checkout_id();
105
		$wpdb->query(
106
			$wpdb->prepare(
107
				"DELETE FROM $wpdb->options WHERE option_name LIKE %s",
108
				"jpsq_{$this->id}-%"
109
			)
110
		);
111
	}
112
113
	function size() {
114
		global $wpdb;
115
116
		return (int) $wpdb->get_var(
117
			$wpdb->prepare(
118
				"SELECT count(*) FROM $wpdb->options WHERE option_name LIKE %s",
119
				"jpsq_{$this->id}-%"
120
			)
121
		);
122
	}
123
124
	// we use this peculiar implementation because it's much faster than count(*)
125
	function has_any_items() {
126
		global $wpdb;
127
		$value = $wpdb->get_var(
128
			$wpdb->prepare(
129
				"SELECT exists( SELECT option_name FROM $wpdb->options WHERE option_name LIKE %s )",
130
				"jpsq_{$this->id}-%"
131
			)
132
		);
133
134
		return ( $value === '1' );
135
	}
136
137
	function checkout( $buffer_size ) {
138
		if ( $this->get_checkout_id() ) {
139
			return new \WP_Error( 'unclosed_buffer', 'There is an unclosed buffer' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'unclosed_buffer'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
140
		}
141
142
		$buffer_id = uniqid();
143
144
		$result = $this->set_checkout_id( $buffer_id );
145
146
		if ( ! $result || is_wp_error( $result ) ) {
147
			return $result;
148
		}
149
150
		$items = $this->fetch_items( $buffer_size );
151
152
		if ( count( $items ) === 0 ) {
153
			return false;
154
		}
155
156
		$buffer = new Queue_Buffer( $buffer_id, array_slice( $items, 0, $buffer_size ) );
157
158
		return $buffer;
159
	}
160
161
	// this checks out rows until it either empties the queue or hits a certain memory limit
162
	// it loads the sizes from the DB first so that it doesn't accidentally
163
	// load more data into memory than it needs to.
164
	// The only way it will load more items than $max_size is if a single queue item
165
	// exceeds the memory limit, but in that case it will send that item by itself.
166
	function checkout_with_memory_limit( $max_memory, $max_buffer_size = 500 ) {
167
		if ( $this->get_checkout_id() ) {
168
			return new \WP_Error( 'unclosed_buffer', 'There is an unclosed buffer' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'unclosed_buffer'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
169
		}
170
171
		$buffer_id = uniqid();
172
173
		$result = $this->set_checkout_id( $buffer_id );
174
175
		if ( ! $result || is_wp_error( $result ) ) {
176
			return $result;
177
		}
178
179
		// get the map of buffer_id -> memory_size
180
		global $wpdb;
181
182
		$items_with_size = $wpdb->get_results(
183
			$wpdb->prepare(
184
				"SELECT option_name AS id, LENGTH(option_value) AS value_size FROM $wpdb->options WHERE option_name LIKE %s ORDER BY option_name ASC LIMIT %d",
185
				"jpsq_{$this->id}-%",
186
				$max_buffer_size
187
			),
188
			OBJECT
189
		);
190
191
		if ( count( $items_with_size ) === 0 ) {
192
			return false;
193
		}
194
195
		$total_memory = 0;
196
197
		$min_item_id = $max_item_id = $items_with_size[0]->id;
198
199
		foreach ( $items_with_size as $id => $item_with_size ) {
200
			$total_memory += $item_with_size->value_size;
201
202
			// if this is the first item and it exceeds memory, allow loop to continue
203
			// we will exit on the next iteration instead
204
			if ( $total_memory > $max_memory && $id > 0 ) {
205
				break;
206
			}
207
208
			$max_item_id = $item_with_size->id;
209
		}
210
211
		$query = $wpdb->prepare(
212
			"SELECT option_name AS id, option_value AS value FROM $wpdb->options WHERE option_name >= %s and option_name <= %s ORDER BY option_name ASC",
213
			$min_item_id,
214
			$max_item_id
215
		);
216
217
		$items = $wpdb->get_results( $query, OBJECT );
218
		foreach ( $items as $item ) {
219
			$item->value = maybe_unserialize( $item->value );
220
		}
221
222
		if ( count( $items ) === 0 ) {
223
			$this->delete_checkout_id();
224
225
			return false;
226
		}
227
228
		$buffer = new Queue_Buffer( $buffer_id, $items );
229
230
		return $buffer;
231
	}
232
233
	function checkin( $buffer ) {
234
		$is_valid = $this->validate_checkout( $buffer );
235
236
		if ( is_wp_error( $is_valid ) ) {
237
			return $is_valid;
238
		}
239
240
		$this->delete_checkout_id();
241
242
		return true;
243
	}
244
245
	function close( $buffer, $ids_to_remove = null ) {
246
		$is_valid = $this->validate_checkout( $buffer );
247
248
		if ( is_wp_error( $is_valid ) ) {
249
			return $is_valid;
250
		}
251
252
		$this->delete_checkout_id();
253
254
		// by default clear all items in the buffer
255
		if ( is_null( $ids_to_remove ) ) {
256
			$ids_to_remove = $buffer->get_item_ids();
257
		}
258
259
		global $wpdb;
260
261
		if ( count( $ids_to_remove ) > 0 ) {
262
			$sql   = "DELETE FROM $wpdb->options WHERE option_name IN (" . implode( ', ', array_fill( 0, count( $ids_to_remove ), '%s' ) ) . ')';
263
			$query = call_user_func_array( array( $wpdb, 'prepare' ), array_merge( array( $sql ), $ids_to_remove ) );
264
			$wpdb->query( $query );
265
		}
266
267
		return true;
268
	}
269
270
	function flush_all() {
271
		$items = Utils::get_item_values( $this->fetch_items() );
272
		$this->reset();
273
274
		return $items;
275
	}
276
277
	function get_all() {
278
		return $this->fetch_items();
279
	}
280
281
	// use with caution, this could allow multiple processes to delete
282
	// and send from the queue at the same time
283
	function force_checkin() {
284
		$this->delete_checkout_id();
285
	}
286
287
	// used to lock checkouts from the queue.
288
	// tries to wait up to $timeout seconds for the queue to be empty
289
	function lock( $timeout = 30 ) {
290
		$tries = 0;
291
292
		while ( $this->has_any_items() && $tries < $timeout ) {
293
			sleep( 1 );
294
			$tries += 1;
295
		}
296
297
		if ( $tries === 30 ) {
298
			return new \WP_Error( 'lock_timeout', 'Timeout waiting for sync queue to empty' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'lock_timeout'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
299
		}
300
301
		if ( $this->get_checkout_id() ) {
302
			return new \WP_Error( 'unclosed_buffer', 'There is an unclosed buffer' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'unclosed_buffer'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
303
		}
304
305
		// hopefully this means we can acquire a checkout?
306
		$result = $this->set_checkout_id( 'lock' );
307
308
		if ( ! $result || is_wp_error( $result ) ) {
309
			return $result;
310
		}
311
312
		return true;
313
	}
314
315
	function unlock() {
316
		return $this->delete_checkout_id();
317
	}
318
319
	/**
320
	 * This option is specifically chosen to, as much as possible, preserve time order
321
	 * and minimise the possibility of collisions between multiple processes working
322
	 * at the same time.
323
	 *
324
	 * @return string
325
	 */
326
	protected function generate_option_name_timestamp() {
327
		return sprintf( '%.6f', microtime( true ) );
328
	}
329
330
	private function get_checkout_id() {
331
		global $wpdb;
332
		$checkout_value = $wpdb->get_var(
333
			$wpdb->prepare(
334
				"SELECT option_value FROM $wpdb->options WHERE option_name = %s",
335
				$this->get_lock_option_name()
336
			)
337
		);
338
339
		if ( $checkout_value ) {
340
			list( $checkout_id, $timestamp ) = explode( ':', $checkout_value );
341
			if ( intval( $timestamp ) > time() ) {
342
				return $checkout_id;
343
			}
344
		}
345
346
		return false;
347
	}
348
349
	private function set_checkout_id( $checkout_id ) {
350
		global $wpdb;
351
352
		$expires     = time() + Defaults::$default_sync_queue_lock_timeout;
0 ignored issues
show
Bug introduced by
The property default_sync_queue_lock_timeout cannot be accessed from this context as it is declared private in class Automattic\Jetpack\Sync\Defaults.

This check looks for access to properties that are not accessible from the current context.

If you need to make a property accessible to another context you can either raise its visibility level or provide an accessible getter in the defining class.

Loading history...
353
		$updated_num = $wpdb->query(
354
			$wpdb->prepare(
355
				"UPDATE $wpdb->options SET option_value = %s WHERE option_name = %s",
356
				"$checkout_id:$expires",
357
				$this->get_lock_option_name()
358
			)
359
		);
360
361
		if ( ! $updated_num ) {
362
			$updated_num = $wpdb->query(
363
				$wpdb->prepare(
364
					"INSERT INTO $wpdb->options ( option_name, option_value, autoload ) VALUES ( %s, %s, 'no' )",
365
					$this->get_lock_option_name(),
366
					"$checkout_id:$expires"
367
				)
368
			);
369
		}
370
371
		return $updated_num;
372
	}
373
374
	private function delete_checkout_id() {
375
		global $wpdb;
376
		// rather than delete, which causes fragmentation, we update in place
377
		return $wpdb->query(
378
			$wpdb->prepare(
379
				"UPDATE $wpdb->options SET option_value = %s WHERE option_name = %s",
380
				'0:0',
381
				$this->get_lock_option_name()
382
			)
383
		);
384
385
	}
386
387
	private function get_lock_option_name() {
388
		return "jpsq_{$this->id}_checkout";
389
	}
390
391
	private function get_next_data_row_option_name() {
392
		$timestamp = $this->generate_option_name_timestamp();
393
394
		// row iterator is used to avoid collisions where we're writing data waaay fast in a single process
395
		if ( $this->row_iterator === PHP_INT_MAX ) {
396
			$this->row_iterator = 0;
397
		} else {
398
			$this->row_iterator += 1;
399
		}
400
401
		return 'jpsq_' . $this->id . '-' . $timestamp . '-' . $this->random_int . '-' . $this->row_iterator;
402
	}
403
404
	private function fetch_items( $limit = null ) {
405
		global $wpdb;
406
407
		if ( $limit ) {
408
			$query_sql = $wpdb->prepare( "SELECT option_name AS id, option_value AS value FROM $wpdb->options WHERE option_name LIKE %s ORDER BY option_name ASC LIMIT %d", "jpsq_{$this->id}-%", $limit );
409
		} else {
410
			$query_sql = $wpdb->prepare( "SELECT option_name AS id, option_value AS value FROM $wpdb->options WHERE option_name LIKE %s ORDER BY option_name ASC", "jpsq_{$this->id}-%" );
411
		}
412
413
		$items = $wpdb->get_results( $query_sql, OBJECT );
414
		foreach ( $items as $item ) {
415
			$item->value = maybe_unserialize( $item->value );
416
		}
417
418
		return $items;
419
	}
420
421
	private function validate_checkout( $buffer ) {
422
		if ( ! $buffer instanceof Queue_Buffer ) {
423
			return new \WP_Error( 'not_a_buffer', 'You must checkin an instance of Automattic\\Jetpack\\Sync\\Queue_Buffer' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'not_a_buffer'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
424
		}
425
426
		$checkout_id = $this->get_checkout_id();
427
428
		if ( ! $checkout_id ) {
429
			return new \WP_Error( 'buffer_not_checked_out', 'There are no checked out buffers' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'buffer_not_checked_out'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
430
		}
431
432
		if ( $checkout_id != $buffer->id ) {
433
			return new \WP_Error( 'buffer_mismatch', 'The buffer you checked in was not checked out' );
0 ignored issues
show
Unused Code introduced by
The call to WP_Error::__construct() has too many arguments starting with 'buffer_mismatch'.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
434
		}
435
436
		return true;
437
	}
438
}
439