FileBackend::streamFile()
last analyzed

Size

Total Lines 1

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
nc 1
dl 0
loc 1
c 0
b 0
f 0
1
<?php
2
/**
3
 * @defgroup FileBackend File backend
4
 *
5
 * File backend is used to interact with file storage systems,
6
 * such as the local file system, NFS, or cloud storage systems.
7
 */
8
9
/**
10
 * Base class for all file backends.
11
 *
12
 * This program is free software; you can redistribute it and/or modify
13
 * it under the terms of the GNU General Public License as published by
14
 * the Free Software Foundation; either version 2 of the License, or
15
 * (at your option) any later version.
16
 *
17
 * This program is distributed in the hope that it will be useful,
18
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
19
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
20
 * GNU General Public License for more details.
21
 *
22
 * You should have received a copy of the GNU General Public License along
23
 * with this program; if not, write to the Free Software Foundation, Inc.,
24
 * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
25
 * http://www.gnu.org/copyleft/gpl.html
26
 *
27
 * @file
28
 * @ingroup FileBackend
29
 * @author Aaron Schulz
30
 */
31
use Psr\Log\LoggerAwareInterface;
32
use Psr\Log\LoggerInterface;
33
use Wikimedia\ScopedCallback;
0 ignored issues
show
Bug introduced by
This use statement conflicts with another class in this namespace, ScopedCallback.

Let’s assume that you have a directory layout like this:

.
|-- OtherDir
|   |-- Bar.php
|   `-- Foo.php
`-- SomeDir
    `-- Foo.php

and let’s assume the following content of Bar.php:

// Bar.php
namespace OtherDir;

use SomeDir\Foo; // This now conflicts the class OtherDir\Foo

If both files OtherDir/Foo.php and SomeDir/Foo.php are loaded in the same runtime, you will see a PHP error such as the following:

PHP Fatal error:  Cannot use SomeDir\Foo as Foo because the name is already in use in OtherDir/Foo.php

However, as OtherDir/Foo.php does not necessarily have to be loaded and the error is only triggered if it is loaded before OtherDir/Bar.php, this problem might go unnoticed for a while. In order to prevent this error from surfacing, you must import the namespace with a different alias:

// Bar.php
namespace OtherDir;

use SomeDir\Foo as SomeDirFoo; // There is no conflict anymore.
Loading history...
34
35
/**
36
 * @brief Base class for all file backend classes (including multi-write backends).
37
 *
38
 * This class defines the methods as abstract that subclasses must implement.
39
 * Outside callers can assume that all backends will have these functions.
40
 *
41
 * All "storage paths" are of the format "mwstore://<backend>/<container>/<path>".
42
 * The "backend" portion is unique name for the application to refer to a backend, while
43
 * the "container" portion is a top-level directory of the backend. The "path" portion
44
 * is a relative path that uses UNIX file system (FS) notation, though any particular
45
 * backend may not actually be using a local filesystem. Therefore, the relative paths
46
 * are only virtual.
47
 *
48
 * Backend contents are stored under "domain"-specific container names by default.
49
 * A domain is simply a logical umbrella for entities, such as those belonging to a certain
50
 * application or portion of a website, for example. A domain can be local or global.
51
 * Global (qualified) backends are achieved by configuring the "domain ID" to a constant.
52
 * Global domains are simpler, but local domains can be used by choosing a domain ID based on
53
 * the current context, such as which language of a website is being used.
54
 *
55
 * For legacy reasons, the FSFileBackend class allows manually setting the paths of
56
 * containers to ones that do not respect the "domain ID".
57
 *
58
 * In key/value (object) stores, containers are the only hierarchy (the rest is emulated).
59
 * FS-based backends are somewhat more restrictive due to the existence of real
60
 * directory files; a regular file cannot have the same name as a directory. Other
61
 * backends with virtual directories may not have this limitation. Callers should
62
 * store files in such a way that no files and directories are under the same path.
63
 *
64
 * In general, this class allows for callers to access storage through the same
65
 * interface, without regard to the underlying storage system. However, calling code
66
 * must follow certain patterns and be aware of certain things to ensure compatibility:
67
 *   - a) Always call prepare() on the parent directory before trying to put a file there;
68
 *        key/value stores only need the container to exist first, but filesystems need
69
 *        all the parent directories to exist first (prepare() is aware of all this)
70
 *   - b) Always call clean() on a directory when it might become empty to avoid empty
71
 *        directory buildup on filesystems; key/value stores never have empty directories,
72
 *        so doing this helps preserve consistency in both cases
73
 *   - c) Likewise, do not rely on the existence of empty directories for anything;
74
 *        calling directoryExists() on a path that prepare() was previously called on
75
 *        will return false for key/value stores if there are no files under that path
76
 *   - d) Never alter the resulting FSFile returned from getLocalReference(), as it could
77
 *        either be a copy of the source file in /tmp or the original source file itself
78
 *   - e) Use a file layout that results in never attempting to store files over directories
79
 *        or directories over files; key/value stores allow this but filesystems do not
80
 *   - f) Use ASCII file names (e.g. base32, IDs, hashes) to avoid Unicode issues in Windows
81
 *   - g) Do not assume that move operations are atomic (difficult with key/value stores)
82
 *   - h) Do not assume that file stat or read operations always have immediate consistency;
83
 *        various methods have a "latest" flag that should always be used if up-to-date
84
 *        information is required (this trades performance for correctness as needed)
85
 *   - i) Do not assume that directory listings have immediate consistency
86
 *
87
 * Methods of subclasses should avoid throwing exceptions at all costs.
88
 * As a corollary, external dependencies should be kept to a minimum.
89
 *
90
 * @ingroup FileBackend
91
 * @since 1.19
92
 */
93
abstract class FileBackend implements LoggerAwareInterface {
94
	/** @var string Unique backend name */
95
	protected $name;
96
97
	/** @var string Unique domain name */
98
	protected $domainId;
99
100
	/** @var string Read-only explanation message */
101
	protected $readOnly;
102
103
	/** @var string When to do operations in parallel */
104
	protected $parallelize;
105
106
	/** @var int How many operations can be done in parallel */
107
	protected $concurrency;
108
109
	/** @var string Temporary file directory */
110
	protected $tmpDirectory;
111
112
	/** @var LockManager */
113
	protected $lockManager;
114
	/** @var FileJournal */
115
	protected $fileJournal;
116
	/** @var LoggerInterface */
117
	protected $logger;
118
	/** @var object|string Class name or object With profileIn/profileOut methods */
119
	protected $profiler;
120
121
	/** @var callable */
122
	protected $obResetFunc;
123
	/** @var callable */
124
	protected $streamMimeFunc;
125
	/** @var callable */
126
	protected $statusWrapper;
127
128
	/** Bitfield flags for supported features */
129
	const ATTR_HEADERS = 1; // files can be tagged with standard HTTP headers
130
	const ATTR_METADATA = 2; // files can be stored with metadata key/values
131
	const ATTR_UNICODE_PATHS = 4; // files can have Unicode paths (not just ASCII)
132
133
	/**
134
	 * Create a new backend instance from configuration.
135
	 * This should only be called from within FileBackendGroup.
136
	 *
137
	 * @param array $config Parameters include:
138
	 *   - name : The unique name of this backend.
139
	 *      This should consist of alphanumberic, '-', and '_' characters.
140
	 *      This name should not be changed after use (e.g. with journaling).
141
	 *      Note that the name is *not* used in actual container names.
142
	 *   - domainId : Prefix to container names that is unique to this backend.
143
	 *      It should only consist of alphanumberic, '-', and '_' characters.
144
	 *      This ID is what avoids collisions if multiple logical backends
145
	 *      use the same storage system, so this should be set carefully.
146
	 *   - lockManager : LockManager object to use for any file locking.
147
	 *      If not provided, then no file locking will be enforced.
148
	 *   - fileJournal : FileJournal object to use for logging changes to files.
149
	 *      If not provided, then change journaling will be disabled.
150
	 *   - readOnly : Write operations are disallowed if this is a non-empty string.
151
	 *      It should be an explanation for the backend being read-only.
152
	 *   - parallelize : When to do file operations in parallel (when possible).
153
	 *      Allowed values are "implicit", "explicit" and "off".
154
	 *   - concurrency : How many file operations can be done in parallel.
155
	 *   - tmpDirectory : Directory to use for temporary files. If this is not set or null,
156
	 *      then the backend will try to discover a usable temporary directory.
157
	 *   - obResetFunc : alternative callback to clear the output buffer
158
	 *   - streamMimeFunc : alternative method to determine the content type from the path
159
	 *   - logger : Optional PSR logger object.
160
	 *   - profiler : Optional class name or object With profileIn/profileOut methods.
161
	 * @throws InvalidArgumentException
162
	 */
163
	public function __construct( array $config ) {
164
		$this->name = $config['name'];
165
		$this->domainId = isset( $config['domainId'] )
166
			? $config['domainId'] // e.g. "my_wiki-en_"
167
			: $config['wikiId']; // b/c alias
168
		if ( !preg_match( '!^[a-zA-Z0-9-_]{1,255}$!', $this->name ) ) {
169
			throw new InvalidArgumentException( "Backend name '{$this->name}' is invalid." );
170
		} elseif ( !is_string( $this->domainId ) ) {
171
			throw new InvalidArgumentException(
172
				"Backend domain ID not provided for '{$this->name}'." );
173
		}
174
		$this->lockManager = isset( $config['lockManager'] )
175
			? $config['lockManager']
176
			: new NullLockManager( [] );
177
		$this->fileJournal = isset( $config['fileJournal'] )
178
			? $config['fileJournal']
179
			: FileJournal::factory( [ 'class' => 'NullFileJournal' ], $this->name );
180
		$this->readOnly = isset( $config['readOnly'] )
181
			? (string)$config['readOnly']
182
			: '';
183
		$this->parallelize = isset( $config['parallelize'] )
184
			? (string)$config['parallelize']
185
			: 'off';
186
		$this->concurrency = isset( $config['concurrency'] )
187
			? (int)$config['concurrency']
188
			: 50;
189
		$this->obResetFunc = isset( $config['obResetFunc'] )
190
			? $config['obResetFunc']
191
			: [ $this, 'resetOutputBuffer' ];
192
		$this->streamMimeFunc = isset( $config['streamMimeFunc'] )
193
			? $config['streamMimeFunc']
194
			: null;
195
		$this->statusWrapper = isset( $config['statusWrapper'] ) ? $config['statusWrapper'] : null;
196
197
		$this->profiler = isset( $config['profiler'] ) ? $config['profiler'] : null;
198
		$this->logger = isset( $config['logger'] ) ? $config['logger'] : new \Psr\Log\NullLogger();
199
		$this->statusWrapper = isset( $config['statusWrapper'] ) ? $config['statusWrapper'] : null;
200
		$this->tmpDirectory = isset( $config['tmpDirectory'] ) ? $config['tmpDirectory'] : null;
201
	}
202
203
	public function setLogger( LoggerInterface $logger ) {
204
		$this->logger = $logger;
205
	}
206
207
	/**
208
	 * Get the unique backend name.
209
	 * We may have multiple different backends of the same type.
210
	 * For example, we can have two Swift backends using different proxies.
211
	 *
212
	 * @return string
213
	 */
214
	final public function getName() {
215
		return $this->name;
216
	}
217
218
	/**
219
	 * Get the domain identifier used for this backend (possibly empty).
220
	 *
221
	 * @return string
222
	 * @since 1.28
223
	 */
224
	final public function getDomainId() {
225
		return $this->domainId;
226
	}
227
228
	/**
229
	 * Alias to getDomainId()
230
	 * @return string
231
	 * @since 1.20
232
	 */
233
	final public function getWikiId() {
234
		return $this->getDomainId();
235
	}
236
237
	/**
238
	 * Check if this backend is read-only
239
	 *
240
	 * @return bool
241
	 */
242
	final public function isReadOnly() {
243
		return ( $this->readOnly != '' );
244
	}
245
246
	/**
247
	 * Get an explanatory message if this backend is read-only
248
	 *
249
	 * @return string|bool Returns false if the backend is not read-only
250
	 */
251
	final public function getReadOnlyReason() {
252
		return ( $this->readOnly != '' ) ? $this->readOnly : false;
253
	}
254
255
	/**
256
	 * Get the a bitfield of extra features supported by the backend medium
257
	 *
258
	 * @return int Bitfield of FileBackend::ATTR_* flags
259
	 * @since 1.23
260
	 */
261
	public function getFeatures() {
262
		return self::ATTR_UNICODE_PATHS;
263
	}
264
265
	/**
266
	 * Check if the backend medium supports a field of extra features
267
	 *
268
	 * @param int $bitfield Bitfield of FileBackend::ATTR_* flags
269
	 * @return bool
270
	 * @since 1.23
271
	 */
272
	final public function hasFeatures( $bitfield ) {
273
		return ( $this->getFeatures() & $bitfield ) === $bitfield;
274
	}
275
276
	/**
277
	 * This is the main entry point into the backend for write operations.
278
	 * Callers supply an ordered list of operations to perform as a transaction.
279
	 * Files will be locked, the stat cache cleared, and then the operations attempted.
280
	 * If any serious errors occur, all attempted operations will be rolled back.
281
	 *
282
	 * $ops is an array of arrays. The outer array holds a list of operations.
283
	 * Each inner array is a set of key value pairs that specify an operation.
284
	 *
285
	 * Supported operations and their parameters. The supported actions are:
286
	 *  - create
287
	 *  - store
288
	 *  - copy
289
	 *  - move
290
	 *  - delete
291
	 *  - describe (since 1.21)
292
	 *  - null
293
	 *
294
	 * FSFile/TempFSFile object support was added in 1.27.
295
	 *
296
	 * a) Create a new file in storage with the contents of a string
297
	 * @code
298
	 *     [
299
	 *         'op'                  => 'create',
300
	 *         'dst'                 => <storage path>,
301
	 *         'content'             => <string of new file contents>,
302
	 *         'overwrite'           => <boolean>,
303
	 *         'overwriteSame'       => <boolean>,
304
	 *         'headers'             => <HTTP header name/value map> # since 1.21
305
	 *     ]
306
	 * @endcode
307
	 *
308
	 * b) Copy a file system file into storage
309
	 * @code
310
	 *     [
311
	 *         'op'                  => 'store',
312
	 *         'src'                 => <file system path, FSFile, or TempFSFile>,
313
	 *         'dst'                 => <storage path>,
314
	 *         'overwrite'           => <boolean>,
315
	 *         'overwriteSame'       => <boolean>,
316
	 *         'headers'             => <HTTP header name/value map> # since 1.21
317
	 *     ]
318
	 * @endcode
319
	 *
320
	 * c) Copy a file within storage
321
	 * @code
322
	 *     [
323
	 *         'op'                  => 'copy',
324
	 *         'src'                 => <storage path>,
325
	 *         'dst'                 => <storage path>,
326
	 *         'overwrite'           => <boolean>,
327
	 *         'overwriteSame'       => <boolean>,
328
	 *         'ignoreMissingSource' => <boolean>, # since 1.21
329
	 *         'headers'             => <HTTP header name/value map> # since 1.21
330
	 *     ]
331
	 * @endcode
332
	 *
333
	 * d) Move a file within storage
334
	 * @code
335
	 *     [
336
	 *         'op'                  => 'move',
337
	 *         'src'                 => <storage path>,
338
	 *         'dst'                 => <storage path>,
339
	 *         'overwrite'           => <boolean>,
340
	 *         'overwriteSame'       => <boolean>,
341
	 *         'ignoreMissingSource' => <boolean>, # since 1.21
342
	 *         'headers'             => <HTTP header name/value map> # since 1.21
343
	 *     ]
344
	 * @endcode
345
	 *
346
	 * e) Delete a file within storage
347
	 * @code
348
	 *     [
349
	 *         'op'                  => 'delete',
350
	 *         'src'                 => <storage path>,
351
	 *         'ignoreMissingSource' => <boolean>
352
	 *     ]
353
	 * @endcode
354
	 *
355
	 * f) Update metadata for a file within storage
356
	 * @code
357
	 *     [
358
	 *         'op'                  => 'describe',
359
	 *         'src'                 => <storage path>,
360
	 *         'headers'             => <HTTP header name/value map>
361
	 *     ]
362
	 * @endcode
363
	 *
364
	 * g) Do nothing (no-op)
365
	 * @code
366
	 *     [
367
	 *         'op'                  => 'null',
368
	 *     ]
369
	 * @endcode
370
	 *
371
	 * Boolean flags for operations (operation-specific):
372
	 *   - ignoreMissingSource : The operation will simply succeed and do
373
	 *                           nothing if the source file does not exist.
374
	 *   - overwrite           : Any destination file will be overwritten.
375
	 *   - overwriteSame       : If a file already exists at the destination with the
376
	 *                           same contents, then do nothing to the destination file
377
	 *                           instead of giving an error. This does not compare headers.
378
	 *                           This option is ignored if 'overwrite' is already provided.
379
	 *   - headers             : If supplied, the result of merging these headers with any
380
	 *                           existing source file headers (replacing conflicting ones)
381
	 *                           will be set as the destination file headers. Headers are
382
	 *                           deleted if their value is set to the empty string. When a
383
	 *                           file has headers they are included in responses to GET and
384
	 *                           HEAD requests to the backing store for that file.
385
	 *                           Header values should be no larger than 255 bytes, except for
386
	 *                           Content-Disposition. The system might ignore or truncate any
387
	 *                           headers that are too long to store (exact limits will vary).
388
	 *                           Backends that don't support metadata ignore this. (since 1.21)
389
	 *
390
	 * $opts is an associative of boolean flags, including:
391
	 *   - force               : Operation precondition errors no longer trigger an abort.
392
	 *                           Any remaining operations are still attempted. Unexpected
393
	 *                           failures may still cause remaining operations to be aborted.
394
	 *   - nonLocking          : No locks are acquired for the operations.
395
	 *                           This can increase performance for non-critical writes.
396
	 *                           This has no effect unless the 'force' flag is set.
397
	 *   - nonJournaled        : Don't log this operation batch in the file journal.
398
	 *                           This limits the ability of recovery scripts.
399
	 *   - parallelize         : Try to do operations in parallel when possible.
400
	 *   - bypassReadOnly      : Allow writes in read-only mode. (since 1.20)
401
	 *   - preserveCache       : Don't clear the process cache before checking files.
402
	 *                           This should only be used if all entries in the process
403
	 *                           cache were added after the files were already locked. (since 1.20)
404
	 *
405
	 * @remarks Remarks on locking:
406
	 * File system paths given to operations should refer to files that are
407
	 * already locked or otherwise safe from modification from other processes.
408
	 * Normally these files will be new temp files, which should be adequate.
409
	 *
410
	 * @par Return value:
411
	 *
412
	 * This returns a Status, which contains all warnings and fatals that occurred
413
	 * during the operation. The 'failCount', 'successCount', and 'success' members
414
	 * will reflect each operation attempted.
415
	 *
416
	 * The StatusValue will be "OK" unless:
417
	 *   - a) unexpected operation errors occurred (network partitions, disk full...)
418
	 *   - b) significant operation errors occurred and 'force' was not set
419
	 *
420
	 * @param array $ops List of operations to execute in order
421
	 * @param array $opts Batch operation options
422
	 * @return StatusValue
423
	 */
424 View Code Duplication
	final public function doOperations( array $ops, array $opts = [] ) {
425
		if ( empty( $opts['bypassReadOnly'] ) && $this->isReadOnly() ) {
426
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
427
		}
428
		if ( !count( $ops ) ) {
429
			return $this->newStatus(); // nothing to do
430
		}
431
432
		$ops = $this->resolveFSFileObjects( $ops );
433
		if ( empty( $opts['force'] ) ) { // sanity
434
			unset( $opts['nonLocking'] );
435
		}
436
437
		/** @noinspection PhpUnusedLocalVariableInspection */
438
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
439
440
		return $this->doOperationsInternal( $ops, $opts );
441
	}
442
443
	/**
444
	 * @see FileBackend::doOperations()
445
	 * @param array $ops
446
	 * @param array $opts
447
	 */
448
	abstract protected function doOperationsInternal( array $ops, array $opts );
449
450
	/**
451
	 * Same as doOperations() except it takes a single operation.
452
	 * If you are doing a batch of operations that should either
453
	 * all succeed or all fail, then use that function instead.
454
	 *
455
	 * @see FileBackend::doOperations()
456
	 *
457
	 * @param array $op Operation
458
	 * @param array $opts Operation options
459
	 * @return StatusValue
460
	 */
461
	final public function doOperation( array $op, array $opts = [] ) {
462
		return $this->doOperations( [ $op ], $opts );
463
	}
464
465
	/**
466
	 * Performs a single create operation.
467
	 * This sets $params['op'] to 'create' and passes it to doOperation().
468
	 *
469
	 * @see FileBackend::doOperation()
470
	 *
471
	 * @param array $params Operation parameters
472
	 * @param array $opts Operation options
473
	 * @return StatusValue
474
	 */
475
	final public function create( array $params, array $opts = [] ) {
476
		return $this->doOperation( [ 'op' => 'create' ] + $params, $opts );
477
	}
478
479
	/**
480
	 * Performs a single store operation.
481
	 * This sets $params['op'] to 'store' and passes it to doOperation().
482
	 *
483
	 * @see FileBackend::doOperation()
484
	 *
485
	 * @param array $params Operation parameters
486
	 * @param array $opts Operation options
487
	 * @return StatusValue
488
	 */
489
	final public function store( array $params, array $opts = [] ) {
490
		return $this->doOperation( [ 'op' => 'store' ] + $params, $opts );
491
	}
492
493
	/**
494
	 * Performs a single copy operation.
495
	 * This sets $params['op'] to 'copy' and passes it to doOperation().
496
	 *
497
	 * @see FileBackend::doOperation()
498
	 *
499
	 * @param array $params Operation parameters
500
	 * @param array $opts Operation options
501
	 * @return StatusValue
502
	 */
503
	final public function copy( array $params, array $opts = [] ) {
504
		return $this->doOperation( [ 'op' => 'copy' ] + $params, $opts );
505
	}
506
507
	/**
508
	 * Performs a single move operation.
509
	 * This sets $params['op'] to 'move' and passes it to doOperation().
510
	 *
511
	 * @see FileBackend::doOperation()
512
	 *
513
	 * @param array $params Operation parameters
514
	 * @param array $opts Operation options
515
	 * @return StatusValue
516
	 */
517
	final public function move( array $params, array $opts = [] ) {
518
		return $this->doOperation( [ 'op' => 'move' ] + $params, $opts );
519
	}
520
521
	/**
522
	 * Performs a single delete operation.
523
	 * This sets $params['op'] to 'delete' and passes it to doOperation().
524
	 *
525
	 * @see FileBackend::doOperation()
526
	 *
527
	 * @param array $params Operation parameters
528
	 * @param array $opts Operation options
529
	 * @return StatusValue
530
	 */
531
	final public function delete( array $params, array $opts = [] ) {
532
		return $this->doOperation( [ 'op' => 'delete' ] + $params, $opts );
533
	}
534
535
	/**
536
	 * Performs a single describe operation.
537
	 * This sets $params['op'] to 'describe' and passes it to doOperation().
538
	 *
539
	 * @see FileBackend::doOperation()
540
	 *
541
	 * @param array $params Operation parameters
542
	 * @param array $opts Operation options
543
	 * @return StatusValue
544
	 * @since 1.21
545
	 */
546
	final public function describe( array $params, array $opts = [] ) {
547
		return $this->doOperation( [ 'op' => 'describe' ] + $params, $opts );
548
	}
549
550
	/**
551
	 * Perform a set of independent file operations on some files.
552
	 *
553
	 * This does no locking, nor journaling, and possibly no stat calls.
554
	 * Any destination files that already exist will be overwritten.
555
	 * This should *only* be used on non-original files, like cache files.
556
	 *
557
	 * Supported operations and their parameters:
558
	 *  - create
559
	 *  - store
560
	 *  - copy
561
	 *  - move
562
	 *  - delete
563
	 *  - describe (since 1.21)
564
	 *  - null
565
	 *
566
	 * FSFile/TempFSFile object support was added in 1.27.
567
	 *
568
	 * a) Create a new file in storage with the contents of a string
569
	 * @code
570
	 *     [
571
	 *         'op'                  => 'create',
572
	 *         'dst'                 => <storage path>,
573
	 *         'content'             => <string of new file contents>,
574
	 *         'headers'             => <HTTP header name/value map> # since 1.21
575
	 *     ]
576
	 * @endcode
577
	 *
578
	 * b) Copy a file system file into storage
579
	 * @code
580
	 *     [
581
	 *         'op'                  => 'store',
582
	 *         'src'                 => <file system path, FSFile, or TempFSFile>,
583
	 *         'dst'                 => <storage path>,
584
	 *         'headers'             => <HTTP header name/value map> # since 1.21
585
	 *     ]
586
	 * @endcode
587
	 *
588
	 * c) Copy a file within storage
589
	 * @code
590
	 *     [
591
	 *         'op'                  => 'copy',
592
	 *         'src'                 => <storage path>,
593
	 *         'dst'                 => <storage path>,
594
	 *         'ignoreMissingSource' => <boolean>, # since 1.21
595
	 *         'headers'             => <HTTP header name/value map> # since 1.21
596
	 *     ]
597
	 * @endcode
598
	 *
599
	 * d) Move a file within storage
600
	 * @code
601
	 *     [
602
	 *         'op'                  => 'move',
603
	 *         'src'                 => <storage path>,
604
	 *         'dst'                 => <storage path>,
605
	 *         'ignoreMissingSource' => <boolean>, # since 1.21
606
	 *         'headers'             => <HTTP header name/value map> # since 1.21
607
	 *     ]
608
	 * @endcode
609
	 *
610
	 * e) Delete a file within storage
611
	 * @code
612
	 *     [
613
	 *         'op'                  => 'delete',
614
	 *         'src'                 => <storage path>,
615
	 *         'ignoreMissingSource' => <boolean>
616
	 *     ]
617
	 * @endcode
618
	 *
619
	 * f) Update metadata for a file within storage
620
	 * @code
621
	 *     [
622
	 *         'op'                  => 'describe',
623
	 *         'src'                 => <storage path>,
624
	 *         'headers'             => <HTTP header name/value map>
625
	 *     ]
626
	 * @endcode
627
	 *
628
	 * g) Do nothing (no-op)
629
	 * @code
630
	 *     [
631
	 *         'op'                  => 'null',
632
	 *     ]
633
	 * @endcode
634
	 *
635
	 * @par Boolean flags for operations (operation-specific):
636
	 *   - ignoreMissingSource : The operation will simply succeed and do
637
	 *                           nothing if the source file does not exist.
638
	 *   - headers             : If supplied with a header name/value map, the backend will
639
	 *                           reply with these headers when GETs/HEADs of the destination
640
	 *                           file are made. Header values should be smaller than 256 bytes.
641
	 *                           Content-Disposition headers can be longer, though the system
642
	 *                           might ignore or truncate ones that are too long to store.
643
	 *                           Existing headers will remain, but these will replace any
644
	 *                           conflicting previous headers, and headers will be removed
645
	 *                           if they are set to an empty string.
646
	 *                           Backends that don't support metadata ignore this. (since 1.21)
647
	 *
648
	 * $opts is an associative of boolean flags, including:
649
	 *   - bypassReadOnly      : Allow writes in read-only mode (since 1.20)
650
	 *
651
	 * @par Return value:
652
	 * This returns a Status, which contains all warnings and fatals that occurred
653
	 * during the operation. The 'failCount', 'successCount', and 'success' members
654
	 * will reflect each operation attempted for the given files. The StatusValue will be
655
	 * considered "OK" as long as no fatal errors occurred.
656
	 *
657
	 * @param array $ops Set of operations to execute
658
	 * @param array $opts Batch operation options
659
	 * @return StatusValue
660
	 * @since 1.20
661
	 */
662 View Code Duplication
	final public function doQuickOperations( array $ops, array $opts = [] ) {
663
		if ( empty( $opts['bypassReadOnly'] ) && $this->isReadOnly() ) {
664
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
665
		}
666
		if ( !count( $ops ) ) {
667
			return $this->newStatus(); // nothing to do
668
		}
669
670
		$ops = $this->resolveFSFileObjects( $ops );
671
		foreach ( $ops as &$op ) {
672
			$op['overwrite'] = true; // avoids RTTs in key/value stores
673
		}
674
675
		/** @noinspection PhpUnusedLocalVariableInspection */
676
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
677
678
		return $this->doQuickOperationsInternal( $ops );
679
	}
680
681
	/**
682
	 * @see FileBackend::doQuickOperations()
683
	 * @param array $ops
684
	 * @since 1.20
685
	 */
686
	abstract protected function doQuickOperationsInternal( array $ops );
687
688
	/**
689
	 * Same as doQuickOperations() except it takes a single operation.
690
	 * If you are doing a batch of operations, then use that function instead.
691
	 *
692
	 * @see FileBackend::doQuickOperations()
693
	 *
694
	 * @param array $op Operation
695
	 * @return StatusValue
696
	 * @since 1.20
697
	 */
698
	final public function doQuickOperation( array $op ) {
699
		return $this->doQuickOperations( [ $op ] );
700
	}
701
702
	/**
703
	 * Performs a single quick create operation.
704
	 * This sets $params['op'] to 'create' and passes it to doQuickOperation().
705
	 *
706
	 * @see FileBackend::doQuickOperation()
707
	 *
708
	 * @param array $params Operation parameters
709
	 * @return StatusValue
710
	 * @since 1.20
711
	 */
712
	final public function quickCreate( array $params ) {
713
		return $this->doQuickOperation( [ 'op' => 'create' ] + $params );
714
	}
715
716
	/**
717
	 * Performs a single quick store operation.
718
	 * This sets $params['op'] to 'store' and passes it to doQuickOperation().
719
	 *
720
	 * @see FileBackend::doQuickOperation()
721
	 *
722
	 * @param array $params Operation parameters
723
	 * @return StatusValue
724
	 * @since 1.20
725
	 */
726
	final public function quickStore( array $params ) {
727
		return $this->doQuickOperation( [ 'op' => 'store' ] + $params );
728
	}
729
730
	/**
731
	 * Performs a single quick copy operation.
732
	 * This sets $params['op'] to 'copy' and passes it to doQuickOperation().
733
	 *
734
	 * @see FileBackend::doQuickOperation()
735
	 *
736
	 * @param array $params Operation parameters
737
	 * @return StatusValue
738
	 * @since 1.20
739
	 */
740
	final public function quickCopy( array $params ) {
741
		return $this->doQuickOperation( [ 'op' => 'copy' ] + $params );
742
	}
743
744
	/**
745
	 * Performs a single quick move operation.
746
	 * This sets $params['op'] to 'move' and passes it to doQuickOperation().
747
	 *
748
	 * @see FileBackend::doQuickOperation()
749
	 *
750
	 * @param array $params Operation parameters
751
	 * @return StatusValue
752
	 * @since 1.20
753
	 */
754
	final public function quickMove( array $params ) {
755
		return $this->doQuickOperation( [ 'op' => 'move' ] + $params );
756
	}
757
758
	/**
759
	 * Performs a single quick delete operation.
760
	 * This sets $params['op'] to 'delete' and passes it to doQuickOperation().
761
	 *
762
	 * @see FileBackend::doQuickOperation()
763
	 *
764
	 * @param array $params Operation parameters
765
	 * @return StatusValue
766
	 * @since 1.20
767
	 */
768
	final public function quickDelete( array $params ) {
769
		return $this->doQuickOperation( [ 'op' => 'delete' ] + $params );
770
	}
771
772
	/**
773
	 * Performs a single quick describe operation.
774
	 * This sets $params['op'] to 'describe' and passes it to doQuickOperation().
775
	 *
776
	 * @see FileBackend::doQuickOperation()
777
	 *
778
	 * @param array $params Operation parameters
779
	 * @return StatusValue
780
	 * @since 1.21
781
	 */
782
	final public function quickDescribe( array $params ) {
783
		return $this->doQuickOperation( [ 'op' => 'describe' ] + $params );
784
	}
785
786
	/**
787
	 * Concatenate a list of storage files into a single file system file.
788
	 * The target path should refer to a file that is already locked or
789
	 * otherwise safe from modification from other processes. Normally,
790
	 * the file will be a new temp file, which should be adequate.
791
	 *
792
	 * @param array $params Operation parameters, include:
793
	 *   - srcs        : ordered source storage paths (e.g. chunk1, chunk2, ...)
794
	 *   - dst         : file system path to 0-byte temp file
795
	 *   - parallelize : try to do operations in parallel when possible
796
	 * @return StatusValue
797
	 */
798
	abstract public function concatenate( array $params );
799
800
	/**
801
	 * Prepare a storage directory for usage.
802
	 * This will create any required containers and parent directories.
803
	 * Backends using key/value stores only need to create the container.
804
	 *
805
	 * The 'noAccess' and 'noListing' parameters works the same as in secure(),
806
	 * except they are only applied *if* the directory/container had to be created.
807
	 * These flags should always be set for directories that have private files.
808
	 * However, setting them is not guaranteed to actually do anything.
809
	 * Additional server configuration may be needed to achieve the desired effect.
810
	 *
811
	 * @param array $params Parameters include:
812
	 *   - dir            : storage directory
813
	 *   - noAccess       : try to deny file access (since 1.20)
814
	 *   - noListing      : try to deny file listing (since 1.20)
815
	 *   - bypassReadOnly : allow writes in read-only mode (since 1.20)
816
	 * @return StatusValue
817
	 */
818 View Code Duplication
	final public function prepare( array $params ) {
819
		if ( empty( $params['bypassReadOnly'] ) && $this->isReadOnly() ) {
820
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
821
		}
822
		/** @noinspection PhpUnusedLocalVariableInspection */
823
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
824
		return $this->doPrepare( $params );
825
	}
826
827
	/**
828
	 * @see FileBackend::prepare()
829
	 * @param array $params
830
	 */
831
	abstract protected function doPrepare( array $params );
832
833
	/**
834
	 * Take measures to block web access to a storage directory and
835
	 * the container it belongs to. FS backends might add .htaccess
836
	 * files whereas key/value store backends might revoke container
837
	 * access to the storage user representing end-users in web requests.
838
	 *
839
	 * This is not guaranteed to actually make files or listings publically hidden.
840
	 * Additional server configuration may be needed to achieve the desired effect.
841
	 *
842
	 * @param array $params Parameters include:
843
	 *   - dir            : storage directory
844
	 *   - noAccess       : try to deny file access
845
	 *   - noListing      : try to deny file listing
846
	 *   - bypassReadOnly : allow writes in read-only mode (since 1.20)
847
	 * @return StatusValue
848
	 */
849 View Code Duplication
	final public function secure( array $params ) {
850
		if ( empty( $params['bypassReadOnly'] ) && $this->isReadOnly() ) {
851
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
852
		}
853
		/** @noinspection PhpUnusedLocalVariableInspection */
854
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
855
		return $this->doSecure( $params );
856
	}
857
858
	/**
859
	 * @see FileBackend::secure()
860
	 * @param array $params
861
	 */
862
	abstract protected function doSecure( array $params );
863
864
	/**
865
	 * Remove measures to block web access to a storage directory and
866
	 * the container it belongs to. FS backends might remove .htaccess
867
	 * files whereas key/value store backends might grant container
868
	 * access to the storage user representing end-users in web requests.
869
	 * This essentially can undo the result of secure() calls.
870
	 *
871
	 * This is not guaranteed to actually make files or listings publically viewable.
872
	 * Additional server configuration may be needed to achieve the desired effect.
873
	 *
874
	 * @param array $params Parameters include:
875
	 *   - dir            : storage directory
876
	 *   - access         : try to allow file access
877
	 *   - listing        : try to allow file listing
878
	 *   - bypassReadOnly : allow writes in read-only mode (since 1.20)
879
	 * @return StatusValue
880
	 * @since 1.20
881
	 */
882 View Code Duplication
	final public function publish( array $params ) {
883
		if ( empty( $params['bypassReadOnly'] ) && $this->isReadOnly() ) {
884
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
885
		}
886
		/** @noinspection PhpUnusedLocalVariableInspection */
887
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
888
		return $this->doPublish( $params );
889
	}
890
891
	/**
892
	 * @see FileBackend::publish()
893
	 * @param array $params
894
	 */
895
	abstract protected function doPublish( array $params );
896
897
	/**
898
	 * Delete a storage directory if it is empty.
899
	 * Backends using key/value stores may do nothing unless the directory
900
	 * is that of an empty container, in which case it will be deleted.
901
	 *
902
	 * @param array $params Parameters include:
903
	 *   - dir            : storage directory
904
	 *   - recursive      : recursively delete empty subdirectories first (since 1.20)
905
	 *   - bypassReadOnly : allow writes in read-only mode (since 1.20)
906
	 * @return StatusValue
907
	 */
908 View Code Duplication
	final public function clean( array $params ) {
909
		if ( empty( $params['bypassReadOnly'] ) && $this->isReadOnly() ) {
910
			return $this->newStatus( 'backend-fail-readonly', $this->name, $this->readOnly );
911
		}
912
		/** @noinspection PhpUnusedLocalVariableInspection */
913
		$scope = $this->getScopedPHPBehaviorForOps(); // try to ignore client aborts
0 ignored issues
show
Unused Code introduced by
$scope is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
914
		return $this->doClean( $params );
915
	}
916
917
	/**
918
	 * @see FileBackend::clean()
919
	 * @param array $params
920
	 */
921
	abstract protected function doClean( array $params );
922
923
	/**
924
	 * Enter file operation scope.
925
	 * This just makes PHP ignore user aborts/disconnects until the return
926
	 * value leaves scope. This returns null and does nothing in CLI mode.
927
	 *
928
	 * @return ScopedCallback|null
929
	 */
930 View Code Duplication
	final protected function getScopedPHPBehaviorForOps() {
931
		if ( PHP_SAPI != 'cli' ) { // https://bugs.php.net/bug.php?id=47540
932
			$old = ignore_user_abort( true ); // avoid half-finished operations
933
			return new ScopedCallback( function () use ( $old ) {
934
				ignore_user_abort( $old );
935
			} );
936
		}
937
938
		return null;
939
	}
940
941
	/**
942
	 * Check if a file exists at a storage path in the backend.
943
	 * This returns false if only a directory exists at the path.
944
	 *
945
	 * @param array $params Parameters include:
946
	 *   - src    : source storage path
947
	 *   - latest : use the latest available data
948
	 * @return bool|null Returns null on failure
949
	 */
950
	abstract public function fileExists( array $params );
951
952
	/**
953
	 * Get the last-modified timestamp of the file at a storage path.
954
	 *
955
	 * @param array $params Parameters include:
956
	 *   - src    : source storage path
957
	 *   - latest : use the latest available data
958
	 * @return string|bool TS_MW timestamp or false on failure
959
	 */
960
	abstract public function getFileTimestamp( array $params );
961
962
	/**
963
	 * Get the contents of a file at a storage path in the backend.
964
	 * This should be avoided for potentially large files.
965
	 *
966
	 * @param array $params Parameters include:
967
	 *   - src    : source storage path
968
	 *   - latest : use the latest available data
969
	 * @return string|bool Returns false on failure
970
	 */
971
	final public function getFileContents( array $params ) {
972
		$contents = $this->getFileContentsMulti(
973
			[ 'srcs' => [ $params['src'] ] ] + $params );
974
975
		return $contents[$params['src']];
976
	}
977
978
	/**
979
	 * Like getFileContents() except it takes an array of storage paths
980
	 * and returns a map of storage paths to strings (or null on failure).
981
	 * The map keys (paths) are in the same order as the provided list of paths.
982
	 *
983
	 * @see FileBackend::getFileContents()
984
	 *
985
	 * @param array $params Parameters include:
986
	 *   - srcs        : list of source storage paths
987
	 *   - latest      : use the latest available data
988
	 *   - parallelize : try to do operations in parallel when possible
989
	 * @return array Map of (path name => string or false on failure)
990
	 * @since 1.20
991
	 */
992
	abstract public function getFileContentsMulti( array $params );
993
994
	/**
995
	 * Get metadata about a file at a storage path in the backend.
996
	 * If the file does not exist, then this returns false.
997
	 * Otherwise, the result is an associative array that includes:
998
	 *   - headers  : map of HTTP headers used for GET/HEAD requests (name => value)
999
	 *   - metadata : map of file metadata (name => value)
1000
	 * Metadata keys and headers names will be returned in all lower-case.
1001
	 * Additional values may be included for internal use only.
1002
	 *
1003
	 * Use FileBackend::hasFeatures() to check how well this is supported.
1004
	 *
1005
	 * @param array $params
1006
	 * $params include:
1007
	 *   - src    : source storage path
1008
	 *   - latest : use the latest available data
1009
	 * @return array|bool Returns false on failure
1010
	 * @since 1.23
1011
	 */
1012
	abstract public function getFileXAttributes( array $params );
1013
1014
	/**
1015
	 * Get the size (bytes) of a file at a storage path in the backend.
1016
	 *
1017
	 * @param array $params Parameters include:
1018
	 *   - src    : source storage path
1019
	 *   - latest : use the latest available data
1020
	 * @return int|bool Returns false on failure
1021
	 */
1022
	abstract public function getFileSize( array $params );
1023
1024
	/**
1025
	 * Get quick information about a file at a storage path in the backend.
1026
	 * If the file does not exist, then this returns false.
1027
	 * Otherwise, the result is an associative array that includes:
1028
	 *   - mtime  : the last-modified timestamp (TS_MW)
1029
	 *   - size   : the file size (bytes)
1030
	 * Additional values may be included for internal use only.
1031
	 *
1032
	 * @param array $params Parameters include:
1033
	 *   - src    : source storage path
1034
	 *   - latest : use the latest available data
1035
	 * @return array|bool|null Returns null on failure
1036
	 */
1037
	abstract public function getFileStat( array $params );
1038
1039
	/**
1040
	 * Get a SHA-1 hash of the file at a storage path in the backend.
1041
	 *
1042
	 * @param array $params Parameters include:
1043
	 *   - src    : source storage path
1044
	 *   - latest : use the latest available data
1045
	 * @return string|bool Hash string or false on failure
1046
	 */
1047
	abstract public function getFileSha1Base36( array $params );
1048
1049
	/**
1050
	 * Get the properties of the file at a storage path in the backend.
1051
	 * This gives the result of FSFile::getProps() on a local copy of the file.
1052
	 *
1053
	 * @param array $params Parameters include:
1054
	 *   - src    : source storage path
1055
	 *   - latest : use the latest available data
1056
	 * @return array Returns FSFile::placeholderProps() on failure
1057
	 */
1058
	abstract public function getFileProps( array $params );
1059
1060
	/**
1061
	 * Stream the file at a storage path in the backend.
1062
	 *
1063
	 * If the file does not exists, an HTTP 404 error will be given.
1064
	 * Appropriate HTTP headers (Status, Content-Type, Content-Length)
1065
	 * will be sent if streaming began, while none will be sent otherwise.
1066
	 * Implementations should flush the output buffer before sending data.
1067
	 *
1068
	 * @param array $params Parameters include:
1069
	 *   - src      : source storage path
1070
	 *   - headers  : list of additional HTTP headers to send if the file exists
1071
	 *   - options  : HTTP request header map with lower case keys (since 1.28). Supports:
1072
	 *                range             : format is "bytes=(\d*-\d*)"
1073
	 *                if-modified-since : format is an HTTP date
1074
	 *   - headless : only include the body (and headers from "headers") (since 1.28)
1075
	 *   - latest   : use the latest available data
1076
	 *   - allowOB  : preserve any output buffers (since 1.28)
1077
	 * @return StatusValue
1078
	 */
1079
	abstract public function streamFile( array $params );
1080
1081
	/**
1082
	 * Returns a file system file, identical to the file at a storage path.
1083
	 * The file returned is either:
1084
	 *   - a) A local copy of the file at a storage path in the backend.
1085
	 *        The temporary copy will have the same extension as the source.
1086
	 *   - b) An original of the file at a storage path in the backend.
1087
	 * Temporary files may be purged when the file object falls out of scope.
1088
	 *
1089
	 * Write operations should *never* be done on this file as some backends
1090
	 * may do internal tracking or may be instances of FileBackendMultiWrite.
1091
	 * In that latter case, there are copies of the file that must stay in sync.
1092
	 * Additionally, further calls to this function may return the same file.
1093
	 *
1094
	 * @param array $params Parameters include:
1095
	 *   - src    : source storage path
1096
	 *   - latest : use the latest available data
1097
	 * @return FSFile|null Returns null on failure
1098
	 */
1099
	final public function getLocalReference( array $params ) {
1100
		$fsFiles = $this->getLocalReferenceMulti(
1101
			[ 'srcs' => [ $params['src'] ] ] + $params );
1102
1103
		return $fsFiles[$params['src']];
1104
	}
1105
1106
	/**
1107
	 * Like getLocalReference() except it takes an array of storage paths
1108
	 * and returns a map of storage paths to FSFile objects (or null on failure).
1109
	 * The map keys (paths) are in the same order as the provided list of paths.
1110
	 *
1111
	 * @see FileBackend::getLocalReference()
1112
	 *
1113
	 * @param array $params Parameters include:
1114
	 *   - srcs        : list of source storage paths
1115
	 *   - latest      : use the latest available data
1116
	 *   - parallelize : try to do operations in parallel when possible
1117
	 * @return array Map of (path name => FSFile or null on failure)
1118
	 * @since 1.20
1119
	 */
1120
	abstract public function getLocalReferenceMulti( array $params );
1121
1122
	/**
1123
	 * Get a local copy on disk of the file at a storage path in the backend.
1124
	 * The temporary copy will have the same file extension as the source.
1125
	 * Temporary files may be purged when the file object falls out of scope.
1126
	 *
1127
	 * @param array $params Parameters include:
1128
	 *   - src    : source storage path
1129
	 *   - latest : use the latest available data
1130
	 * @return TempFSFile|null Returns null on failure
1131
	 */
1132
	final public function getLocalCopy( array $params ) {
1133
		$tmpFiles = $this->getLocalCopyMulti(
1134
			[ 'srcs' => [ $params['src'] ] ] + $params );
1135
1136
		return $tmpFiles[$params['src']];
1137
	}
1138
1139
	/**
1140
	 * Like getLocalCopy() except it takes an array of storage paths and
1141
	 * returns a map of storage paths to TempFSFile objects (or null on failure).
1142
	 * The map keys (paths) are in the same order as the provided list of paths.
1143
	 *
1144
	 * @see FileBackend::getLocalCopy()
1145
	 *
1146
	 * @param array $params Parameters include:
1147
	 *   - srcs        : list of source storage paths
1148
	 *   - latest      : use the latest available data
1149
	 *   - parallelize : try to do operations in parallel when possible
1150
	 * @return array Map of (path name => TempFSFile or null on failure)
1151
	 * @since 1.20
1152
	 */
1153
	abstract public function getLocalCopyMulti( array $params );
1154
1155
	/**
1156
	 * Return an HTTP URL to a given file that requires no authentication to use.
1157
	 * The URL may be pre-authenticated (via some token in the URL) and temporary.
1158
	 * This will return null if the backend cannot make an HTTP URL for the file.
1159
	 *
1160
	 * This is useful for key/value stores when using scripts that seek around
1161
	 * large files and those scripts (and the backend) support HTTP Range headers.
1162
	 * Otherwise, one would need to use getLocalReference(), which involves loading
1163
	 * the entire file on to local disk.
1164
	 *
1165
	 * @param array $params Parameters include:
1166
	 *   - src : source storage path
1167
	 *   - ttl : lifetime (seconds) if pre-authenticated; default is 1 day
1168
	 * @return string|null
1169
	 * @since 1.21
1170
	 */
1171
	abstract public function getFileHttpUrl( array $params );
1172
1173
	/**
1174
	 * Check if a directory exists at a given storage path.
1175
	 * Backends using key/value stores will check if the path is a
1176
	 * virtual directory, meaning there are files under the given directory.
1177
	 *
1178
	 * Storage backends with eventual consistency might return stale data.
1179
	 *
1180
	 * @param array $params Parameters include:
1181
	 *   - dir : storage directory
1182
	 * @return bool|null Returns null on failure
1183
	 * @since 1.20
1184
	 */
1185
	abstract public function directoryExists( array $params );
1186
1187
	/**
1188
	 * Get an iterator to list *all* directories under a storage directory.
1189
	 * If the directory is of the form "mwstore://backend/container",
1190
	 * then all directories in the container will be listed.
1191
	 * If the directory is of form "mwstore://backend/container/dir",
1192
	 * then all directories directly under that directory will be listed.
1193
	 * Results will be storage directories relative to the given directory.
1194
	 *
1195
	 * Storage backends with eventual consistency might return stale data.
1196
	 *
1197
	 * Failures during iteration can result in FileBackendError exceptions (since 1.22).
1198
	 *
1199
	 * @param array $params Parameters include:
1200
	 *   - dir     : storage directory
1201
	 *   - topOnly : only return direct child dirs of the directory
1202
	 * @return Traversable|array|null Returns null on failure
1203
	 * @since 1.20
1204
	 */
1205
	abstract public function getDirectoryList( array $params );
1206
1207
	/**
1208
	 * Same as FileBackend::getDirectoryList() except only lists
1209
	 * directories that are immediately under the given directory.
1210
	 *
1211
	 * Storage backends with eventual consistency might return stale data.
1212
	 *
1213
	 * Failures during iteration can result in FileBackendError exceptions (since 1.22).
1214
	 *
1215
	 * @param array $params Parameters include:
1216
	 *   - dir : storage directory
1217
	 * @return Traversable|array|null Returns null on failure
1218
	 * @since 1.20
1219
	 */
1220
	final public function getTopDirectoryList( array $params ) {
1221
		return $this->getDirectoryList( [ 'topOnly' => true ] + $params );
1222
	}
1223
1224
	/**
1225
	 * Get an iterator to list *all* stored files under a storage directory.
1226
	 * If the directory is of the form "mwstore://backend/container",
1227
	 * then all files in the container will be listed.
1228
	 * If the directory is of form "mwstore://backend/container/dir",
1229
	 * then all files under that directory will be listed.
1230
	 * Results will be storage paths relative to the given directory.
1231
	 *
1232
	 * Storage backends with eventual consistency might return stale data.
1233
	 *
1234
	 * Failures during iteration can result in FileBackendError exceptions (since 1.22).
1235
	 *
1236
	 * @param array $params Parameters include:
1237
	 *   - dir        : storage directory
1238
	 *   - topOnly    : only return direct child files of the directory (since 1.20)
1239
	 *   - adviseStat : set to true if stat requests will be made on the files (since 1.22)
1240
	 * @return Traversable|array|null Returns null on failure
1241
	 */
1242
	abstract public function getFileList( array $params );
1243
1244
	/**
1245
	 * Same as FileBackend::getFileList() except only lists
1246
	 * files that are immediately under the given directory.
1247
	 *
1248
	 * Storage backends with eventual consistency might return stale data.
1249
	 *
1250
	 * Failures during iteration can result in FileBackendError exceptions (since 1.22).
1251
	 *
1252
	 * @param array $params Parameters include:
1253
	 *   - dir        : storage directory
1254
	 *   - adviseStat : set to true if stat requests will be made on the files (since 1.22)
1255
	 * @return Traversable|array|null Returns null on failure
1256
	 * @since 1.20
1257
	 */
1258
	final public function getTopFileList( array $params ) {
1259
		return $this->getFileList( [ 'topOnly' => true ] + $params );
1260
	}
1261
1262
	/**
1263
	 * Preload persistent file stat cache and property cache into in-process cache.
1264
	 * This should be used when stat calls will be made on a known list of a many files.
1265
	 *
1266
	 * @see FileBackend::getFileStat()
1267
	 *
1268
	 * @param array $paths Storage paths
1269
	 */
1270
	abstract public function preloadCache( array $paths );
1271
1272
	/**
1273
	 * Invalidate any in-process file stat and property cache.
1274
	 * If $paths is given, then only the cache for those files will be cleared.
1275
	 *
1276
	 * @see FileBackend::getFileStat()
1277
	 *
1278
	 * @param array $paths Storage paths (optional)
1279
	 */
1280
	abstract public function clearCache( array $paths = null );
1281
1282
	/**
1283
	 * Preload file stat information (concurrently if possible) into in-process cache.
1284
	 *
1285
	 * This should be used when stat calls will be made on a known list of a many files.
1286
	 * This does not make use of the persistent file stat cache.
1287
	 *
1288
	 * @see FileBackend::getFileStat()
1289
	 *
1290
	 * @param array $params Parameters include:
1291
	 *   - srcs        : list of source storage paths
1292
	 *   - latest      : use the latest available data
1293
	 * @return bool All requests proceeded without I/O errors (since 1.24)
1294
	 * @since 1.23
1295
	 */
1296
	abstract public function preloadFileStat( array $params );
1297
1298
	/**
1299
	 * Lock the files at the given storage paths in the backend.
1300
	 * This will either lock all the files or none (on failure).
1301
	 *
1302
	 * Callers should consider using getScopedFileLocks() instead.
1303
	 *
1304
	 * @param array $paths Storage paths
1305
	 * @param int $type LockManager::LOCK_* constant
1306
	 * @param int $timeout Timeout in seconds (0 means non-blocking) (since 1.24)
1307
	 * @return StatusValue
1308
	 */
1309
	final public function lockFiles( array $paths, $type, $timeout = 0 ) {
1310
		$paths = array_map( 'FileBackend::normalizeStoragePath', $paths );
1311
1312
		return $this->wrapStatus( $this->lockManager->lock( $paths, $type, $timeout ) );
0 ignored issues
show
Bug introduced by
It seems like $this->lockManager->lock($paths, $type, $timeout) can be null; however, wrapStatus() does not accept null, maybe add an additional type check?

Unless you are absolutely sure that the expression can never be null because of other conditions, we strongly recommend to add an additional type check to your code:

/** @return stdClass|null */
function mayReturnNull() { }

function doesNotAcceptNull(stdClass $x) { }

// With potential error.
function withoutCheck() {
    $x = mayReturnNull();
    doesNotAcceptNull($x); // Potential error here.
}

// Safe - Alternative 1
function withCheck1() {
    $x = mayReturnNull();
    if ( ! $x instanceof stdClass) {
        throw new \LogicException('$x must be defined.');
    }
    doesNotAcceptNull($x);
}

// Safe - Alternative 2
function withCheck2() {
    $x = mayReturnNull();
    if ($x instanceof stdClass) {
        doesNotAcceptNull($x);
    }
}
Loading history...
1313
	}
1314
1315
	/**
1316
	 * Unlock the files at the given storage paths in the backend.
1317
	 *
1318
	 * @param array $paths Storage paths
1319
	 * @param int $type LockManager::LOCK_* constant
1320
	 * @return StatusValue
1321
	 */
1322
	final public function unlockFiles( array $paths, $type ) {
1323
		$paths = array_map( 'FileBackend::normalizeStoragePath', $paths );
1324
1325
		return $this->wrapStatus( $this->lockManager->unlock( $paths, $type ) );
1326
	}
1327
1328
	/**
1329
	 * Lock the files at the given storage paths in the backend.
1330
	 * This will either lock all the files or none (on failure).
1331
	 * On failure, the StatusValue object will be updated with errors.
1332
	 *
1333
	 * Once the return value goes out scope, the locks will be released and
1334
	 * the StatusValue updated. Unlock fatals will not change the StatusValue "OK" value.
1335
	 *
1336
	 * @see ScopedLock::factory()
1337
	 *
1338
	 * @param array $paths List of storage paths or map of lock types to path lists
1339
	 * @param int|string $type LockManager::LOCK_* constant or "mixed"
1340
	 * @param StatusValue $status StatusValue to update on lock/unlock
1341
	 * @param int $timeout Timeout in seconds (0 means non-blocking) (since 1.24)
1342
	 * @return ScopedLock|null Returns null on failure
1343
	 */
1344
	final public function getScopedFileLocks(
1345
		array $paths, $type, StatusValue $status, $timeout = 0
1346
	) {
1347
		if ( $type === 'mixed' ) {
1348
			foreach ( $paths as &$typePaths ) {
1349
				$typePaths = array_map( 'FileBackend::normalizeStoragePath', $typePaths );
1350
			}
1351
		} else {
1352
			$paths = array_map( 'FileBackend::normalizeStoragePath', $paths );
1353
		}
1354
1355
		return ScopedLock::factory( $this->lockManager, $paths, $type, $status, $timeout );
1356
	}
1357
1358
	/**
1359
	 * Get an array of scoped locks needed for a batch of file operations.
1360
	 *
1361
	 * Normally, FileBackend::doOperations() handles locking, unless
1362
	 * the 'nonLocking' param is passed in. This function is useful if you
1363
	 * want the files to be locked for a broader scope than just when the
1364
	 * files are changing. For example, if you need to update DB metadata,
1365
	 * you may want to keep the files locked until finished.
1366
	 *
1367
	 * @see FileBackend::doOperations()
1368
	 *
1369
	 * @param array $ops List of file operations to FileBackend::doOperations()
1370
	 * @param StatusValue $status StatusValue to update on lock/unlock
1371
	 * @return ScopedLock|null
1372
	 * @since 1.20
1373
	 */
1374
	abstract public function getScopedLocksForOps( array $ops, StatusValue $status );
1375
1376
	/**
1377
	 * Get the root storage path of this backend.
1378
	 * All container paths are "subdirectories" of this path.
1379
	 *
1380
	 * @return string Storage path
1381
	 * @since 1.20
1382
	 */
1383
	final public function getRootStoragePath() {
1384
		return "mwstore://{$this->name}";
1385
	}
1386
1387
	/**
1388
	 * Get the storage path for the given container for this backend
1389
	 *
1390
	 * @param string $container Container name
1391
	 * @return string Storage path
1392
	 * @since 1.21
1393
	 */
1394
	final public function getContainerStoragePath( $container ) {
1395
		return $this->getRootStoragePath() . "/{$container}";
1396
	}
1397
1398
	/**
1399
	 * Get the file journal object for this backend
1400
	 *
1401
	 * @return FileJournal
1402
	 */
1403
	final public function getJournal() {
1404
		return $this->fileJournal;
1405
	}
1406
1407
	/**
1408
	 * Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile)
1409
	 *
1410
	 * The 'srcRef' field keeps any TempFSFile objects in scope for the backend to have it
1411
	 * around as long it needs (which may vary greatly depending on configuration)
1412
	 *
1413
	 * @param array $ops File operation batch for FileBaclend::doOperations()
1414
	 * @return array File operation batch
1415
	 */
1416
	protected function resolveFSFileObjects( array $ops ) {
1417
		foreach ( $ops as &$op ) {
1418
			$src = isset( $op['src'] ) ? $op['src'] : null;
1419
			if ( $src instanceof FSFile ) {
1420
				$op['srcRef'] = $src;
1421
				$op['src'] = $src->getPath();
1422
			}
1423
		}
1424
		unset( $op );
1425
1426
		return $ops;
1427
	}
1428
1429
	/**
1430
	 * Check if a given path is a "mwstore://" path.
1431
	 * This does not do any further validation or any existence checks.
1432
	 *
1433
	 * @param string $path
1434
	 * @return bool
1435
	 */
1436
	final public static function isStoragePath( $path ) {
1437
		return ( strpos( $path, 'mwstore://' ) === 0 );
1438
	}
1439
1440
	/**
1441
	 * Split a storage path into a backend name, a container name,
1442
	 * and a relative file path. The relative path may be the empty string.
1443
	 * This does not do any path normalization or traversal checks.
1444
	 *
1445
	 * @param string $storagePath
1446
	 * @return array (backend, container, rel object) or (null, null, null)
1447
	 */
1448
	final public static function splitStoragePath( $storagePath ) {
1449
		if ( self::isStoragePath( $storagePath ) ) {
1450
			// Remove the "mwstore://" prefix and split the path
1451
			$parts = explode( '/', substr( $storagePath, 10 ), 3 );
1452
			if ( count( $parts ) >= 2 && $parts[0] != '' && $parts[1] != '' ) {
1453
				if ( count( $parts ) == 3 ) {
1454
					return $parts; // e.g. "backend/container/path"
1455
				} else {
1456
					return [ $parts[0], $parts[1], '' ]; // e.g. "backend/container"
1457
				}
1458
			}
1459
		}
1460
1461
		return [ null, null, null ];
1462
	}
1463
1464
	/**
1465
	 * Normalize a storage path by cleaning up directory separators.
1466
	 * Returns null if the path is not of the format of a valid storage path.
1467
	 *
1468
	 * @param string $storagePath
1469
	 * @return string|null
1470
	 */
1471
	final public static function normalizeStoragePath( $storagePath ) {
1472
		list( $backend, $container, $relPath ) = self::splitStoragePath( $storagePath );
1473
		if ( $relPath !== null ) { // must be for this backend
1474
			$relPath = self::normalizeContainerPath( $relPath );
1475
			if ( $relPath !== null ) {
1476
				return ( $relPath != '' )
1477
					? "mwstore://{$backend}/{$container}/{$relPath}"
1478
					: "mwstore://{$backend}/{$container}";
1479
			}
1480
		}
1481
1482
		return null;
1483
	}
1484
1485
	/**
1486
	 * Get the parent storage directory of a storage path.
1487
	 * This returns a path like "mwstore://backend/container",
1488
	 * "mwstore://backend/container/...", or null if there is no parent.
1489
	 *
1490
	 * @param string $storagePath
1491
	 * @return string|null
1492
	 */
1493
	final public static function parentStoragePath( $storagePath ) {
1494
		$storagePath = dirname( $storagePath );
1495
		list( , , $rel ) = self::splitStoragePath( $storagePath );
1496
1497
		return ( $rel === null ) ? null : $storagePath;
1498
	}
1499
1500
	/**
1501
	 * Get the final extension from a storage or FS path
1502
	 *
1503
	 * @param string $path
1504
	 * @param string $case One of (rawcase, uppercase, lowercase) (since 1.24)
1505
	 * @return string
1506
	 */
1507
	final public static function extensionFromPath( $path, $case = 'lowercase' ) {
1508
		$i = strrpos( $path, '.' );
1509
		$ext = $i ? substr( $path, $i + 1 ) : '';
1510
1511
		if ( $case === 'lowercase' ) {
1512
			$ext = strtolower( $ext );
1513
		} elseif ( $case === 'uppercase' ) {
1514
			$ext = strtoupper( $ext );
1515
		}
1516
1517
		return $ext;
1518
	}
1519
1520
	/**
1521
	 * Check if a relative path has no directory traversals
1522
	 *
1523
	 * @param string $path
1524
	 * @return bool
1525
	 * @since 1.20
1526
	 */
1527
	final public static function isPathTraversalFree( $path ) {
1528
		return ( self::normalizeContainerPath( $path ) !== null );
1529
	}
1530
1531
	/**
1532
	 * Build a Content-Disposition header value per RFC 6266.
1533
	 *
1534
	 * @param string $type One of (attachment, inline)
1535
	 * @param string $filename Suggested file name (should not contain slashes)
1536
	 * @throws FileBackendError
1537
	 * @return string
1538
	 * @since 1.20
1539
	 */
1540
	final public static function makeContentDisposition( $type, $filename = '' ) {
1541
		$parts = [];
1542
1543
		$type = strtolower( $type );
1544
		if ( !in_array( $type, [ 'inline', 'attachment' ] ) ) {
1545
			throw new InvalidArgumentException( "Invalid Content-Disposition type '$type'." );
1546
		}
1547
		$parts[] = $type;
1548
1549
		if ( strlen( $filename ) ) {
1550
			$parts[] = "filename*=UTF-8''" . rawurlencode( basename( $filename ) );
1551
		}
1552
1553
		return implode( ';', $parts );
1554
	}
1555
1556
	/**
1557
	 * Validate and normalize a relative storage path.
1558
	 * Null is returned if the path involves directory traversal.
1559
	 * Traversal is insecure for FS backends and broken for others.
1560
	 *
1561
	 * This uses the same traversal protection as Title::secureAndSplit().
1562
	 *
1563
	 * @param string $path Storage path relative to a container
1564
	 * @return string|null
1565
	 */
1566
	final protected static function normalizeContainerPath( $path ) {
1567
		// Normalize directory separators
1568
		$path = strtr( $path, '\\', '/' );
1569
		// Collapse any consecutive directory separators
1570
		$path = preg_replace( '![/]{2,}!', '/', $path );
1571
		// Remove any leading directory separator
1572
		$path = ltrim( $path, '/' );
1573
		// Use the same traversal protection as Title::secureAndSplit()
1574
		if ( strpos( $path, '.' ) !== false ) {
1575
			if (
1576
				$path === '.' ||
1577
				$path === '..' ||
1578
				strpos( $path, './' ) === 0 ||
1579
				strpos( $path, '../' ) === 0 ||
1580
				strpos( $path, '/./' ) !== false ||
1581
				strpos( $path, '/../' ) !== false
1582
			) {
1583
				return null;
1584
			}
1585
		}
1586
1587
		return $path;
1588
	}
1589
1590
	/**
1591
	 * Yields the result of the status wrapper callback on either:
1592
	 *   - StatusValue::newGood() if this method is called without parameters
1593
	 *   - StatusValue::newFatal() with all parameters to this method if passed in
1594
	 *
1595
	 * @param ... string
1596
	 * @return StatusValue
1597
	 */
1598
	final protected function newStatus() {
1599
		$args = func_get_args();
1600
		if ( count( $args ) ) {
1601
			$sv = call_user_func_array( [ 'StatusValue', 'newFatal' ], $args );
1602
		} else {
1603
			$sv = StatusValue::newGood();
1604
		}
1605
1606
		return $this->wrapStatus( $sv );
1607
	}
1608
1609
	/**
1610
	 * @param StatusValue $sv
1611
	 * @return StatusValue Modified status or StatusValue subclass
1612
	 */
1613
	final protected function wrapStatus( StatusValue $sv ) {
1614
		return $this->statusWrapper ? call_user_func( $this->statusWrapper, $sv ) : $sv;
1615
	}
1616
1617
	/**
1618
	 * @param string $section
1619
	 * @return ScopedCallback|null
1620
	 */
1621
	protected function scopedProfileSection( $section ) {
1622
		if ( $this->profiler ) {
1623
			call_user_func( [ $this->profiler, 'profileIn' ], $section );
1624
			return new ScopedCallback( [ $this->profiler, 'profileOut' ], [ $section ] );
1625
		}
1626
1627
		return null;
1628
	}
1629
1630
	protected function resetOutputBuffer() {
1631
		while ( ob_get_status() ) {
1632
			if ( !ob_end_clean() ) {
1633
				// Could not remove output buffer handler; abort now
1634
				// to avoid getting in some kind of infinite loop.
1635
				break;
1636
			}
1637
		}
1638
	}
1639
}
1640