Completed
Pull Request — master (#513)
by
unknown
02:20
created

AwsS3::multipartUpload()   B

Complexity

Conditions 3
Paths 6

Size

Total Lines 27
Code Lines 16

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
dl 0
loc 27
rs 8.8571
c 0
b 0
f 0
cc 3
eloc 16
nc 6
nop 2
1
<?php
2
3
namespace Gaufrette\Adapter;
4
5
use Gaufrette\Adapter;
6
use Aws\S3\S3Client;
7
use Aws\S3\Exception\S3Exception;
8
use Gaufrette\Util;
9
10
/**
11
 * Amazon S3 adapter using the AWS SDK for PHP v2.x.
12
 *
13
 * @author  Michael Dowling <[email protected]>
14
 */
15
class AwsS3 implements Adapter,
0 ignored issues
show
Coding Style introduced by
The first item in a multi-line implements list must be on the line following the implements keyword
Loading history...
16
                       MetadataSupporter,
0 ignored issues
show
Coding Style introduced by
Expected 4 spaces before interface name; 23 found
Loading history...
17
                       ListKeysAware,
0 ignored issues
show
Coding Style introduced by
Expected 4 spaces before interface name; 23 found
Loading history...
18
                       SizeCalculator,
0 ignored issues
show
Coding Style introduced by
Expected 4 spaces before interface name; 23 found
Loading history...
19
                       MimeTypeProvider
0 ignored issues
show
Coding Style introduced by
Expected 4 spaces before interface name; 23 found
Loading history...
20
{
21
    /*
22
     * Amazon S3 does not not allow uploads > 5Go
23
     */
24
    const MAX_CONTENT_SIZE = 5368709120;
25
    const DEFAULT_PART_SIZE = 5242880;
26
27
    /** @var S3Client */
28
    protected $service;
29
    /** @var string */
30
    protected $bucket;
31
    /** @var array */
32
    protected $options;
33
    /** @var bool */
34
    protected $bucketExists;
35
    /** @var array */
36
    protected $metadata = [];
37
    /** @var bool */
38
    protected $detectContentType;
39
    /** @var int */
40
    protected $sizeLimit = self::MAX_CONTENT_SIZE;
41
    /** @var int */
42
    protected $partSize = self::DEFAULT_PART_SIZE;
43
44
    /**
45
     * @param S3Client $service
46
     * @param string   $bucket
47
     * @param array    $options
48
     * @param bool     $detectContentType
49
     */
50
    public function __construct(S3Client $service, $bucket, array $options = [], $detectContentType = false)
51
    {
52
        $this->service = $service;
53
        $this->bucket = $bucket;
54
55
        if (isset($options['size_limit']) && $options['size_limit'] <= self::MAX_CONTENT_SIZE) {
56
            $this->sizeLimit = $options['size_limit'];
57
        }
58
59
        if (isset($options['part_size']) && $options['part_size'] >= self::DEFAULT_PART_SIZE) {
60
            $this->partSize = $options['part_size'];
61
        }
62
63
64
        $this->options = array_replace(
65
            [
66
                'create' => false,
67
                'directory' => '',
68
                'acl' => 'private',
69
            ],
70
            $options
71
        );
72
73
        $this->detectContentType = $detectContentType;
74
    }
75
76
    /**
77
     * Gets the publicly accessible URL of an Amazon S3 object.
78
     *
79
     * @param string $key     Object key
80
     * @param array  $options Associative array of options used to buld the URL
81
     *                        - expires: The time at which the URL should expire
82
     *                        represented as a UNIX timestamp
83
     *                        - Any options available in the Amazon S3 GetObject
84
     *                        operation may be specified.
85
     *
86
     * @return string
87
     *
88
     * @deprecated 1.0 Resolving object path into URLs is out of the scope of this repository since v0.4. gaufrette/extras
89
     *                 provides a Filesystem decorator with a regular resolve() method. You should use it instead.
90
     *
91
     * @see https://github.com/Gaufrette/extras
92
     */
93
    public function getUrl($key, array $options = [])
94
    {
95
        @trigger_error(
0 ignored issues
show
Security Best Practice introduced by
It seems like you do not handle an error condition here. This can introduce security issues, and is generally not recommended.

If you suppress an error, we recommend checking for the error condition explicitly:

// For example instead of
@mkdir($dir);

// Better use
if (@mkdir($dir) === false) {
    throw new \RuntimeException('The directory '.$dir.' could not be created.');
}
Loading history...
96
            E_USER_DEPRECATED,
97
            'Using AwsS3::getUrl() method was deprecated since v0.4. Please chek gaufrette/extras package if you want this feature'
98
        );
99
100
        return $this->service->getObjectUrl(
101
            $this->bucket,
102
            $this->computePath($key),
103
            isset($options['expires']) ? $options['expires'] : null,
0 ignored issues
show
Unused Code introduced by
The call to S3Client::getObjectUrl() has too many arguments starting with isset($options['expires'...tions['expires'] : null.

This check compares calls to functions or methods with their respective definitions. If the call has more arguments than are defined, it raises an issue.

If a function is defined several times with a different number of parameters, the check may pick up the wrong definition and report false positives. One codebase where this has been known to happen is Wordpress.

In this case you can add the @ignore PhpDoc annotation to the duplicate definition and it will be ignored.

Loading history...
104
            $options
105
        );
106
    }
107
108
    /**
109
     * {@inheritdoc}
110
     */
111
    public function setMetadata($key, $metadata)
112
    {
113
        // BC with AmazonS3 adapter
114
        if (isset($metadata['contentType'])) {
115
            $metadata['ContentType'] = $metadata['contentType'];
116
            unset($metadata['contentType']);
117
        }
118
119
        $this->metadata[$key] = $metadata;
120
    }
121
122
    /**
123
     * {@inheritdoc}
124
     */
125
    public function getMetadata($key)
126
    {
127
        return isset($this->metadata[$key]) ? $this->metadata[$key] : [];
128
    }
129
130
    /**
131
     * {@inheritdoc}
132
     */
133
    public function read($key)
134
    {
135
        $this->ensureBucketExists();
136
        $options = $this->getOptions($key);
137
138
        try {
139
            // Get remote object
140
            $object = $this->service->getObject($options);
141
            // If there's no metadata array set up for this object, set it up
142
            if (!array_key_exists($key, $this->metadata) || !is_array($this->metadata[$key])) {
143
                $this->metadata[$key] = [];
144
            }
145
            // Make remote ContentType metadata available locally
146
            $this->metadata[$key]['ContentType'] = $object->get('ContentType');
147
148
            return (string) $object->get('Body');
149
        } catch (\Exception $e) {
150
            return false;
151
        }
152
    }
153
154
    /**
155
     * {@inheritdoc}
156
     */
157
    public function rename($sourceKey, $targetKey)
158
    {
159
        $this->ensureBucketExists();
160
        $options = $this->getOptions(
161
            $targetKey,
162
            ['CopySource' => $this->bucket.'/'.$this->computePath($sourceKey)]
163
        );
164
165
        try {
166
            $this->service->copyObject(array_merge($options, $this->getMetadata($targetKey)));
167
168
            return $this->delete($sourceKey);
169
        } catch (\Exception $e) {
170
            return false;
171
        }
172
    }
173
174
    /**
175
     * {@inheritdoc}
176
     */
177
    public function write($key, $content)
178
    {
179
        $this->ensureBucketExists();
180
        $options = $this->getOptions($key, ['Body' => $content]);
181
182
        /*
183
         * If the ContentType was not already set in the metadata, then we autodetect
184
         * it to prevent everything being served up as binary/octet-stream.
185
         */
186
        if (!isset($options['ContentType']) && $this->detectContentType) {
187
            $options['ContentType'] = $this->guessContentType($content);
188
        }
189
190
        if ($isResource = is_resource($content)) {
191
            $size = Util\Size::fromResource($content);
192
        } else {
193
            $size =  Util\Size::fromContent($content);
194
        }
195
196
        try {
197
            $success = true;
198
199
            if ($size >= $this->sizeLimit && $isResource) {
200
                $success = $this->multipartUpload($key, $content);
201
            } else if ($size <= self::MAX_CONTENT_SIZE) {
202
                $this->service->putObject($options);
203
            } else {
204
                /*
205
                 * content is a string & too big to be uploaded in one shot
206
                 * We may want to throw an exception here ?
207
                 */
208
                return false;
209
            }
210
211
            return $success ? $size : false;
0 ignored issues
show
Bug Compatibility introduced by
The expression $success ? $size : false; of type string|integer|false adds the type string to the return on line 211 which is incompatible with the return type declared by the interface Gaufrette\Adapter::write of type integer|boolean.
Loading history...
212
        } catch (\Exception $e) {
213
            return false;
214
        }
215
    }
216
217
    /**
218
     * {@inheritdoc}
219
     */
220
    public function exists($key)
221
    {
222
        return $this->service->doesObjectExist($this->bucket, $this->computePath($key));
223
    }
224
225
    /**
226
     * {@inheritdoc}
227
     */
228 View Code Duplication
    public function mtime($key)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
229
    {
230
        try {
231
            $result = $this->service->headObject($this->getOptions($key));
232
233
            return strtotime($result['LastModified']);
234
        } catch (\Exception $e) {
235
            return false;
236
        }
237
    }
238
239
    /**
240
     * {@inheritdoc}
241
     */
242 View Code Duplication
    public function size($key)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
243
    {
244
        try {
245
            $result = $this->service->headObject($this->getOptions($key));
246
247
            return $result['ContentLength'];
248
        } catch (\Exception $e) {
249
            return false;
0 ignored issues
show
Bug Best Practice introduced by
The return type of return false; (false) is incompatible with the return type declared by the interface Gaufrette\Adapter\SizeCalculator::size of type integer.

If you return a value from a function or method, it should be a sub-type of the type that is given by the parent type f.e. an interface, or abstract method. This is more formally defined by the Lizkov substitution principle, and guarantees that classes that depend on the parent type can use any instance of a child type interchangably. This principle also belongs to the SOLID principles for object oriented design.

Let’s take a look at an example:

class Author {
    private $name;

    public function __construct($name) {
        $this->name = $name;
    }

    public function getName() {
        return $this->name;
    }
}

abstract class Post {
    public function getAuthor() {
        return 'Johannes';
    }
}

class BlogPost extends Post {
    public function getAuthor() {
        return new Author('Johannes');
    }
}

class ForumPost extends Post { /* ... */ }

function my_function(Post $post) {
    echo strtoupper($post->getAuthor());
}

Our function my_function expects a Post object, and outputs the author of the post. The base class Post returns a simple string and outputting a simple string will work just fine. However, the child class BlogPost which is a sub-type of Post instead decided to return an object, and is therefore violating the SOLID principles. If a BlogPost were passed to my_function, PHP would not complain, but ultimately fail when executing the strtoupper call in its body.

Loading history...
250
        }
251
    }
252
253
    /**
254
     * {@inheritdoc}
255
     */
256
    public function keys()
257
    {
258
        return $this->listKeys();
259
    }
260
261
    /**
262
     * {@inheritdoc}
263
     */
264
    public function listKeys($prefix = '')
265
    {
266
        $this->ensureBucketExists();
267
268
        $options = ['Bucket' => $this->bucket];
269 View Code Duplication
        if ((string) $prefix != '') {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
270
            $options['Prefix'] = $this->computePath($prefix);
271
        } elseif (!empty($this->options['directory'])) {
272
            $options['Prefix'] = $this->options['directory'];
273
        }
274
275
        $keys = [];
276
        $iter = $this->service->getIterator('ListObjects', $options);
277
        foreach ($iter as $file) {
278
            $keys[] = $this->computeKey($file['Key']);
279
        }
280
281
        return $keys;
282
    }
283
284
    /**
285
     * {@inheritdoc}
286
     */
287
    public function delete($key)
288
    {
289
        try {
290
            $this->service->deleteObject($this->getOptions($key));
291
292
            return true;
293
        } catch (\Exception $e) {
294
            return false;
295
        }
296
    }
297
298
    /**
299
     * {@inheritdoc}
300
     */
301
    public function isDirectory($key)
302
    {
303
        $result = $this->service->listObjects([
304
            'Bucket' => $this->bucket,
305
            'Prefix' => rtrim($this->computePath($key), '/').'/',
306
            'MaxKeys' => 1,
307
        ]);
308
309
        return count($result['Contents']) > 0;
310
    }
311
312
    /**
313
     * Ensures the specified bucket exists. If the bucket does not exists
314
     * and the create option is set to true, it will try to create the
315
     * bucket. The bucket is created using the same region as the supplied
316
     * client object.
317
     *
318
     * @throws \RuntimeException if the bucket does not exists or could not be
319
     *                           created
320
     */
321
    protected function ensureBucketExists()
322
    {
323
        if ($this->bucketExists) {
324
            return true;
325
        }
326
327
        if ($this->bucketExists = $this->service->doesBucketExist($this->bucket)) {
328
            return true;
329
        }
330
331
        if (!$this->options['create']) {
332
            throw new \RuntimeException(sprintf(
333
                'The configured bucket "%s" does not exist.',
334
                $this->bucket
335
            ));
336
        }
337
338
        $this->service->createBucket([
339
            'Bucket' => $this->bucket,
340
            'LocationConstraint' => $this->service->getRegion()
341
        ]);
342
        $this->bucketExists = true;
343
344
        return true;
345
    }
346
347
    protected function getOptions($key, array $options = [])
348
    {
349
        $options['ACL'] = $this->options['acl'];
350
        $options['Bucket'] = $this->bucket;
351
        $options['Key'] = $this->computePath($key);
352
353
        /*
354
         * Merge global options for adapter, which are set in the constructor, with metadata.
355
         * Metadata will override global options.
356
         */
357
        $options = array_merge($this->options, $options, $this->getMetadata($key));
358
359
        return $options;
360
    }
361
362 View Code Duplication
    protected function computePath($key)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
363
    {
364
        if (empty($this->options['directory'])) {
365
            return $key;
366
        }
367
368
        return sprintf('%s/%s', $this->options['directory'], $key);
369
    }
370
371
372
    /**
373
     * MultiPart upload for big files (exceeding size_limit)
374
     *
375
     * @param $key
376
     * @param $content
377
     *
378
     * @return bool
379
     */
380
    protected function multipartUpload($key, $content)
381
    {
382
        $uploadId = $this->initiateMultipartUpload($key);
383
384
        $parts = [];
385
        $partNumber = 1;
386
387
        rewind($content);
388
389
        try {
390
            while (!feof($content)) {
391
                $result = $this->uploadNextPart($key, $content, $uploadId, $partNumber);
392
                $parts[] = [
393
                    'PartNumber' => $partNumber++,
394
                    'ETag' => $result['ETag'],
395
                ];
396
            }
397
        } catch (S3Exception $e) {
398
            $this->abortMultipartUpload($key, $uploadId);
399
400
            return false;
401
        }
402
403
        $this->completeMultipartUpload($key, $uploadId, $parts);
404
405
        return true;
406
    }
407
408
    /**
409
     * @param $key
410
     *
411
     * @return mixed
412
     */
413
    protected function initiateMultipartUpload($key)
414
    {
415
        $options = $this->getOptions($key);
416
        $result = $this->service->createMultipartUpload($options);
417
418
        return $result['UploadId'];
419
    }
420
421
    /**
422
     * @param $key
423
     * @param $content
424
     * @param $uploadId
425
     * @param $partNumber
426
     *
427
     * @return \Aws\Result
428
     */
429
    protected function uploadNextPart($key, $content, $uploadId, $partNumber)
430
    {
431
        $options = $this->getOptions(
432
            $key,
433
            [
434
                'UploadId' => $uploadId,
435
                'PartNumber' => $partNumber,
436
                'Body' => fread($content, $this->partSize),
437
            ]
438
        );
439
440
        $options = $this->getOptions($key, $options);
441
442
        return $this->service->uploadPart($options);
443
    }
444
445
    /**
446
     * @param $key
447
     * @param $uploadId
448
     * @param $parts
449
     */
450
    protected function completeMultipartUpload($key, $uploadId, $parts)
451
    {
452
       $options = $this->getOptions(
453
           $key,
454
           [
455
               'UploadId' => $uploadId,
456
               'MultipartUpload' => ['Parts' => $parts],
457
           ]
458
        );
459
        $this->service->completeMultipartUpload($options);
460
    }
461
462
    protected function abortMultipartUpload($key, $uploadId)
463
    {
464
        $options = $this->getOptions($key, ['UploadId' => $uploadId]);
465
        $this->service->abortMultipartUpload($options);
466
    }
467
468
    /**
469
     * Computes the key from the specified path.
470
     *
471
     * @param string $path
472
     *
473
     * @return string
474
     */
475
    protected function computeKey($path)
476
    {
477
        return ltrim(substr($path, strlen($this->options['directory'])), '/');
478
    }
479
480
    /**
481
     * @param string $content
482
     *
483
     * @return string
484
     */
485 View Code Duplication
    private function guessContentType($content)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
486
    {
487
        $fileInfo = new \finfo(FILEINFO_MIME_TYPE);
488
489
        if (is_resource($content)) {
490
            return $fileInfo->file(stream_get_meta_data($content)['uri']);
491
        }
492
493
        return $fileInfo->buffer($content);
494
    }
495
496 View Code Duplication
    public function mimeType($key)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
497
    {
498
        try {
499
            $result = $this->service->headObject($this->getOptions($key));
500
            return ($result['ContentType']);
501
        } catch (\Exception $e) {
502
            return false;
0 ignored issues
show
Bug Best Practice introduced by
The return type of return false; (false) is incompatible with the return type declared by the interface Gaufrette\Adapter\MimeTypeProvider::mimeType of type string.

If you return a value from a function or method, it should be a sub-type of the type that is given by the parent type f.e. an interface, or abstract method. This is more formally defined by the Lizkov substitution principle, and guarantees that classes that depend on the parent type can use any instance of a child type interchangably. This principle also belongs to the SOLID principles for object oriented design.

Let’s take a look at an example:

class Author {
    private $name;

    public function __construct($name) {
        $this->name = $name;
    }

    public function getName() {
        return $this->name;
    }
}

abstract class Post {
    public function getAuthor() {
        return 'Johannes';
    }
}

class BlogPost extends Post {
    public function getAuthor() {
        return new Author('Johannes');
    }
}

class ForumPost extends Post { /* ... */ }

function my_function(Post $post) {
    echo strtoupper($post->getAuthor());
}

Our function my_function expects a Post object, and outputs the author of the post. The base class Post returns a simple string and outputting a simple string will work just fine. However, the child class BlogPost which is a sub-type of Post instead decided to return an object, and is therefore violating the SOLID principles. If a BlogPost were passed to my_function, PHP would not complain, but ultimately fail when executing the strtoupper call in its body.

Loading history...
503
        }
504
    }
505
}
506