Completed
Push — master ( b77c8b...5ec1d2 )
by
unknown
16s queued 13s
created

ResumeUploader::uploadV1()   F

Complexity

Conditions 30
Paths 636

Size

Total Lines 113
Code Lines 71

Duplication

Lines 0
Ratio 0 %

Code Coverage

Tests 4
CRAP Score 30

Importance

Changes 0
Metric Value
cc 30
eloc 71
nc 636
nop 2
dl 0
loc 113
ccs 4
cts 4
cp 1
crap 30
rs 0.5054
c 0
b 0
f 0

How to fix   Long Method    Complexity   

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

1
<?php
2
3
namespace Qiniu\Storage;
4
5
use Qiniu\Config;
6
use Qiniu\Http\Client;
7
use Qiniu\Http\Error;
8
use Qiniu\Enum\SplitUploadVersion;
9
use Qiniu\Http\RequestOptions;
10
11
/**
12
 * 断点续上传类, 该类主要实现了断点续上传中的分块上传,
13
 * 以及相应地创建块和创建文件过程.
14
 *
15
 * @link http://developer.qiniu.com/docs/v6/api/reference/up/mkblk.html
16
 * @link http://developer.qiniu.com/docs/v6/api/reference/up/mkfile.html
17
 */
18
final class ResumeUploader
19
{
20
    private $upToken;
21
    private $key;
22
    private $inputStream;
23
    private $size;
24
    private $params;
25
    private $mime;
26
    private $contexts;
27
    private $finishedEtags;
28
    private $host;
29
    private $bucket;
30
    private $currentUrl;
31
    private $config;
32
    private $resumeRecordFile;
33
    private $version;
34
    private $partSize;
35
    /**
36
     * @var RequestOptions
37
     */
38
    private $reqOpt;
39
40 6
    /**
41
     * 上传二进制流到七牛
42
     *
43
     * @param string $upToken 上传凭证
44
     * @param string $key 上传文件名
45
     * @param resource $inputStream 上传二进制流
46
     * @param int $size 上传流的大小
47
     * @param array<string, string> $params 自定义变量
48
     * @param string $mime 上传数据的mimeType
49
     * @param Config $config
50 6
     * @param string $resumeRecordFile 断点续传的已上传的部分信息记录文件
51 6
     * @param string $version 分片上传版本 目前支持v1/v2版本 默认v1
52 6
     * @param int $partSize 分片上传v2字段 默认大小为4MB 分片大小范围为1 MB - 1 GB
53 6
     * @param RequestOptions $reqOpt 分片上传v2字段 默认大小为4MB 分片大小范围为1 MB - 1 GB
54 6
     * @throws \Exception
55 6
     *
56 6
     * @link http://developer.qiniu.com/docs/v6/api/overview/up/response/vars.html#xvar
57 6
     */
58
    public function __construct(
59 6
        $upToken,
60 6
        $key,
61
        $inputStream,
62
        $size,
63
        $params,
64 6
        $mime,
65 6
        $config,
66
        $resumeRecordFile = null,
67
        $version = 'v1',
68 6
        $partSize = config::BLOCK_SIZE,
69 6
        $reqOpt = null
70
    ) {
71
72
        $this->upToken = $upToken;
73
        $this->key = $key;
74 6
        $this->inputStream = $inputStream;
75
        $this->size = $size;
76 6
        $this->params = $params;
77 6
        $this->mime = $mime;
78 6
        $this->contexts = array();
79 6
        $this->finishedEtags = array("etags" => array(), "uploadId" => "", "expiredAt" => 0, "uploaded" => 0);
80 6
        $this->config = $config;
81
        $this->resumeRecordFile = $resumeRecordFile ? $resumeRecordFile : null;
82
        $this->partSize = $partSize ? $partSize : config::BLOCK_SIZE;
83 6
84 6
        if ($reqOpt === null) {
85 6
            $reqOpt = new RequestOptions();
86 6
        }
87 6
        $this->reqOpt = $reqOpt;
88 6
89 6
        try {
90 3
            $this->version = SplitUploadVersion::from($version ? $version : 'v1');
91 3
        } catch (\Exception $e) {
92
            throw new \Exception("only support v1/v2 now!", 0, $e);
93
        }
94
95 3
        list($accessKey, $bucket, $err) = \Qiniu\explodeUpToken($upToken);
96 3
        $this->bucket = $bucket;
97 3
        if ($err != null) {
98 6
            return array(null, $err);
99 3
        }
100 3
101 3
        list($upHost, $err) = $config->getUpHostV2($accessKey, $bucket);
102
        if ($err != null) {
103 6
            throw new \Exception($err->message(), 1);
104
        }
105
        $this->host = $upHost;
106 6
    }
107 6
108 6
    /**
109 6
     * 上传操作
110
     * @param $fname string 文件名
111
     *
112
     * @throws \Exception
113
     */
114
    public function upload($fname)
115 6
    {
116
        $blkputRets = null;
117 6
        // get upload record from resumeRecordFile
118 6
        if ($this->resumeRecordFile != null) {
0 ignored issues
show
Bug introduced by
It seems like you are loosely comparing $this->resumeRecordFile of type null|string against null; this is ambiguous if the string can be empty. Consider using a strict comparison !== instead.
Loading history...
119
            if (file_exists($this->resumeRecordFile)) {
120
                $stream = fopen($this->resumeRecordFile, 'r');
121 6
                if ($stream) {
0 ignored issues
show
introduced by
$stream is of type resource, thus it always evaluated to false.
Loading history...
122
                    $streamLen = filesize($this->resumeRecordFile);
123 6
                    if ($streamLen > 0) {
124 6
                        $contents = fread($stream, $streamLen);
125 6
                        fclose($stream);
126 6
                        if ($contents) {
127 6
                            $blkputRets = json_decode($contents, true);
128 6
                            if ($blkputRets === null) {
129 6
                                error_log("resumeFile contents decode error");
130
                            }
131
                        } else {
132
                            error_log("read resumeFile failed");
133
                        }
134
                    } else {
135 6
                        error_log("resumeFile is empty");
136
                    }
137
                } else {
138
                    error_log("resumeFile open failed");
139
                }
140
            } else {
141 6
                error_log("resumeFile not exists");
142
            }
143 6
        }
144 6
145 6
        if ($this->version == SplitUploadVersion::V1) {
0 ignored issues
show
introduced by
The condition $this->version == Qiniu\...\SplitUploadVersion::V1 is always false.
Loading history...
146 6
            return $this->uploadV1($fname, $blkputRets);
147
        } elseif ($this->version == SplitUploadVersion::V2) {
0 ignored issues
show
introduced by
The condition $this->version == Qiniu\...\SplitUploadVersion::V2 is always false.
Loading history...
148
            return $this->uploadV2($fname, $blkputRets);
149 6
        } else {
150
            throw new \Exception("only support v1/v2 now!");
151
        }
152 6
    }
153
154
    /**
155 6
     * @param string $fname 文件名
156
     * @param null|array $blkputRets
157 6
     *
158 6
     * @throws \Exception
159 6
     */
160
    private function uploadV1($fname, $blkputRets = null)
161
    {
162 6
        // 尝试恢复恢复已上传的数据
163
        $isResumeUpload = $blkputRets !== null;
164 6
        $this->contexts = array();
165 6
166
        if ($blkputRets) {
167 6
            if (isset($blkputRets['contexts']) && isset($blkputRets['uploaded']) &&
168
                is_array($blkputRets['contexts']) && is_int($blkputRets['uploaded'])
169
            ) {
170
                $this->contexts = array_map(function ($ctx) {
171
                    if (is_array($ctx)) {
172
                        return $ctx;
173
                    } else {
174
                        // 兼容旧版本(旧版本没有存储 expireAt)
175
                        return array(
176
                            "ctx" => $ctx,
177
                            "expiredAt" => 0,
178
                        );
179
                    }
180
                }, $blkputRets['contexts']);
181
            }
182
        }
183
184
        // 上传分片
185
        $uploaded = 0;
186
        while ($uploaded < $this->size) {
187
            $blockSize = $this->blockSize($uploaded);
188
            $blockIndex = $uploaded / $this->partSize;
189
            if (!is_int($blockIndex)) {
190
                throw new \Exception("v1 part size changed");
191
            }
192
            // 如果已上传该分片且没有过期
193
            if (isset($this->contexts[$blockIndex]) && $this->contexts[$blockIndex]["expiredAt"] >= time()) {
194
                $uploaded += $blockSize;
195
                fseek($this->inputStream, $blockSize, SEEK_CUR);
196
                continue;
197
            }
198
            $data = fread($this->inputStream, $blockSize);
199
            if ($data === false) {
200
                throw new \Exception("file read failed", 1);
201
            }
202
            $crc = \Qiniu\crc32_data($data);
203
            $response = $this->makeBlock($data, $blockSize);
204
205
206
            $ret = null;
207
            if ($response->ok() && $response->json() != null) {
208
                $ret = $response->json();
209
            }
210
            if ($response->statusCode < 0) {
211
                list($accessKey, $bucket, $err) = \Qiniu\explodeUpToken($this->upToken);
212
                if ($err != null) {
213
                    return array(null, $err);
214
                }
215
                list($upHostBackup, $err) = $this->config->getUpBackupHostV2($accessKey, $bucket);
216
                if ($err != null) {
217
                    return array(null, $err);
218
                }
219
                $this->host = $upHostBackup;
220
            }
221
222
            if ($response->needRetry() || !isset($ret['crc32']) || $crc != $ret['crc32']) {
223
                $response = $this->makeBlock($data, $blockSize);
224
                $ret = $response->json();
225
            }
226
            if (!$response->ok() || !isset($ret['crc32']) || $crc != $ret['crc32']) {
227
                return array(null, new Error($this->currentUrl, $response));
228
            }
229
230
            // 如果可以在已上传取到说明是过期分片直接修改已上传信息,否则是新的片添加到已上传分片尾部
231
            if (isset($this->contexts[$blockIndex])) {
232
                $this->contexts[$blockIndex] = array(
233
                    'ctx' => $ret['ctx'],
234
                    'expiredAt' => $ret['expired_at'],
235
                );
236
            } else {
237
                array_push($this->contexts, array(
238
                    'ctx' => $ret['ctx'],
239
                    'expiredAt' => $ret['expired_at'],
240
                ));
241
            }
242
            $uploaded += $blockSize;
243
244
            // 记录断点
245
            if ($this->resumeRecordFile !== null) {
246
                $recordData = array(
247
                    'contexts' => $this->contexts,
248
                    'uploaded' => $uploaded
249
                );
250
                $recordData = json_encode($recordData);
251
252
                if ($recordData) {
253
                    $isWritten = file_put_contents($this->resumeRecordFile, $recordData);
254
                    if ($isWritten === false) {
255
                        error_log("write resumeRecordFile failed");
256
                    }
257
                } else {
258
                    error_log('resumeRecordData encode failed');
259
                }
260
            }
261
        }
262
263
        // 完成上传
264
        list($ret, $err) = $this->makeFile($fname);
265
        if ($err !== null) {
266
            $response = $err->getResponse();
267
            if ($isResumeUpload && $response->statusCode === 701) {
268
                fseek($this->inputStream, 0);
269
                return $this->uploadV1($fname);
270
            }
271
        }
272
        return array($ret, $err);
273
    }
274
275
    /**
276
     * @param string $fname 文件名
277
     * @param null|array $blkputRets
278
     *
279
     * @throws \Exception
280
     */
281
    private function uploadV2($fname, $blkputRets = null)
282
    {
283
        $uploaded = 0;
284
        $partNumber = 1;
285
        $encodedObjectName = $this->key ? \Qiniu\base64_urlSafeEncode($this->key) : '~';
286
287
        $isResumeUpload = $blkputRets !== null;
288
        if ($blkputRets) {
289
            if (isset($blkputRets["etags"]) && isset($blkputRets["uploadId"]) &&
290
                isset($blkputRets["expiredAt"]) && $blkputRets["expiredAt"] > time() &&
291
                $blkputRets["uploaded"] > 0 && is_array($blkputRets["etags"]) &&
292
                is_string($blkputRets["uploadId"]) && is_int($blkputRets["expiredAt"])
293
            ) {
294
                $this->finishedEtags['etags'] = $blkputRets["etags"];
295
                $this->finishedEtags["uploadId"] = $blkputRets["uploadId"];
296
                $this->finishedEtags["expiredAt"] = $blkputRets["expiredAt"];
297
                $this->finishedEtags["uploaded"] = $blkputRets["uploaded"];
298
                $uploaded = $blkputRets["uploaded"];
299
                $partNumber = count($this->finishedEtags["etags"]) + 1;
300
            } else {
301
                $this->makeInitReq($encodedObjectName);
302
            }
303
        } else {
304
            $this->makeInitReq($encodedObjectName);
305
        }
306
307
        fseek($this->inputStream, $uploaded);
308
        while ($uploaded < $this->size) {
309
            $blockSize = $this->blockSize($uploaded);
310
            $data = fread($this->inputStream, $blockSize);
311
            if ($data === false) {
312
                throw new \Exception("file read failed", 1);
313
            }
314
            $md5 = md5($data);
315
            $response = $this->uploadPart(
316
                $data,
317
                $partNumber,
318
                $this->finishedEtags["uploadId"],
319
                $encodedObjectName,
320
                $md5
321
            );
322
323
            $ret = null;
324
            if ($response->ok() && $response->json() != null) {
325
                $ret = $response->json();
326
            }
327
            if ($response->statusCode < 0) {
328
                list($accessKey, $bucket, $err) = \Qiniu\explodeUpToken($this->upToken);
329
                if ($err != null) {
330
                    return array(null, $err);
331
                }
332
                list($upHostBackup, $err) = $this->config->getUpBackupHostV2($accessKey, $bucket);
333
                if ($err != null) {
334
                    return array(null, $err);
335
                }
336
                $this->host = $upHostBackup;
337
            }
338
339
            if ($response->needRetry() || !isset($ret['md5']) || $md5 != $ret['md5']) {
340
                $response = $this->uploadPart(
341
                    $data,
342
                    $partNumber,
343
                    $this->finishedEtags["uploadId"],
344
                    $encodedObjectName,
345
                    $md5
346
                );
347
                $ret = $response->json();
348
            }
349
            if ($isResumeUpload && $response->statusCode === 612) {
350
                return $this->uploadV2($fname);
351
            }
352
            if (!$response->ok() || !isset($ret['md5']) || $md5 != $ret['md5']) {
353
                return array(null, new Error($this->currentUrl, $response));
354
            }
355
            $blockStatus = array('etag' => $ret['etag'], 'partNumber' => $partNumber);
356
            array_push($this->finishedEtags['etags'], $blockStatus);
357
            $partNumber += 1;
358
359
            $uploaded += $blockSize;
360
            $this->finishedEtags['uploaded'] = $uploaded;
361
362
            if ($this->resumeRecordFile !== null) {
363
                $recordData = json_encode($this->finishedEtags);
364
                if ($recordData) {
365
                    $isWritten = file_put_contents($this->resumeRecordFile, $recordData);
366
                    if ($isWritten === false) {
367
                        error_log("write resumeRecordFile failed");
368
                    }
369
                } else {
370
                    error_log('resumeRecordData encode failed');
371
                }
372
            }
373
        }
374
375
        list($ret, $err) = $this->completeParts($fname, $this->finishedEtags['uploadId'], $encodedObjectName);
376
        if ($err !== null) {
377
            $response = $err->getResponse();
378
            if ($isResumeUpload && $response->statusCode === 612) {
379
                return $this->uploadV2($fname);
380
            }
381
        }
382
        return array($ret, $err);
383
    }
384
385
    /**
386
     * 创建块
387
     */
388
    private function makeBlock($block, $blockSize)
389
    {
390
        $url = $this->host . '/mkblk/' . $blockSize;
391
        return $this->post($url, $block);
392
    }
393
394
    private function fileUrl($fname)
395
    {
396
        $url = $this->host . '/mkfile/' . $this->size;
397
        $url .= '/mimeType/' . \Qiniu\base64_urlSafeEncode($this->mime);
398
        if ($this->key != null) {
399
            $url .= '/key/' . \Qiniu\base64_urlSafeEncode($this->key);
400
        }
401
        $url .= '/fname/' . \Qiniu\base64_urlSafeEncode($fname);
402
        if (!empty($this->params)) {
403
            foreach ($this->params as $key => $value) {
404
                $val = \Qiniu\base64_urlSafeEncode($value);
405
                $url .= "/$key/$val";
406
            }
407
        }
408
        return $url;
409
    }
410
411
    /**
412
     * 创建文件
413
     *
414
     * @param string $fname 文件名
415
     * @return array{array | null, Error | null}
0 ignored issues
show
Documentation Bug introduced by
The doc comment array{array | null, Error | null} at position 2 could not be parsed: Expected ':' at position 2, but found 'array'.
Loading history...
416
     */
417
    private function makeFile($fname)
418
    {
419
        $url = $this->fileUrl($fname);
420
        $body = implode(',', array_map(function ($ctx) {
421
            return $ctx['ctx'];
422
        }, $this->contexts));
423
        $response = $this->post($url, $body);
424
        if ($response->needRetry()) {
425
            $response = $this->post($url, $body);
426
        }
427
        if ($response->statusCode === 200 || $response->statusCode === 701) {
428
            if ($this->resumeRecordFile !== null) {
429
                @unlink($this->resumeRecordFile);
0 ignored issues
show
Security Best Practice introduced by
It seems like you do not handle an error condition for unlink(). This can introduce security issues, and is generally not recommended. ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-unhandled  annotation

429
                /** @scrutinizer ignore-unhandled */ @unlink($this->resumeRecordFile);

If you suppress an error, we recommend checking for the error condition explicitly:

// For example instead of
@mkdir($dir);

// Better use
if (@mkdir($dir) === false) {
    throw new \RuntimeException('The directory '.$dir.' could not be created.');
}
Loading history...
430
            }
431
        }
432
        if (!$response->ok()) {
433
            return array(null, new Error($this->currentUrl, $response));
434
        }
435
        return array($response->json(), null);
436
    }
437
438
    private function post($url, $data)
439
    {
440
        $this->currentUrl = $url;
441
        $headers = array('Authorization' => 'UpToken ' . $this->upToken);
442
        return Client::post($url, $data, $headers, $this->reqOpt);
443
    }
444
445
    private function blockSize($uploaded)
446
    {
447
        if ($this->size < $uploaded + $this->partSize) {
448
            return $this->size - $uploaded;
449
        }
450
        return $this->partSize;
451
    }
452
453
    private function makeInitReq($encodedObjectName)
454
    {
455
        $res = $this->initReq($encodedObjectName);
456
        $this->finishedEtags["uploadId"] = $res['uploadId'];
457
        $this->finishedEtags["expiredAt"] = $res['expireAt'];
458
    }
459
460
    /**
461
     * 初始化上传任务
462
     */
463
    private function initReq($encodedObjectName)
464
    {
465
        $url = $this->host . '/buckets/' . $this->bucket . '/objects/' . $encodedObjectName . '/uploads';
466
        $headers = array(
467
            'Authorization' => 'UpToken ' . $this->upToken,
468
            'Content-Type' => 'application/json'
469
        );
470
        $response = $this->postWithHeaders($url, null, $headers);
471
        return $response->json();
472
    }
473
474
    /**
475
     * 分块上传v2
476
     */
477
    private function uploadPart($block, $partNumber, $uploadId, $encodedObjectName, $md5)
478
    {
479
        $headers = array(
480
            'Authorization' => 'UpToken ' . $this->upToken,
481
            'Content-Type' => 'application/octet-stream',
482
            'Content-MD5' => $md5
483
        );
484
        $url = $this->host . '/buckets/' . $this->bucket . '/objects/' . $encodedObjectName .
485
            '/uploads/' . $uploadId . '/' . $partNumber;
486
        $response = $this->put($url, $block, $headers);
487
        if ($response->statusCode === 612) {
488
            if ($this->resumeRecordFile !== null) {
489
                @unlink($this->resumeRecordFile);
0 ignored issues
show
Security Best Practice introduced by
It seems like you do not handle an error condition for unlink(). This can introduce security issues, and is generally not recommended. ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-unhandled  annotation

489
                /** @scrutinizer ignore-unhandled */ @unlink($this->resumeRecordFile);

If you suppress an error, we recommend checking for the error condition explicitly:

// For example instead of
@mkdir($dir);

// Better use
if (@mkdir($dir) === false) {
    throw new \RuntimeException('The directory '.$dir.' could not be created.');
}
Loading history...
490
            }
491
        }
492
        return $response;
493
    }
494
495
    /**
496
     * 完成分片上传V2
497
     *
498
     * @param string $fname 文件名
499
     * @param int $uploadId 由 {@see initReq} 获取
500
     * @param string $encodedObjectName 经过编码的存储路径
501
     * @return array{array | null, Error | null}
0 ignored issues
show
Documentation Bug introduced by
The doc comment array{array | null, Error | null} at position 2 could not be parsed: Expected ':' at position 2, but found 'array'.
Loading history...
502
     */
503
    private function completeParts($fname, $uploadId, $encodedObjectName)
504
    {
505
        $headers = array(
506
            'Authorization' => 'UpToken ' . $this->upToken,
507
            'Content-Type' => 'application/json'
508
        );
509
        $etags = $this->finishedEtags['etags'];
510
        $sortedEtags = \Qiniu\arraySort($etags, 'partNumber');
511
        $metadata = array();
512
        $customVars = array();
513
        if ($this->params) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $this->params of type array<string,string> is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
514
            foreach ($this->params as $k => $v) {
515
                if (strpos($k, 'x:') === 0) {
516
                    $customVars[$k] = $v;
517
                } elseif (strpos($k, 'x-qn-meta-') === 0) {
518
                    $metadata[$k] = $v;
519
                }
520
            }
521
        }
522
        if (empty($metadata)) {
523
            $metadata = null;
524
        }
525
        if (empty($customVars)) {
526
            $customVars = null;
527
        }
528
        $body = array(
529
            'fname' => $fname,
530
            'mimeType' => $this->mime,
531
            'metadata' => $metadata,
532
            'customVars' => $customVars,
533
            'parts' => $sortedEtags
534
        );
535
        $jsonBody = json_encode($body);
536
        $url = $this->host . '/buckets/' . $this->bucket . '/objects/' . $encodedObjectName . '/uploads/' . $uploadId;
537
        $response = $this->postWithHeaders($url, $jsonBody, $headers);
538
        if ($response->needRetry()) {
539
            $response = $this->postWithHeaders($url, $jsonBody, $headers);
540
        }
541
        if ($response->statusCode === 200 || $response->statusCode === 612) {
542
            if ($this->resumeRecordFile !== null) {
543
                @unlink($this->resumeRecordFile);
0 ignored issues
show
Security Best Practice introduced by
It seems like you do not handle an error condition for unlink(). This can introduce security issues, and is generally not recommended. ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-unhandled  annotation

543
                /** @scrutinizer ignore-unhandled */ @unlink($this->resumeRecordFile);

If you suppress an error, we recommend checking for the error condition explicitly:

// For example instead of
@mkdir($dir);

// Better use
if (@mkdir($dir) === false) {
    throw new \RuntimeException('The directory '.$dir.' could not be created.');
}
Loading history...
544
            }
545
        }
546
        if (!$response->ok()) {
547
            return array(null, new Error($this->currentUrl, $response));
548
        }
549
        return array($response->json(), null);
550
    }
551
552
    private function put($url, $data, $headers)
553
    {
554
        $this->currentUrl = $url;
555
        return Client::put($url, $data, $headers, $this->reqOpt);
556
    }
557
558
    private function postWithHeaders($url, $data, $headers)
559
    {
560
        $this->currentUrl = $url;
561
        return Client::post($url, $data, $headers, $this->reqOpt);
562
    }
563
}
564