Passed
Push — master ( be0918...94f3f1 )
by Thomas
13:41
created

ExcelBulkLoader::load()   F

Complexity

Conditions 16
Paths 816

Size

Total Lines 59
Code Lines 32

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
cc 16
eloc 32
nc 816
nop 1
dl 0
loc 59
rs 1.6555
c 0
b 0
f 0

How to fix   Long Method    Complexity   

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

1
<?php
2
3
namespace LeKoala\ExcelImportExport;
4
5
use Exception;
6
use LeKoala\SpreadCompat\SpreadCompat;
0 ignored issues
show
Bug introduced by
The type LeKoala\SpreadCompat\SpreadCompat was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
7
use SilverStripe\Dev\BulkLoader;
8
use SilverStripe\ORM\DataObject;
9
use SilverStripe\Core\Environment;
10
use SilverStripe\Dev\BulkLoader_Result;
11
use SilverStripe\Control\HTTPResponse_Exception;
12
use SilverStripe\Core\ClassInfo;
13
use SilverStripe\ORM\DB;
14
15
/**
16
 * @author Koala
17
 */
18
class ExcelBulkLoader extends BulkLoader
19
{
20
    private bool $useTransaction = false;
21
22
    /**
23
     * Delimiter character
24
     * We use auto detection for csv because we can't ask the user what he is using
25
     *
26
     * @var string
27
     */
28
    public $delimiter = 'auto';
29
30
    /**
31
     * Enclosure character (Default: doublequote)
32
     *
33
     * @var string
34
     */
35
    public $enclosure = '"';
36
37
    /**
38
     * Identifies if the file has a header row.
39
     *
40
     * @var boolean
41
     */
42
    public $hasHeaderRow = true;
43
44
    /**
45
     * @var array<string,string>
46
     */
47
    public $duplicateChecks = [
48
        'ID' => 'ID',
49
    ];
50
51
    /**
52
     * The uploaded file infos
53
     * @var array<mixed>
54
     */
55
    protected $uploadFile = null;
56
57
    /**
58
     *
59
     * @var DataObject
60
     */
61
    protected $singleton = null;
62
63
    /**
64
     * @var array<mixed>
65
     */
66
    protected $db = [];
67
68
    /**
69
     * Type of file if not able to determine through uploaded file
70
     *
71
     * @var string
72
     */
73
    protected $fileType = 'xlsx';
74
75
    /**
76
     * @return BulkLoader_Result
77
     */
78
    public function preview($filepath)
79
    {
80
        return $this->processAll($filepath, true);
0 ignored issues
show
Bug Best Practice introduced by
The expression return $this->processAll($filepath, true) returns the type SilverStripe\Dev\BulkLoader_Result which is incompatible with the return type mandated by SilverStripe\Dev\BulkLoader::preview() of array.

In the issue above, the returned value is violating the contract defined by the mentioned interface.

Let's take a look at an example:

interface HasName {
    /** @return string */
    public function getName();
}

class Name {
    public $name;
}

class User implements HasName {
    /** @return string|Name */
    public function getName() {
        return new Name('foo'); // This is a violation of the ``HasName`` interface
                                // which only allows a string value to be returned.
    }
}
Loading history...
81
    }
82
83
    /**
84
     * Load the given file via {@link self::processAll()} and {@link self::processRecord()}.
85
     * Optionally truncates (clear) the table before it imports.
86
     *
87
     * @return BulkLoader_Result See {@link self::processAll()}
88
     */
89
    public function load($filepath)
90
    {
91
        // A small hack to allow model admin import form to work properly
92
        if (!is_array($filepath) && isset($_FILES['_CsvFile'])) {
93
            $filepath = $_FILES['_CsvFile'];
94
        }
95
        if (is_array($filepath)) {
96
            $this->uploadFile = $filepath;
97
            $filepath = $filepath['tmp_name'];
98
        }
99
        if (is_string($filepath)) {
100
            $ext = pathinfo($filepath, PATHINFO_EXTENSION);
101
            if ($ext == 'csv' || $ext == 'xlsx') {
102
                $this->fileType = $ext;
103
            }
104
        }
105
106
        // upload is resource intensive
107
        Environment::increaseTimeLimitTo(3600);
108
        Environment::increaseMemoryLimitTo('512M');
109
110
        if ($this->useTransaction) {
111
            DB::get_conn()->transactionStart();
112
        }
113
114
        try {
115
            //get all instances of the to be imported data object
116
            if ($this->deleteExistingRecords) {
117
                if ($this->getCheckPermissions()) {
118
                    // We need to check each record, in case there's some fancy conditional logic in the canDelete method.
119
                    // If we can't delete even a single record, we should bail because otherwise the result would not be
120
                    // what the user expects.
121
                    /** @var DataObject $record */
122
                    foreach (DataObject::get($this->objectClass) as $record) {
123
                        if (!$record->canDelete()) {
124
                            $type = $record->i18n_singular_name();
125
                            throw new HTTPResponse_Exception(
126
                                _t(__CLASS__ . '.CANNOT_DELETE', "Not allowed to delete '{type}' records", ["type" => $type]),
127
                                403
128
                            );
129
                        }
130
                    }
131
                }
132
                DataObject::get($this->objectClass)->removeAll();
133
            }
134
135
            $result = $this->processAll($filepath);
136
137
            if ($this->useTransaction) {
138
                DB::get_conn()->transactionEnd();
139
            }
140
        } catch (Exception $e) {
141
            if ($this->useTransaction) {
142
                DB::get_conn()->transactionRollback();
143
            }
144
            $code = $e->getCode() ?: 500;
145
            throw new HTTPResponse_Exception($e->getMessage(), $code);
146
        }
147
        return $result;
148
    }
149
150
    /**
151
     * @return string
152
     */
153
    protected function getUploadFileExtension()
154
    {
155
        if ($this->uploadFile) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $this->uploadFile of type array<mixed,mixed> is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
156
            return pathinfo($this->uploadFile['name'], PATHINFO_EXTENSION);
0 ignored issues
show
Bug Best Practice introduced by
The expression return pathinfo($this->u...ort\PATHINFO_EXTENSION) also could return the type array which is incompatible with the documented return type string.
Loading history...
157
        }
158
        return $this->fileType;
159
    }
160
161
    /**
162
     * Merge a row with its headers
163
     *
164
     * @param array $row
165
     * @param array $headers
166
     * @param int $headersCount (optional) Limit to a specifc number of headers
167
     * @return array
168
     */
169
    protected function mergeRowWithHeaders($row, $headers, $headersCount = null)
170
    {
171
        if ($headersCount === null) {
172
            $headersCount = count($headers);
173
        }
174
        $row = array_slice($row, 0, $headersCount);
175
        $row = array_combine($headers, $row);
176
        return $row;
177
    }
178
179
    /**
180
     * @param string $filepath
181
     * @param boolean $preview
182
     */
183
    protected function processAll($filepath, $preview = false)
184
    {
185
        $this->extend('onBeforeProcessAll', $filepath, $preview);
186
187
        $results = new BulkLoader_Result();
188
        $ext = $this->getUploadFileExtension();
189
190
        if (!is_readable($filepath)) {
191
            throw new Exception("Cannot read $filepath");
192
        }
193
194
        $opts = [
195
            'separator' => $this->delimiter,
196
            'enclosure' => $this->enclosure,
197
            'extension' => $ext,
198
        ];
199
        if ($this->hasHeaderRow) {
200
            $opts['assoc'] = true;
201
        }
202
203
        $data = SpreadCompat::read($filepath, ...$opts);
204
205
        $objectClass = $this->objectClass;
206
        $objectConfig = $objectClass::config();
207
        $this->db = $objectConfig->db;
208
        $this->singleton = singleton($objectClass);
209
210
        foreach ($data as $row) {
211
            $this->processRecord(
212
                $row,
213
                $this->columnMap,
214
                $results,
215
                $preview
216
            );
217
        }
218
219
        $this->extend('onAfterProcessAll', $result, $preview);
220
221
        return $results;
222
    }
223
224
    /**
225
     *
226
     * @param array $record
227
     * @param array $columnMap
228
     * @param BulkLoader_Result $results
229
     * @param boolean $preview
230
     *
231
     * @return int
232
     */
233
    protected function processRecord(
234
        $record,
235
        $columnMap,
236
        &$results,
237
        $preview = false,
238
        $makeRelations = false
239
    ) {
240
        // find existing object, or create new one
241
        $existingObj = $this->findExistingObject($record, $columnMap);
242
        $alreadyExists = (bool) $existingObj;
243
244
        // If we can't edit the existing object, bail early.
245
        if ($this->getCheckPermissions() && !$preview && $alreadyExists && !$existingObj->canEdit()) {
246
            $type = $existingObj->i18n_singular_name();
247
            throw new HTTPResponse_Exception(
248
                _t(BulkLoader::class . '.CANNOT_EDIT', "Not allowed to edit '{type}' records", ["type" => $type]),
249
                403
250
            );
251
        }
252
253
        $class = $record['ClassName'] ?? $this->objectClass;
254
        $obj = $existingObj ? $existingObj : new $class();
255
256
        // If we can't create a new record, bail out early.
257
        if ($this->getCheckPermissions() && !$preview && !$alreadyExists && !$obj->canCreate()) {
258
            $type = $obj->i18n_singular_name();
259
            throw new HTTPResponse_Exception(
260
                _t(BulkLoader::class . '.CANNOT_CREATE', "Not allowed to create '{type}' records", ["type" => $type]),
261
                403
262
            );
263
        }
264
265
        // first run: find/create any relations and store them on the object
266
        // we can't combine runs, as other columns might rely on the relation being present
267
        if ($makeRelations) {
268
            foreach ($record as $fieldName => $val) {
269
                // don't bother querying of value is not set
270
                if ($this->isNullValue($val)) {
271
                    continue;
272
                }
273
274
                // checking for existing relations
275
                if (isset($this->relationCallbacks[$fieldName])) {
276
                    // trigger custom search method for finding a relation based on the given value
277
                    // and write it back to the relation (or create a new object)
278
                    $relationName = $this->relationCallbacks[$fieldName]['relationname'];
279
                    if ($this->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
280
                        $relationObj = $this->{$this->relationCallbacks[$fieldName]['callback']}(
281
                            $obj,
282
                            $val,
283
                            $record
284
                        );
285
                    } elseif ($obj->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
286
                        $relationObj = $obj->{$this->relationCallbacks[$fieldName]['callback']}(
287
                            $val,
288
                            $record
289
                        );
290
                    }
291
                    if (!$relationObj || !$relationObj->exists()) {
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $relationObj does not seem to be defined for all execution paths leading up to this point.
Loading history...
292
                        $relationClass = $obj->hasOneComponent($relationName);
293
                        $relationObj = new $relationClass();
294
                        //write if we aren't previewing
295
                        if (!$preview) {
296
                            $relationObj->write();
297
                        }
298
                    }
299
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
300
                    //write if we are not previewing
301
                    if (!$preview) {
302
                        $obj->write();
303
                        $obj->flushCache(); // avoid relation caching confusion
304
                    }
305
                } elseif (strpos($fieldName, '.') !== false) {
306
                    // we have a relation column with dot notation
307
                    list($relationName) = explode('.', $fieldName);
308
                    // always gives us an component (either empty or existing)
309
                    $relationObj = $obj->getComponent($relationName);
310
                    if (!$preview) {
311
                        $relationObj->write();
312
                    }
313
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
314
315
                    //write if we are not previewing
316
                    if (!$preview) {
317
                        $obj->write();
318
                        $obj->flushCache(); // avoid relation caching confusion
319
                    }
320
                }
321
            }
322
        }
323
324
325
        // second run: save data
326
327
        $db = $this->db;
328
329
        foreach ($record as $fieldName => $val) {
330
            // break out of the loop if we are previewing
331
            if ($preview) {
332
                break;
333
            }
334
335
            // Do not update ID if any exist
336
            if ($fieldName == 'ID' && $obj->ID) {
337
                continue;
338
            }
339
340
            // look up the mapping to see if this needs to map to callback
341
            $mapping = ($columnMap && isset($columnMap[$fieldName])) ? $columnMap[$fieldName]
0 ignored issues
show
Bug Best Practice introduced by
The expression $columnMap of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
342
                : null;
343
344
            // Mapping that starts with -> map to a method
345
            if ($mapping && strpos($mapping, '->') === 0) {
346
                $funcName = substr($mapping, 2);
347
348
                $this->$funcName($obj, $val, $record);
349
            } elseif ($obj->hasMethod("import{$fieldName}")) {
350
                // Try to call import_myFieldName
351
                $obj->{"import{$fieldName}"}($val, $record);
352
            } else {
353
                // Map column to field
354
                $usedName = $mapping ? $mapping : $fieldName;
355
356
                // Basic value mapping based on datatype if needed
357
                if (isset($db[$usedName])) {
358
                    switch ($db[$usedName]) {
359
                        case 'Boolean':
360
                            if ((string) $val == 'yes') {
361
                                $val = true;
362
                            } elseif ((string) $val == 'no') {
363
                                $val = false;
364
                            }
365
                    }
366
                }
367
368
                $obj->update(array($usedName => $val));
369
            }
370
        }
371
372
        // write record
373
        if (!$preview) {
374
            $obj->write();
375
        }
376
377
        // @todo better message support
378
        $message = '';
379
380
        // save to results
381
        if ($existingObj) {
382
            $results->addUpdated($obj, $message);
383
        } else {
384
            $results->addCreated($obj, $message);
385
        }
386
387
        $objID = $obj->ID;
388
389
        $obj->destroy();
390
391
        // memory usage
392
        unset($existingObj);
393
        unset($obj);
394
395
        return $objID;
396
    }
397
398
    /**
399
     * Find an existing objects based on one or more uniqueness columns
400
     * specified via {@link self::$duplicateChecks}.
401
     *
402
     * @param array $record CSV data column
403
     * @param array $columnMap
404
     *
405
     * @return DataObject|false
406
     */
407
    public function findExistingObject($record, $columnMap = [])
408
    {
409
        $SNG_objectClass = $this->singleton;
410
411
        // checking for existing records (only if not already found)
412
        foreach ($this->duplicateChecks as $fieldName => $duplicateCheck) {
413
            $existingRecord = null;
414
            if (is_string($duplicateCheck)) {
415
                // Skip current duplicate check if field value is empty
416
                if (empty($record[$duplicateCheck])) {
417
                    continue;
418
                }
419
420
                $dbFieldValue = $record[$duplicateCheck];
421
422
                // Even if $record['ClassName'] is a subclass, this will work
423
                $existingRecord = DataObject::get($this->objectClass)
424
                    ->filter($duplicateCheck, $dbFieldValue)
425
                    ->first();
426
427
                if ($existingRecord) {
428
                    return $existingRecord;
429
                }
430
            } elseif (is_array($duplicateCheck) && isset($duplicateCheck['callback'])) {
431
                if ($this->hasMethod($duplicateCheck['callback'])) {
432
                    $existingRecord = $this->{$duplicateCheck['callback']}(
433
                        $record[$fieldName],
434
                        $record
435
                    );
436
                } elseif ($SNG_objectClass->hasMethod($duplicateCheck['callback'])) {
437
                    $existingRecord = $SNG_objectClass->{$duplicateCheck['callback']}(
438
                        $record[$fieldName],
439
                        $record
440
                    );
441
                } else {
442
                    throw new \RuntimeException(
443
                        "ExcelBulkLoader::processRecord():"
444
                            . " {$duplicateCheck['callback']} not found on importer or object class."
445
                    );
446
                }
447
448
                if ($existingRecord) {
449
                    return $existingRecord;
450
                }
451
            } else {
452
                throw new \InvalidArgumentException(
453
                    'ExcelBulkLoader::processRecord(): Wrong format for $duplicateChecks'
454
                );
455
            }
456
        }
457
458
        return false;
459
    }
460
461
    /**
462
     * Determine whether any loaded files should be parsed with a
463
     * header-row (otherwise we rely on {@link self::$columnMap}.
464
     *
465
     * @return boolean
466
     */
467
    public function hasHeaderRow()
468
    {
469
        return ($this->hasHeaderRow || isset($this->columnMap));
470
    }
471
472
    /**
473
     * Set file type as import
474
     *
475
     * @param string $fileType
476
     * @return void
477
     */
478
    public function setFileType($fileType)
479
    {
480
        $this->fileType = $fileType;
481
    }
482
483
    /**
484
     * Get file type (default is xlsx)
485
     *
486
     * @return string
487
     */
488
    public function getFileType()
489
    {
490
        return $this->fileType;
491
    }
492
493
    /**
494
     * If true, will wrap everything in a transaction
495
     */
496
    public function getUseTransaction(): bool
497
    {
498
        return $this->useTransaction;
499
    }
500
501
    /**
502
     * Determines if everything will be wrapped in a transaction
503
     */
504
    public function setCheckPermissions(bool $value): ExcelBulkLoader
505
    {
506
        $this->useTransaction = $value;
507
        return $this;
508
    }
509
}
510