Passed
Push — master ( dca8c4...597870 )
by Thomas
02:46
created

ExcelBulkLoader::load()   B

Complexity

Conditions 8
Paths 24

Size

Total Lines 28
Code Lines 14

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
cc 8
eloc 14
nc 24
nop 1
dl 0
loc 28
rs 8.4444
c 0
b 0
f 0
1
<?php
2
3
namespace LeKoala\ExcelImportExport;
4
5
use Exception;
6
use LeKoala\SpreadCompat\SpreadCompat;
0 ignored issues
show
Bug introduced by
The type LeKoala\SpreadCompat\SpreadCompat was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
7
use SilverStripe\Dev\BulkLoader;
8
use SilverStripe\ORM\DataObject;
9
use SilverStripe\Core\Environment;
10
use SilverStripe\Dev\BulkLoader_Result;
11
12
/**
13
 * @author Koala
14
 */
15
class ExcelBulkLoader extends BulkLoader
16
{
17
    /**
18
     * Delimiter character
19
     * We use auto detection for csv because we can't ask the user what he is using
20
     *
21
     * @var string
22
     */
23
    public $delimiter = 'auto';
24
25
    /**
26
     * Enclosure character (Default: doublequote)
27
     *
28
     * @var string
29
     */
30
    public $enclosure = '"';
31
32
    /**
33
     * Identifies if the file has a header row.
34
     *
35
     * @var boolean
36
     */
37
    public $hasHeaderRow = true;
38
39
    /**
40
     * The uploaded file infos
41
     * @var array
42
     */
43
    protected $uploadFile = null;
44
45
    /**
46
     *
47
     * @var DataObject
48
     */
49
    protected $singleton = null;
50
51
    /**
52
     * @var array
53
     */
54
    protected $db = [];
55
56
    /**
57
     * Type of file if not able to determine through uploaded file
58
     *
59
     * @var string
60
     */
61
    protected $fileType = 'xlsx';
62
63
    /**
64
     * @return BulkLoader_Result
65
     */
66
    public function preview($filepath)
67
    {
68
        return $this->processAll($filepath, true);
0 ignored issues
show
Bug Best Practice introduced by
The expression return $this->processAll($filepath, true) returns the type SilverStripe\Dev\BulkLoader_Result which is incompatible with the return type mandated by SilverStripe\Dev\BulkLoader::preview() of array.

In the issue above, the returned value is violating the contract defined by the mentioned interface.

Let's take a look at an example:

interface HasName {
    /** @return string */
    public function getName();
}

class Name {
    public $name;
}

class User implements HasName {
    /** @return string|Name */
    public function getName() {
        return new Name('foo'); // This is a violation of the ``HasName`` interface
                                // which only allows a string value to be returned.
    }
}
Loading history...
69
    }
70
71
    /**
72
     * Load the given file via {@link self::processAll()} and {@link self::processRecord()}.
73
     * Optionally truncates (clear) the table before it imports.
74
     *
75
     * @return BulkLoader_Result See {@link self::processAll()}
76
     */
77
    public function load($filepath)
78
    {
79
        // A small hack to allow model admin import form to work properly
80
        if (!is_array($filepath) && isset($_FILES['_CsvFile'])) {
81
            $filepath = $_FILES['_CsvFile'];
82
        }
83
        if (is_array($filepath)) {
84
            $this->uploadFile = $filepath;
85
            $filepath = $filepath['tmp_name'];
86
        }
87
        if (is_string($filepath)) {
88
            $ext = pathinfo($filepath, PATHINFO_EXTENSION);
89
            if ($ext == 'csv' || $ext == 'xlsx') {
90
                $this->fileType = $ext;
91
            }
92
        }
93
94
        // upload is resource intensive
95
        Environment::increaseTimeLimitTo(3600);
96
        Environment::increaseMemoryLimitTo('512M');
97
98
        // get all instances of the to be imported data object
99
        if ($this->deleteExistingRecords) {
100
            // warning !!! this removes the records ONE BY ONE it can be REALLY SLOW
101
            DataObject::get($this->objectClass)->removeAll();
102
        }
103
104
        return $this->processAll($filepath);
105
    }
106
107
    /**
108
     * @return string
109
     */
110
    protected function getUploadFileExtension()
111
    {
112
        if ($this->uploadFile) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $this->uploadFile of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
113
            return pathinfo($this->uploadFile['name'], PATHINFO_EXTENSION);
0 ignored issues
show
Bug Best Practice introduced by
The expression return pathinfo($this->u...ort\PATHINFO_EXTENSION) also could return the type array which is incompatible with the documented return type string.
Loading history...
114
        }
115
        return $this->fileType;
116
    }
117
118
    /**
119
     * Merge a row with its headers
120
     *
121
     * @param array $row
122
     * @param array $headers
123
     * @param int $headersCount (optional) Limit to a specifc number of headers
124
     * @return array
125
     */
126
    protected function mergeRowWithHeaders($row, $headers, $headersCount = null)
127
    {
128
        if ($headersCount === null) {
129
            $headersCount = count($headers);
130
        }
131
        $row = array_slice($row, 0, $headersCount);
132
        $row = array_combine($headers, $row);
133
        return $row;
134
    }
135
136
    /**
137
     * @param string $filepath
138
     * @param boolean $preview
139
     */
140
    protected function processAll($filepath, $preview = false)
141
    {
142
        $results = new BulkLoader_Result();
143
        $ext = $this->getUploadFileExtension();
144
145
        if (!is_readable($filepath)) {
146
            throw new Exception("Cannot read $filepath");
147
        }
148
149
        $opts = [
150
            'separator' => $this->delimiter,
151
            'enclosure' => $this->enclosure,
152
            'extension' => $ext,
153
        ];
154
        if ($this->hasHeaderRow) {
155
            $opts['assoc'] = true;
156
        }
157
158
        $data = SpreadCompat::read($filepath, ...$opts);
159
160
        $objectClass = $this->objectClass;
161
        $objectConfig = $objectClass::config();
162
        $this->db = $objectConfig->db;
163
        $this->singleton = singleton($objectClass);
164
165
        foreach ($data as $row) {
166
            $this->processRecord(
167
                $row,
168
                $this->columnMap,
169
                $results,
170
                $preview
171
            );
172
        }
173
174
        return $results;
175
    }
176
177
    /**
178
     *
179
     * @param array $record
180
     * @param array $columnMap
181
     * @param BulkLoader_Result $results
182
     * @param boolean $preview
183
     *
184
     * @return int
185
     */
186
    protected function processRecord(
187
        $record,
188
        $columnMap,
189
        &$results,
190
        $preview = false,
191
        $makeRelations = false
192
    ) {
193
        $class = $this->objectClass;
194
195
        // find existing object, or create new one
196
        $existingObj = $this->findExistingObject($record, $columnMap);
197
198
        /** @var DataObject $obj */
199
        $obj = $existingObj ? $existingObj : new $class();
200
201
        // first run: find/create any relations and store them on the object
202
        // we can't combine runs, as other columns might rely on the relation being present
203
        if ($makeRelations) {
204
            foreach ($record as $fieldName => $val) {
205
                // don't bother querying of value is not set
206
                if ($this->isNullValue($val)) {
207
                    continue;
208
                }
209
210
                // checking for existing relations
211
                if (isset($this->relationCallbacks[$fieldName])) {
212
                    // trigger custom search method for finding a relation based on the given value
213
                    // and write it back to the relation (or create a new object)
214
                    $relationName = $this->relationCallbacks[$fieldName]['relationname'];
215
                    if ($this->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
216
                        $relationObj = $this->{$this->relationCallbacks[$fieldName]['callback']}(
217
                            $obj,
218
                            $val,
219
                            $record
220
                        );
221
                    } elseif ($obj->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
222
                        $relationObj = $obj->{$this->relationCallbacks[$fieldName]['callback']}(
223
                            $val,
224
                            $record
225
                        );
226
                    }
227
                    if (!$relationObj || !$relationObj->exists()) {
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $relationObj does not seem to be defined for all execution paths leading up to this point.
Loading history...
228
                        $relationClass = $obj->hasOneComponent($relationName);
229
                        $relationObj = new $relationClass();
230
                        //write if we aren't previewing
231
                        if (!$preview) {
232
                            $relationObj->write();
233
                        }
234
                    }
235
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
236
                    //write if we are not previewing
237
                    if (!$preview) {
238
                        $obj->write();
239
                        $obj->flushCache(); // avoid relation caching confusion
240
                    }
241
                } elseif (strpos($fieldName, '.') !== false) {
242
                    // we have a relation column with dot notation
243
                    list($relationName) = explode('.', $fieldName);
244
                    // always gives us an component (either empty or existing)
245
                    $relationObj = $obj->getComponent($relationName);
246
                    if (!$preview) {
247
                        $relationObj->write();
248
                    }
249
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
250
251
                    //write if we are not previewing
252
                    if (!$preview) {
253
                        $obj->write();
254
                        $obj->flushCache(); // avoid relation caching confusion
255
                    }
256
                }
257
            }
258
        }
259
260
261
        // second run: save data
262
263
        $db = $this->db;
264
265
        foreach ($record as $fieldName => $val) {
266
            // break out of the loop if we are previewing
267
            if ($preview) {
268
                break;
269
            }
270
271
            // Do not update ID if any exist
272
            if ($fieldName == 'ID' && $obj->ID) {
273
                continue;
274
            }
275
276
            // look up the mapping to see if this needs to map to callback
277
            $mapping = ($columnMap && isset($columnMap[$fieldName])) ? $columnMap[$fieldName]
0 ignored issues
show
Bug Best Practice introduced by
The expression $columnMap of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
278
                : null;
279
280
            // Mapping that starts with -> map to a method
281
            if ($mapping && strpos($mapping, '->') === 0) {
282
                $funcName = substr($mapping, 2);
283
284
                $this->$funcName($obj, $val, $record);
285
            } elseif ($obj->hasMethod("import{$fieldName}")) {
286
                // Try to call import_myFieldName
287
                $obj->{"import{$fieldName}"}($val, $record);
288
            } else {
289
                // Map column to field
290
                $usedName = $mapping ? $mapping : $fieldName;
291
292
                // Basic value mapping based on datatype if needed
293
                if (isset($db[$usedName])) {
294
                    switch ($db[$usedName]) {
295
                        case 'Boolean':
296
                            if ((string) $val == 'yes') {
297
                                $val = true;
298
                            } elseif ((string) $val == 'no') {
299
                                $val = false;
300
                            }
301
                    }
302
                }
303
304
                $obj->update(array($usedName => $val));
305
            }
306
        }
307
308
        // write record
309
        if (!$preview) {
310
            $obj->write();
311
        }
312
313
        // @todo better message support
314
        $message = '';
315
316
        // save to results
317
        if ($existingObj) {
318
            $results->addUpdated($obj, $message);
319
        } else {
320
            $results->addCreated($obj, $message);
321
        }
322
323
        $objID = $obj->ID;
324
325
        $obj->destroy();
326
327
        // memory usage
328
        unset($existingObj);
329
        unset($obj);
330
331
        return $objID;
332
    }
333
334
    /**
335
     * Find an existing objects based on one or more uniqueness columns
336
     * specified via {@link self::$duplicateChecks}.
337
     *
338
     * @param array $record CSV data column
339
     * @param array $columnMap
340
     *
341
     * @return mixed
342
     */
343
    public function findExistingObject($record, $columnMap)
344
    {
345
        $objectClass = $this->objectClass;
346
        $SNG_objectClass = $this->singleton;
347
348
        // checking for existing records (only if not already found)
349
        foreach ($this->duplicateChecks as $fieldName => $duplicateCheck) {
350
            if (is_string($duplicateCheck)) {
351
                // Skip current duplicate check if field value is empty
352
                if (empty($record[$duplicateCheck])) {
353
                    continue;
354
                }
355
356
                $existingRecord = $objectClass::get()
357
                    ->filter($fieldName, $record[$duplicateCheck])
358
                    ->first();
359
360
                if ($existingRecord) {
361
                    return $existingRecord;
362
                }
363
            } elseif (is_array($duplicateCheck) && isset($duplicateCheck['callback'])) {
364
                if ($this->hasMethod($duplicateCheck['callback'])) {
365
                    $existingRecord = $this->{$duplicateCheck['callback']}(
366
                        $record[$fieldName],
367
                        $record
368
                    );
369
                } elseif ($SNG_objectClass->hasMethod($duplicateCheck['callback'])) {
370
                    $existingRecord = $SNG_objectClass->{$duplicateCheck['callback']}(
371
                        $record[$fieldName],
372
                        $record
373
                    );
374
                } else {
375
                    user_error(
376
                        "CsvBulkLoader::processRecord():"
377
                            . " {$duplicateCheck['callback']} not found on importer or object class.",
378
                        E_USER_ERROR
379
                    );
380
                }
381
382
                if ($existingRecord) {
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $existingRecord does not seem to be defined for all execution paths leading up to this point.
Loading history...
383
                    return $existingRecord;
384
                }
385
            } else {
386
                user_error(
387
                    'CsvBulkLoader::processRecord(): Wrong format for $duplicateChecks',
388
                    E_USER_ERROR
389
                );
390
            }
391
        }
392
393
        return false;
394
    }
395
396
    /**
397
     * Determine whether any loaded files should be parsed with a
398
     * header-row (otherwise we rely on {@link self::$columnMap}.
399
     *
400
     * @return boolean
401
     */
402
    public function hasHeaderRow()
403
    {
404
        return ($this->hasHeaderRow || isset($this->columnMap));
405
    }
406
407
    /**
408
     * Set file type as import
409
     *
410
     * @param string $fileType
411
     * @return void
412
     */
413
    public function setFileType($fileType)
414
    {
415
        $this->fileType = $fileType;
416
    }
417
418
    /**
419
     * Get file type (default is xlsx)
420
     *
421
     * @return string
422
     */
423
    public function getFileType()
424
    {
425
        return $this->fileType;
426
    }
427
}
428