Passed
Push — master ( 6e1f19...dca8c4 )
by Thomas
12:39
created

ExcelBulkLoader::load()   A

Complexity

Conditions 5
Paths 8

Size

Total Lines 22
Code Lines 10

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
cc 5
eloc 10
nc 8
nop 1
dl 0
loc 22
rs 9.6111
c 0
b 0
f 0
1
<?php
2
3
namespace LeKoala\ExcelImportExport;
4
5
use Exception;
6
use LeKoala\SpreadCompat\SpreadCompat;
0 ignored issues
show
Bug introduced by
The type LeKoala\SpreadCompat\SpreadCompat was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
7
use SilverStripe\Dev\BulkLoader;
8
use SilverStripe\ORM\DataObject;
9
use SilverStripe\Core\Environment;
10
use SilverStripe\Dev\BulkLoader_Result;
11
12
/**
13
 * @author Koala
14
 */
15
class ExcelBulkLoader extends BulkLoader
16
{
17
    /**
18
     * Delimiter character (Default: comma).
19
     *
20
     * @var string
21
     */
22
    public $delimiter = ',';
23
24
    /**
25
     * Enclosure character (Default: doublequote)
26
     *
27
     * @var string
28
     */
29
    public $enclosure = '"';
30
31
    /**
32
     * Identifies if the file has a header row.
33
     *
34
     * @var boolean
35
     */
36
    public $hasHeaderRow = true;
37
38
    /**
39
     * The uploaded file infos
40
     * @var array
41
     */
42
    protected $uploadFile = null;
43
44
    /**
45
     *
46
     * @var DataObject
47
     */
48
    protected $singleton = null;
49
50
    /**
51
     * @var array
52
     */
53
    protected $db = [];
54
55
    /**
56
     * Type of file if not able to determine through uploaded file
57
     *
58
     * @var string
59
     */
60
    protected $fileType = 'xlsx';
61
62
    /**
63
     * @return BulkLoader_Result
64
     */
65
    public function preview($filepath)
66
    {
67
        return $this->processAll($filepath, true);
0 ignored issues
show
Bug Best Practice introduced by
The expression return $this->processAll($filepath, true) returns the type SilverStripe\Dev\BulkLoader_Result which is incompatible with the return type mandated by SilverStripe\Dev\BulkLoader::preview() of array.

In the issue above, the returned value is violating the contract defined by the mentioned interface.

Let's take a look at an example:

interface HasName {
    /** @return string */
    public function getName();
}

class Name {
    public $name;
}

class User implements HasName {
    /** @return string|Name */
    public function getName() {
        return new Name('foo'); // This is a violation of the ``HasName`` interface
                                // which only allows a string value to be returned.
    }
}
Loading history...
68
    }
69
70
    /**
71
     * Load the given file via {@link self::processAll()} and {@link self::processRecord()}.
72
     * Optionally truncates (clear) the table before it imports.
73
     *
74
     * @return BulkLoader_Result See {@link self::processAll()}
75
     */
76
    public function load($filepath)
77
    {
78
        // A small hack to allow model admin import form to work properly
79
        if (!is_array($filepath) && isset($_FILES['_CsvFile'])) {
80
            $filepath = $_FILES['_CsvFile'];
81
        }
82
        if (is_array($filepath)) {
83
            $this->uploadFile = $filepath;
84
            $filepath = $filepath['tmp_name'];
85
        }
86
87
        // upload is resource intensive
88
        Environment::increaseTimeLimitTo(3600);
89
        Environment::increaseMemoryLimitTo('512M');
90
91
        // get all instances of the to be imported data object
92
        if ($this->deleteExistingRecords) {
93
            // warning !!! this removes the records ONE BY ONE it can be REALLY SLOW
94
            DataObject::get($this->objectClass)->removeAll();
95
        }
96
97
        return $this->processAll($filepath);
98
    }
99
100
    /**
101
     * @return string
102
     */
103
    protected function getUploadFileExtension()
104
    {
105
        if ($this->uploadFile) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $this->uploadFile of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
106
            return pathinfo($this->uploadFile['name'], PATHINFO_EXTENSION);
0 ignored issues
show
Bug Best Practice introduced by
The expression return pathinfo($this->u...ort\PATHINFO_EXTENSION) also could return the type array which is incompatible with the documented return type string.
Loading history...
107
        }
108
        return $this->fileType;
109
    }
110
111
    /**
112
     * Merge a row with its headers
113
     *
114
     * @param array $row
115
     * @param array $headers
116
     * @param int $headersCount (optional) Limit to a specifc number of headers
117
     * @return array
118
     */
119
    protected function mergeRowWithHeaders($row, $headers, $headersCount = null)
120
    {
121
        if ($headersCount === null) {
122
            $headersCount = count($headers);
123
        }
124
        $row = array_slice($row, 0, $headersCount);
125
        $row = array_combine($headers, $row);
126
        return $row;
127
    }
128
129
    /**
130
     * @param string $filepath
131
     * @param boolean $preview
132
     */
133
    protected function processAll($filepath, $preview = false)
134
    {
135
        $results = new BulkLoader_Result();
136
        $ext = $this->getUploadFileExtension();
137
138
        if (!is_readable($filepath)) {
139
            throw new Exception("Cannot read $filepath");
140
        }
141
142
        $opts = [
143
            'separator' => $this->delimiter,
144
            'enclosure' => $this->enclosure,
145
            'extension' => $ext,
146
        ];
147
        if ($this->hasHeaderRow) {
148
            $opts['assoc'] = true;
149
        }
150
151
        $data = SpreadCompat::read($filepath, ...$opts);
152
153
        $objectClass = $this->objectClass;
154
        $objectConfig = $objectClass::config();
155
        $this->db = $objectConfig->db;
156
        $this->singleton = singleton($objectClass);
157
158
        foreach ($data as $row) {
159
            $this->processRecord(
160
                $row,
161
                $this->columnMap,
162
                $results,
163
                $preview
164
            );
165
        }
166
167
        return $results;
168
    }
169
170
    /**
171
     *
172
     * @param array $record
173
     * @param array $columnMap
174
     * @param BulkLoader_Result $results
175
     * @param boolean $preview
176
     *
177
     * @return int
178
     */
179
    protected function processRecord(
180
        $record,
181
        $columnMap,
182
        &$results,
183
        $preview = false,
184
        $makeRelations = false
185
    ) {
186
        $class = $this->objectClass;
187
188
        // find existing object, or create new one
189
        $existingObj = $this->findExistingObject($record, $columnMap);
190
191
        /* @var $obj DataObject */
192
        $obj = $existingObj ? $existingObj : new $class();
193
194
        // first run: find/create any relations and store them on the object
195
        // we can't combine runs, as other columns might rely on the relation being present
196
        if ($makeRelations) {
197
            foreach ($record as $fieldName => $val) {
198
                // don't bother querying of value is not set
199
                if ($this->isNullValue($val)) {
200
                    continue;
201
                }
202
203
                // checking for existing relations
204
                if (isset($this->relationCallbacks[$fieldName])) {
205
                    // trigger custom search method for finding a relation based on the given value
206
                    // and write it back to the relation (or create a new object)
207
                    $relationName = $this->relationCallbacks[$fieldName]['relationname'];
208
                    if ($this->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
209
                        $relationObj = $this->{$this->relationCallbacks[$fieldName]['callback']}(
210
                            $obj,
211
                            $val,
212
                            $record
213
                        );
214
                    } elseif ($obj->hasMethod($this->relationCallbacks[$fieldName]['callback'])) {
215
                        $relationObj = $obj->{$this->relationCallbacks[$fieldName]['callback']}(
216
                            $val,
217
                            $record
218
                        );
219
                    }
220
                    if (!$relationObj || !$relationObj->exists()) {
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $relationObj does not seem to be defined for all execution paths leading up to this point.
Loading history...
221
                        $relationClass = $obj->hasOneComponent($relationName);
222
                        $relationObj = new $relationClass();
223
                        //write if we aren't previewing
224
                        if (!$preview) {
225
                            $relationObj->write();
226
                        }
227
                    }
228
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
229
                    //write if we are not previewing
230
                    if (!$preview) {
231
                        $obj->write();
232
                        $obj->flushCache(); // avoid relation caching confusion
233
                    }
234
                } elseif (strpos($fieldName, '.') !== false) {
235
                    // we have a relation column with dot notation
236
                    list($relationName) = explode('.', $fieldName);
237
                    // always gives us an component (either empty or existing)
238
                    $relationObj = $obj->getComponent($relationName);
239
                    if (!$preview) {
240
                        $relationObj->write();
241
                    }
242
                    $obj->{"{$relationName}ID"} = $relationObj->ID;
243
244
                    //write if we are not previewing
245
                    if (!$preview) {
246
                        $obj->write();
247
                        $obj->flushCache(); // avoid relation caching confusion
248
                    }
249
                }
250
            }
251
        }
252
253
254
        // second run: save data
255
256
        $db = $this->db;
257
258
        foreach ($record as $fieldName => $val) {
259
            // break out of the loop if we are previewing
260
            if ($preview) {
261
                break;
262
            }
263
264
            // Do not update ID if any exist
265
            if ($fieldName == 'ID' && $obj->ID) {
266
                continue;
267
            }
268
269
            // look up the mapping to see if this needs to map to callback
270
            $mapping = ($columnMap && isset($columnMap[$fieldName])) ? $columnMap[$fieldName]
0 ignored issues
show
Bug Best Practice introduced by
The expression $columnMap of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
271
                : null;
272
273
            // Mapping that starts with -> map to a method
274
            if ($mapping && strpos($mapping, '->') === 0) {
275
                $funcName = substr($mapping, 2);
276
277
                $this->$funcName($obj, $val, $record);
278
            } elseif ($obj->hasMethod("import{$fieldName}")) {
279
                // Try to call import_myFieldName
280
                $obj->{"import{$fieldName}"}($val, $record);
281
            } else {
282
                // Map column to field
283
                $usedName = $mapping ? $mapping : $fieldName;
284
285
                // Basic value mapping based on datatype if needed
286
                if (isset($db[$usedName])) {
287
                    switch ($db[$usedName]) {
288
                        case 'Boolean':
289
                            if ((string) $val == 'yes') {
290
                                $val = true;
291
                            } elseif ((string) $val == 'no') {
292
                                $val = false;
293
                            }
294
                    }
295
                }
296
297
                $obj->update(array($usedName => $val));
298
            }
299
        }
300
301
        // write record
302
        if (!$preview) {
303
            $obj->write();
304
        }
305
306
        // @todo better message support
307
        $message = '';
308
309
        // save to results
310
        if ($existingObj) {
311
            $results->addUpdated($obj, $message);
312
        } else {
313
            $results->addCreated($obj, $message);
314
        }
315
316
        $objID = $obj->ID;
317
318
        $obj->destroy();
319
320
        // memory usage
321
        unset($existingObj);
322
        unset($obj);
323
324
        return $objID;
325
    }
326
327
    /**
328
     * Find an existing objects based on one or more uniqueness columns
329
     * specified via {@link self::$duplicateChecks}.
330
     *
331
     * @param array $record CSV data column
332
     * @param array $columnMap
333
     *
334
     * @return mixed
335
     */
336
    public function findExistingObject($record, $columnMap)
337
    {
338
        $objectClass = $this->objectClass;
339
        $SNG_objectClass = $this->singleton;
340
341
        // checking for existing records (only if not already found)
342
        foreach ($this->duplicateChecks as $fieldName => $duplicateCheck) {
343
            if (is_string($duplicateCheck)) {
344
                // Skip current duplicate check if field value is empty
345
                if (empty($record[$duplicateCheck])) {
346
                    continue;
347
                }
348
349
                $existingRecord = $objectClass::get()
350
                    ->filter($fieldName, $record[$duplicateCheck])
351
                    ->first();
352
353
                if ($existingRecord) {
354
                    return $existingRecord;
355
                }
356
            } elseif (is_array($duplicateCheck) && isset($duplicateCheck['callback'])) {
357
                if ($this->hasMethod($duplicateCheck['callback'])) {
358
                    $existingRecord = $this->{$duplicateCheck['callback']}(
359
                        $record[$fieldName],
360
                        $record
361
                    );
362
                } elseif ($SNG_objectClass->hasMethod($duplicateCheck['callback'])) {
363
                    $existingRecord = $SNG_objectClass->{$duplicateCheck['callback']}(
364
                        $record[$fieldName],
365
                        $record
366
                    );
367
                } else {
368
                    user_error(
369
                        "CsvBulkLoader::processRecord():"
370
                            . " {$duplicateCheck['callback']} not found on importer or object class.",
371
                        E_USER_ERROR
372
                    );
373
                }
374
375
                if ($existingRecord) {
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $existingRecord does not seem to be defined for all execution paths leading up to this point.
Loading history...
376
                    return $existingRecord;
377
                }
378
            } else {
379
                user_error(
380
                    'CsvBulkLoader::processRecord(): Wrong format for $duplicateChecks',
381
                    E_USER_ERROR
382
                );
383
            }
384
        }
385
386
        return false;
387
    }
388
389
    /**
390
     * Determine whether any loaded files should be parsed with a
391
     * header-row (otherwise we rely on {@link self::$columnMap}.
392
     *
393
     * @return boolean
394
     */
395
    public function hasHeaderRow()
396
    {
397
        return ($this->hasHeaderRow || isset($this->columnMap));
398
    }
399
400
    /**
401
     * Set file type as import
402
     *
403
     * @param string $fileType
404
     * @return void
405
     */
406
    public function setFileType($fileType)
407
    {
408
        $this->fileType = $fileType;
409
    }
410
411
    /**
412
     * Get file type (default is xlsx)
413
     *
414
     * @return string
415
     */
416
    public function getFileType()
417
    {
418
        return $this->fileType;
419
    }
420
}
421