Completed
Push — 3.2 ( 8c2d5f...e78c57 )
by
unknown
08:28 queued 06:51
created

ZohoDatabaseCopier   F

Complexity

Total Complexity 89

Size/Duplication

Total Lines 519
Duplicated Lines 5.97 %

Coupling/Cohesion

Components 1
Dependencies 21

Importance

Changes 0
Metric Value
wmc 89
lcom 1
cbo 21
dl 31
loc 519
rs 2
c 0
b 0
f 0

4 Methods

Rating   Name   Duplication   Size   Complexity  
A __construct() 13 13 2
D fetchUserFromZoho() 0 61 18
F fetchFromZoho() 6 161 25
F fetchFromZohoInBulk() 12 229 44

How to fix   Duplicated Code    Complexity   

Duplicated Code

Duplicate code is one of the most pungent code smells. A rule that is often used is to re-structure code once it is duplicated in three or more places.

Common duplication problems, and corresponding solutions are:

Complex Class

 Tip:   Before tackling complexity, make sure that you eliminate any duplication first. This often can reduce the size of classes significantly.

Complex classes like ZohoDatabaseCopier often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes. You can also have a look at the cohesion graph to spot any un-connected, or weakly-connected components.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

While breaking up the class, it is a good idea to analyze how other classes use ZohoDatabaseCopier, and based on these observations, apply Extract Interface, too.

1
<?php
2
3
namespace Wabel\Zoho\CRM\Copy;
4
5
use Doctrine\DBAL\Connection;
6
use Psr\Log\LoggerInterface;
7
use Psr\Log\NullLogger;
8
use Wabel\Zoho\CRM\AbstractZohoDao;
9
use Wabel\Zoho\CRM\ZohoClient;
10
use zcrmsdk\crm\crud\ZCRMRecord;
11
use zcrmsdk\crm\exception\ZCRMException;
12
use ZipArchive;
13
14
/**
15
 * This class is in charge of synchronizing one table of your database with Zoho records.
16
 */
17
class ZohoDatabaseCopier
18
{
19
    /**
20
     * @var Connection
21
     */
22
    private $connection;
23
24
    private $prefix;
25
26
    /**
27
     * @var ZohoChangeListener[]
28
     */
29
    private $listeners;
30
31
    /**
32
     * @var LoggerInterface
33
     */
34
    private $logger;
35
36
    /**
37
     * @var LocalChangesTracker
38
     */
39
    private $localChangesTracker;
40
    /**
41
     * @var ZohoUserService
42
     */
43
    private $zohoUserService;
44
45
    /**
46
     * ZohoDatabaseCopier constructor.
47
     *
48
     * @param Connection $connection
49
     * @param string $prefix Prefix for the table name in DB
50
     * @param ZohoChangeListener[] $listeners The list of listeners called when a record is inserted or updated.
51
     */
52 View Code Duplication
    public function __construct(Connection $connection, ZohoUserService $zohoUserService, $prefix = 'zoho_', array $listeners = [], LoggerInterface $logger = null)
0 ignored issues
show
Duplication introduced by
This method seems to be duplicated in your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
53
    {
54
        $this->connection = $connection;
55
        $this->prefix = $prefix;
56
        $this->listeners = $listeners;
57
        if ($logger === null) {
58
            $this->logger = new NullLogger();
59
        } else {
60
            $this->logger = $logger;
61
        }
62
        $this->localChangesTracker = new LocalChangesTracker($connection, $this->logger);
63
        $this->zohoUserService = $zohoUserService;
64
    }
65
66
    /**
67
     * @throws \Doctrine\DBAL\DBALException
68
     * @throws \Doctrine\DBAL\Schema\SchemaException
69
     * @throws \Wabel\Zoho\CRM\Exception\ZohoCRMResponseException
70
     */
71
    public function fetchUserFromZoho()
72
    {
73
        $users = $this->zohoUserService->getUsers();
74
        $tableName = 'users';
75
        $this->logger->info('Fetched ' . count($users) . ' records for table ' . $tableName);
76
77
        $table = $this->connection->getSchemaManager()->createSchema()->getTable($tableName);
78
79
        $select = $this->connection->prepare('SELECT * FROM ' . $tableName . ' WHERE id = :id');
80
81
        $this->connection->beginTransaction();
82
        foreach ($users as $user) {
0 ignored issues
show
Bug introduced by
The expression $users of type array|null is not guaranteed to be traversable. How about adding an additional type check?

There are different options of fixing this problem.

  1. If you want to be on the safe side, you can add an additional type-check:

    $collection = json_decode($data, true);
    if ( ! is_array($collection)) {
        throw new \RuntimeException('$collection must be an array.');
    }
    
    foreach ($collection as $item) { /** ... */ }
    
  2. If you are sure that the expression is traversable, you might want to add a doc comment cast to improve IDE auto-completion and static analysis:

    /** @var array $collection */
    $collection = json_decode($data, true);
    
    foreach ($collection as $item) { /** .. */ }
    
  3. Mark the issue as a false-positive: Just hover the remove button, in the top-right corner of this issue for more options.

Loading history...
83
            $data = [];
84
            $types = [];
85
            foreach ($table->getColumns() as $column) {
86
                if ($column->getName() === 'id') {
87
                    continue;
88
                } else {
89
                    $fieldMethod = ZohoDatabaseHelper::getUserMethodNameFromField($column->getName());
90
                    if (method_exists($user, $fieldMethod)
91
                        && (!is_array($user->{$fieldMethod}()) && !is_object($user->{$fieldMethod}()))
92
                    ) {
93
                        $data[$column->getName()] = $user->{$fieldMethod}();
94
                    } elseif (method_exists($user, $fieldMethod)
95
                        && is_array($user->{$fieldMethod}())
96
                        && array_key_exists('name', $user->{$fieldMethod}())
97
                        && array_key_exists('id', $user->{$fieldMethod}())
98
                    ) {
99
                        $data[$column->getName()] = $user->{$fieldMethod}()['name'];
100
                    } elseif (method_exists($user, $fieldMethod)
101
                        && is_object($user->{$fieldMethod}()) && method_exists($user->{$fieldMethod}(), 'getName')
102
                    ) {
103
                        $object = $user->{$fieldMethod}();
104
                        $data[$column->getName()] = $object->getName();
105
                    } elseif ($column->getName() === 'Currency') {
106
                        //Todo: Do a pull request about \ZCRMUser::geCurrency() to \ZCRMUser::getCurrency()
107
                        $data[$column->getName()] = $user->geCurrency();
108
                    } else {
109
                        continue;
110
                    }
111
                }
112
            }
113
            $select->execute(['id' => $user->getId()]);
114
            $result = $select->fetch(\PDO::FETCH_ASSOC);
115
            if ($result === false && $data) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $data of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
116
                $this->logger->debug(sprintf('Inserting record with ID \'%s\' in table %s...', $user->getId(), $tableName));
117
118
                $data['id'] = $user->getId();
119
                $types['id'] = 'string';
120
121
                $this->connection->insert($tableName, $data, $types);
122
            } elseif ($data) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $data of type array is implicitly converted to a boolean; are you sure this is intended? If so, consider using ! empty($expr) instead to make it clear that you intend to check for an array without elements.

This check marks implicit conversions of arrays to boolean values in a comparison. While in PHP an empty array is considered to be equal (but not identical) to false, this is not always apparent.

Consider making the comparison explicit by using empty(..) or ! empty(...) instead.

Loading history...
123
                $this->logger->debug(sprintf('Updating record with ID \'%s\' in table %s...', $user->getId(), $tableName));
124
                $identifier = ['id' => $user->getId()];
125
                $types['id'] = 'string';
126
                $this->connection->update($tableName, $data, $identifier, $types);
127
            }
128
129
        }
130
        $this->connection->commit();
131
    }
132
133
    /**
134
     * @param AbstractZohoDao $dao
135
     * @param bool $incrementalSync Whether we synchronize only the modified files or everything.
136
     * @param bool $twoWaysSync
137
     * @param bool $throwErrors
138
     * @param string $modifiedSince
139
     *
140
     * @throws \Doctrine\DBAL\DBALException
141
     * @throws \Doctrine\DBAL\Schema\SchemaException
142
     * @throws \Wabel\Zoho\CRM\Exception\ZohoCRMResponseException
143
     */
144
    public function fetchFromZoho(AbstractZohoDao $dao, $incrementalSync = true, $twoWaysSync = true, $throwErrors = true, $modifiedSince = null)
145
    {
146
        $tableName = ZohoDatabaseHelper::getTableName($dao, $this->prefix);
147
148
        $totalRecords = 0;
149
        $totalRecordsDeleted = 0;
150
151
        try {
152
            if ($incrementalSync) {
153
                // Let's get the last modification date:
154
                $tableDetail = $this->connection->getSchemaManager()->listTableDetails($tableName);
155
                $lastActivityTime = null;
156
                if ($modifiedSince) {
0 ignored issues
show
Bug Best Practice introduced by
The expression $modifiedSince of type string|null is loosely compared to true; this is ambiguous if the string can be empty. You might want to explicitly use !== null instead.

In PHP, under loose comparison (like ==, or !=, or switch conditions), values of different types might be equal.

For string values, the empty string '' is a special case, in particular the following results might be unexpected:

''   == false // true
''   == null  // true
'ab' == false // false
'ab' == null  // false

// It is often better to use strict comparison
'' === false // false
'' === null  // false
Loading history...
157
                    $lastActivityTime = new \DateTime($modifiedSince);
158
                } else {
159
                    if ($tableDetail->hasColumn('modifiedTime')) {
160
                        $lastActivityTime = $this->connection->fetchColumn('SELECT MAX(modifiedTime) FROM ' . $tableName);
161
                    }
162
                    if (!$lastActivityTime && $tableDetail->hasColumn('createdTime')) {
163
                        $lastActivityTime = $this->connection->fetchColumn('SELECT MAX(createdTime) FROM ' . $tableName);
164
                    }
165
166
                    if ($lastActivityTime !== null) {
167
                        $lastActivityTime = new \DateTime($lastActivityTime, new \DateTimeZone($dao->getZohoClient()->getTimezone()));
168
                        // Let's add one second to the last activity time (otherwise, we are fetching again the last record in DB).
169
                        $lastActivityTime->add(new \DateInterval('PT1S'));
170
                    }
171
                }
172
173
                if ($lastActivityTime) {
174
                    $this->logger->info(sprintf('Incremental copy from %s started for module %s', $lastActivityTime->format(\DateTime::ATOM), $dao->getPluralModuleName()));
175
                } else {
176
                    $this->logger->info(sprintf('Incremental copy started for module %s', $dao->getPluralModuleName()));
177
                }
178
179
                $this->logger->notice(sprintf('Fetching the records to insert/update for module %s...', $dao->getPluralModuleName()));
180
                $records = $dao->getRecords(null, null, null, $lastActivityTime);
181
                $totalRecords = count($records);
182
                $this->logger->debug($totalRecords . ' records fetched.');
183
                $this->logger->notice(sprintf('Fetching the records to delete for module %s...', $dao->getPluralModuleName()));
184
                $deletedRecords = $dao->getDeletedRecordIds($lastActivityTime);
185
                $totalRecordsDeleted = count($deletedRecords);
186
                $this->logger->debug($totalRecordsDeleted . ' records fetched.');
187
            } else {
188
                $this->logger->info(sprintf('Full copy started for module %s', $dao->getPluralModuleName()));
189
                $this->logger->notice(sprintf('Fetching the records to insert/update for module ...%s', $dao->getPluralModuleName()));
190
                $records = $dao->getRecords();
191
                $totalRecords = count($records);
192
                $this->logger->debug($totalRecords . ' records fetched.');
193
                $deletedRecords = [];
194
            }
195
        } catch (ZCRMException $exception) {
196
            $this->logger->error('Error when getting records for module ' . $dao->getPluralModuleName() . ': ' . $exception->getMessage(), [
197
                'exception' => $exception
198
            ]);
199
            if ($throwErrors) {
200
                throw $exception;
201
            }
202
            return;
203
        }
204
        $this->logger->info(sprintf('Inserting/updating %s records into table %s...', $totalRecords, $tableName));
205
206
        $table = $this->connection->getSchemaManager()->createSchema()->getTable($tableName);
207
208
        $select = $this->connection->prepare('SELECT * FROM ' . $tableName . ' WHERE id = :id');
209
210
        $this->connection->beginTransaction();
211
212
        $recordsModificationCounts = [
213
            'insert' => 0,
214
            'update' => 0,
215
            'delete' => 0,
216
        ];
217
218
        $logOffset = $totalRecords >= 500 ? 100 : 50;
219
        $processedRecords = 0;
220
        foreach ($records as $record) {
221 View Code Duplication
            if (($processedRecords % $logOffset) === 0) {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
222
                $this->logger->info(sprintf('%d/%s records processed for module %s', $processedRecords, $totalRecords, $dao->getPluralModuleName()));
223
            }
224
            ++$processedRecords;
225
            $data = [];
226
            $types = [];
227
            foreach ($table->getColumns() as $column) {
228
                if (in_array($column->getName(), ['id', 'uid'])) {
229
                    continue;
230
                }
231
                $field = $dao->getFieldFromFieldName($column->getName());
0 ignored issues
show
Bug introduced by
Are you sure the assignment to $field is correct as $dao->getFieldFromFieldName($column->getName()) (which targets Wabel\Zoho\CRM\AbstractZ...getFieldFromFieldName()) seems to always return null.

This check looks for function or method calls that always return null and whose return value is assigned to a variable.

class A
{
    function getObject()
    {
        return null;
    }

}

$a = new A();
$object = $a->getObject();

The method getObject() can return nothing but null, so it makes no sense to assign that value to a variable.

The reason is most likely that a function or method is imcomplete or has been reduced for debug purposes.

Loading history...
232
                if (!$field) {
233
                    continue;
234
                }
235
                $getterName = $field->getGetter();
236
                $dataValue = $record->$getterName();
237
                $finalFieldData = null;
0 ignored issues
show
Unused Code introduced by
$finalFieldData is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
238
                if ($dataValue instanceof ZCRMRecord) {
239
                    $finalFieldData = $dataValue->getEntityId();
240
                } elseif (is_array($dataValue)) {
241
                    $finalFieldData = implode(';', $dataValue);
242
                } else {
243
                    $finalFieldData = $dataValue;
244
                }
245
                $data[$column->getName()] = $finalFieldData;
246
                $types[$column->getName()] = $column->getType()->getName();
247
            }
248
249
            $select->execute(['id' => $record->getZohoId()]);
250
            $result = $select->fetch(\PDO::FETCH_ASSOC);
251
            if ($result === false) {
252
                $this->logger->debug(sprintf('Inserting record with ID \'%s\' in table %s...', $record->getZohoId(), $tableName));
253
254
                $data['id'] = $record->getZohoId();
255
                $types['id'] = 'string';
256
257
                $recordsModificationCounts['insert'] += $this->connection->insert($tableName, $data, $types);
258
259
                foreach ($this->listeners as $listener) {
260
                    $listener->onInsert($data, $dao);
261
                }
262
            } else {
263
                $this->logger->debug(sprintf('Updating record with ID \'%s\' in table %s...', $record->getZohoId(), $tableName));
264
                $identifier = ['id' => $record->getZohoId()];
265
                $types['id'] = 'string';
266
267
                $recordsModificationCounts['update'] += $this->connection->update($tableName, $data, $identifier, $types);
268
269
                // Let's add the id for the update trigger
270
                $data['id'] = $record->getZohoId();
271
                foreach ($this->listeners as $listener) {
272
                    $listener->onUpdate($data, $result, $dao);
273
                }
274
            }
275
        }
276
277
        $this->logger->info(sprintf('Deleting %d records from table %s...', $totalRecordsDeleted, $tableName));
278
        $sqlStatementUid = 'select uid from ' . $this->connection->quoteIdentifier($tableName) . ' where id = :id';
279
        $processedRecords = 0;
280
        $logOffset = $totalRecordsDeleted >= 500 ? 100 : 50;
281
        foreach ($deletedRecords as $deletedRecord) {
282 View Code Duplication
            if (($processedRecords % $logOffset) === 0) {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
283
                $this->logger->info(sprintf('%d/%d records processed for module %s', $processedRecords, $totalRecordsDeleted, $dao->getPluralModuleName()));
284
            }
285
            ++$processedRecords;
286
            $this->logger->debug(sprintf('Deleting record with ID \'%s\' in table %s...', $deletedRecord->getEntityId(), $tableName));
287
            $uid = $this->connection->fetchColumn($sqlStatementUid, ['id' => $deletedRecord->getEntityId()]);
288
            $recordsModificationCounts['delete'] += $this->connection->delete($tableName, ['id' => $deletedRecord->getEntityId()]);
289
            if ($twoWaysSync) {
290
                // TODO: we could detect if there are changes to be updated to the server and try to warn with a log message
291
                // Also, let's remove the newly created field (because of the trigger) to avoid looping back to Zoho
292
                $this->connection->delete('local_delete', ['table_name' => $tableName, 'id' => $deletedRecord->getEntityId()]);
293
                $this->connection->delete('local_update', ['table_name' => $tableName, 'uid' => $uid]);
294
            }
295
        }
296
297
        $this->logger->notice(sprintf('Copy finished with %d item(s) inserted, %d item(s) updated and %d item(s) deleted.',
298
            $recordsModificationCounts['insert'],
299
            $recordsModificationCounts['update'],
300
            $recordsModificationCounts['delete']
301
        ));
302
303
        $this->connection->commit();
304
    }
305
306
    public function fetchFromZohoInBulk(AbstractZohoDao $dao)
307
    {
308
        /*
309
         * This method is really dirty, and do not use the php sdk because late development for the zoho v1 EOL in december.
310
         * Should be re-written to make it clean.
311
         */
312
        // Doc: https://www.zoho.com/crm/developer/docs/api/bulk-read/create-job.html
313
314
        $tableName = ZohoDatabaseHelper::getTableName($dao, $this->prefix);
315
        $table = $this->connection->getSchemaManager()->createSchema()->getTable($tableName);
316
        $apiModuleName = $dao->getPluralModuleName();
317
318
        $this->logger->notice('Starting bulk fetch for module ' . $apiModuleName . '...');
319
320
        $zohoClient = new ZohoClient([
321
            'client_id' => ZOHO_CRM_CLIENT_ID,
322
            'client_secret' => ZOHO_CRM_CLIENT_SECRET,
323
            'redirect_uri' => ZOHO_CRM_CLIENT_REDIRECT_URI,
324
            'currentUserEmail' => ZOHO_CRM_CLIENT_CURRENT_USER_EMAIL,
325
            'applicationLogFilePath' => ZOHO_CRM_CLIENT_APPLICATION_LOGFILEPATH,
326
            'persistence_handler_class' => ZOHO_CRM_CLIENT_PERSISTENCE_HANDLER_CLASS,
327
            'token_persistence_path' => ZOHO_CRM_CLIENT_PERSITENCE_PATH,
328
            'sandbox' => ZOHO_CRM_SANDBOX
329
        ], 'Europe/Paris');
330
331
        $oauthToken = $zohoClient->getZohoOAuthClient()->getAccessToken(ZOHO_CRM_CLIENT_CURRENT_USER_EMAIL);
332
333
        $client = new \GuzzleHttp\Client();
334
        $page = 1;
335
        while (true) {
336
            // Step 1: Create a bulk read job
337
            $this->logger->info('Creating read job for module ' . $apiModuleName . ' and page ' . $page . '...');
338
            $response = $client->request('POST', 'https://' . (ZOHO_CRM_SANDBOX === 'true' ? 'sandbox' : 'www') . '.zohoapis.com/crm/bulk/v2/read', [
339
                'http_errors' => false,
340
                'headers' => [
341
                    'Authorization' => 'Zoho-oauthtoken ' . $oauthToken
342
                ],
343
                'json' => [
344
                    'query' => [
345
                        'module' => $apiModuleName,
346
                        'page' => $page
347
                    ]
348
                ]
349
            ]);
350
            $jobId = null;
0 ignored issues
show
Unused Code introduced by
$jobId is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
351
            if ($response->getStatusCode() >= 200 && $response->getStatusCode() < 300) {
352
                $resultStr = $response->getBody()->getContents();
353
                $json = json_decode($resultStr, true);
354
355
                $jobId = $json['data'][0]['details']['id'];
356
357
                // We don't care about the job status right now, it will be checked later
358 View Code Duplication
            } else {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
359
                $this->logger->error('Cannot create bulk read query for module ' . $apiModuleName . ': status: ' . $response->getStatusCode() . '. Status: ' . $response->getBody()->getContents());
360
                break;
361
            }
362
363
            if ($jobId === null) {
364
                $this->logger->error('JobID cannot be null. json:' . $resultStr);
365
                break;
366
            }
367
368
            // Step 2: Check job status
369
            $jobDetails = null;
370
            while (true) {
371
                $this->logger->info('Checking job ' . $jobId . ' status for module ' . $apiModuleName . ' and page ' . $page . '...');
372
                $response = $client->request('GET', 'https://' . (ZOHO_CRM_SANDBOX === 'true' ? 'sandbox' : 'www') . '.zohoapis.com/crm/bulk/v2/read/' . $jobId, [
373
                    'http_errors' => false,
374
                    'headers' => [
375
                        'Authorization' => 'Zoho-oauthtoken ' . $oauthToken
376
                    ]
377
                ]);
378
                if ($response->getStatusCode() >= 200 && $response->getStatusCode() < 300) {
379
                    $resultStr = $response->getBody()->getContents();
380
                    $json = json_decode($resultStr, true);
381
382
                    if (isset($json['data'][0]['state'])) {
383
                        $status = $json['data'][0]['state'];
384
                        if ($status === 'ADDED' || $status === 'QUEUED') {
385
                            $this->logger->info('Job still waiting for process');
386
                        } else if ($status === 'IN PROGRESS') {
387
                            $this->logger->info('Job in progress');
388
                        } else if ($status === 'COMPLETED') {
389
                            $this->logger->info('Job completed');
390
                            $jobDetails = $json;
391
                            break;
392
                        } else {
393
                            $this->logger->info('Unsupported job status: ' . $resultStr);
394
                            break;
395
                        }
396
                    } else {
397
                        $this->logger->error('Unsupported response: ' . $resultStr);
398
                        break;
399
                    }
400 View Code Duplication
                } else {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
401
                    $this->logger->error('Cannot get bulk job status query for module ' . $apiModuleName . ': status: ' . $response->getStatusCode() . '. Status: ' . $response->getBody()->getContents());
402
                    break;
403
                }
404
                sleep(15);
405
            }
406
407
            // Step 3: Download the result
408
            if ($jobDetails === null) {
409
                $this->logger->error('JobDetails cannot be empty. json:' . $resultStr);
410
                break;
411
            }
412
            $this->logger->debug(json_encode($jobDetails));
413
            $this->logger->info('Downloading zip file for module ' . $apiModuleName . ' and page ' . $page . '...');
414
            $jobZipFile = '/tmp/job_' . $dao->getZCRMModule()->getAPIName() . '_' . $jobDetails['data'][0]['id'] . '.zip';
415
            $jobCsvPath = '/tmp/job_extract';
416
            $jobCsvFile = '/tmp/job_extract/' . $jobDetails['data'][0]['id'] . '.csv';
417
            $canProcessCsv = false;
0 ignored issues
show
Unused Code introduced by
$canProcessCsv is not used, you could remove the assignment.

This check looks for variable assignements that are either overwritten by other assignments or where the variable is not used subsequently.

$myVar = 'Value';
$higher = false;

if (rand(1, 6) > 3) {
    $higher = true;
} else {
    $higher = false;
}

Both the $myVar assignment in line 1 and the $higher assignment in line 2 are dead. The first because $myVar is never used and the second because $higher is always overwritten for every possible time line.

Loading history...
418
419
            $response = $client->request('GET', 'https://' . (ZOHO_CRM_SANDBOX === 'true' ? 'sandbox' : 'www') . '.zohoapis.com/crm/bulk/v2/read/' . $jobId . '/result', [
420
                'http_errors' => false,
421
                'headers' => [
422
                    'Authorization' => 'Zoho-oauthtoken ' . $oauthToken
423
                ],
424
                'sink' => $jobZipFile
425
            ]);
426
            if ($response->getStatusCode() >= 200 && $response->getStatusCode() < 300) {
427
                $this->logger->info('Extracting ' . $jobZipFile . ' file for module ' . $apiModuleName . ' and page ' . $page . '...');
428
                $zip = new ZipArchive();
429
                $res = $zip->open($jobZipFile);
430
                if ($res === TRUE) {
431
                    $zip->extractTo($jobCsvPath);
432
                    $zip->close();
433
                    $this->logger->info('File extracted in ' . $jobCsvFile);
434
                    $canProcessCsv = true;
435
                } else {
436
                    switch ($res) {
437
                        case ZipArchive::ER_EXISTS:
438
                            $zipErrorMessage = 'File already exists.';
439
                            break;
440
                        case ZipArchive::ER_INCONS:
441
                            $zipErrorMessage = 'Zip archive inconsistent.';
442
                            break;
443
                        case ZipArchive::ER_MEMORY:
444
                            $zipErrorMessage = 'Malloc failure.';
445
                            break;
446
                        case ZipArchive::ER_NOENT:
447
                            $zipErrorMessage = 'No such file.';
448
                            break;
449
                        case ZipArchive::ER_NOZIP:
450
                            $zipErrorMessage = 'Not a zip archive.';
451
                            break;
452
                        case ZipArchive::ER_OPEN:
453
                            $zipErrorMessage = "Can't open file.";
454
                            break;
455
                        case ZipArchive::ER_READ:
456
                            $zipErrorMessage = 'Read error.';
457
                            break;
458
                        case ZipArchive::ER_SEEK:
459
                            $zipErrorMessage = 'Seek error.';
460
                            break;
461
                        default:
462
                            $zipErrorMessage = "Unknow (Code $res)";
463
                            break;
464
                    }
465
                    $this->logger->error('Error when extracting zip file: ' . $zipErrorMessage);
466
                    break;
467
                }
468 View Code Duplication
            } else {
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated across your project.

Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation.

You can also find more detailed suggestions in the “Code” section of your repository.

Loading history...
469
                $this->logger->error('Cannot download results for module ' . $apiModuleName . ': status: ' . $response->getStatusCode() . '. Status: ' . $response->getBody()->getContents());
470
                break;
471
            }
472
473
            // Step 4: Save data
474
            if (!$canProcessCsv) {
475
                $this->logger->error('Cannot process CSV');
476
                break;
477
            }
478
479
            $this->logger->info('Building list of users...');
480
            $usersQuery = $this->connection->executeQuery('SELECT id, full_name FROM users');
481
            $usersResults = $usersQuery->fetchAll();
482
            $users = [];
483
            foreach ($usersResults as $user) {
484
                $users[$user['id']] = $user['full_name'];
485
            }
486
487
            $this->logger->info('Saving records to db...');
488
            $nbRecords = $jobDetails['data'][0]['result']['count'];
489
            $whenToLog = ceil($nbRecords / 100);
490
            $this->logger->info($nbRecords . ' records to save');
491
            $nbSaved = 0;
492
            $handle = fopen($jobCsvFile, 'r');
493
            $fields = [];
494
            if ($handle) {
495
                while (($row = fgetcsv($handle)) !== false) {
496
                    if (empty($fields)) {
497
                        $fields = $row;
498
                        continue;
499
                    }
500
                    $recordDataToInsert = [];
501
                    foreach ($row as $k => $value) {
502
                        $columnName = $fields[$k];
503
                        $decodedColumnName = str_replace('_', '', $columnName);
504
                        if ($table->hasColumn($decodedColumnName)) {
505
                            $recordDataToInsert[$decodedColumnName] = $value === '' ? null : $value;
506
                        } else {
507
                            if ($columnName === 'Owner' || $columnName === 'Created_By' || $columnName === 'Modified_By') {
508
                                $recordDataToInsert[$decodedColumnName . '_OwnerID'] = $value === '' ? null : $value;
509
                                $recordDataToInsert[$decodedColumnName . '_OwnerName'] = $users[$value] ?? null;
510
                            } else if ($table->hasColumn($decodedColumnName . '_ID')) {
511
                                $recordDataToInsert[$decodedColumnName . '_ID'] = $value === '' ? null : $value;
512
                            }
513
                        }
514
                    }
515
                    $this->connection->insert($tableName, $recordDataToInsert);
516
                    ++$nbSaved;
517
                    if (($nbSaved % $whenToLog) === 0) {
518
                        $this->logger->info($nbSaved . '/' . $nbRecords . ' records processed');
519
                    }
520
                }
521
                $this->logger->info($nbSaved . ' records saved for module ' . $apiModuleName . ' and page ' . $page);
522
                fclose($handle);
523
            }
524
525
            // Step 5: Check if there is more results
526
            $hasMoreRecords = $jobDetails['data'][0]['result']['more_records'];
527
            if (!$hasMoreRecords) {
528
                $this->logger->info('No more records for the module ' . $apiModuleName);
529
                break;
530
            }
531
            $this->logger->info('More records to fetch for the module ' . $apiModuleName);
532
            ++$page;
533
        }
534
    }
535
}
536