Passed
Push — master ( 278193...cfb0f8 )
by Darko
06:43
created

NZBImport   D

Complexity

Total Complexity 58

Size/Duplication

Total Lines 413
Duplicated Lines 0 %

Importance

Changes 3
Bugs 0 Features 0
Metric Value
wmc 58
eloc 210
c 3
b 0
f 0
dl 0
loc 413
rs 4.5599

7 Methods

Rating   Name   Duplication   Size   Complexity  
F scanNZBFile() 0 110 21
A echoOut() 0 6 3
A getAllGroups() 0 18 4
A __construct() 0 10 2
A updateNzbGuid() 0 6 2
B insertNZB() 0 64 7
D beginImport() 0 114 19

How to fix   Complexity   

Complex Class

Complex classes like NZBImport often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

While breaking up the class, it is a good idea to analyze how other classes use NZBImport, and based on these observations, apply Extract Interface, too.

1
<?php
2
3
namespace Blacklight;
4
5
use App\Models\Release;
6
use App\Models\Settings;
7
use App\Models\UsenetGroup;
8
use Blacklight\utility\Utility;
9
use Illuminate\Support\Carbon;
10
use Illuminate\Support\Facades\File;
11
use Illuminate\Support\Str;
12
13
/**
14
 * Class NZBImport.
15
 */
16
class NZBImport
17
{
18
    protected Binaries $binaries;
19
20
    protected ReleaseCleaning $releaseCleaner;
21
22
    protected \stdClass|bool $site;
23
24
    /**
25
     * @var int
26
     */
27
    protected mixed $crossPostt;
28
29
    protected Categorize $category;
30
31
    /**
32
     * List of all the group names/ids in the DB.
33
     */
34
    protected array $allGroups;
35
36
    /**
37
     * Was this run from the browser?
38
     */
39
    protected bool $browser;
40
41
    /**
42
     * Return value for browser.
43
     */
44
    protected string $retVal;
45
46
    /**
47
     * Guid of the current releases.
48
     */
49
    protected string $relGuid;
50
51
    /**
52
     * @var bool
53
     */
54
    public mixed $echoCLI;
55
56
    public NZB $nzb;
57
58
    /**
59
     * @var string the MD5 hash of the first segment Message-ID of the NZB
60
     */
61
    protected string $nzbGuid;
62
63
    protected ColorCLI $colorCli;
64
65
    public function __construct()
66
    {
67
        $this->echoCLI = config('nntmux.echocli');
68
        $this->binaries = new Binaries();
69
        $this->category = new Categorize();
70
        $this->nzb = new NZB();
71
        $this->releaseCleaner = new ReleaseCleaning();
72
        $this->colorCli = new ColorCLI();
73
        $this->crossPostt = Settings::settingValue('..crossposttime') !== '' ? Settings::settingValue('..crossposttime') : 2;
0 ignored issues
show
introduced by
The condition App\Models\Settings::set....crossposttime') !== '' is always true.
Loading history...
Documentation Bug introduced by
It seems like App\Models\Settings::set...('..crossposttime') : 2 of type App\Models\Settings is incompatible with the declared type integer of property $crossPostt.

Our type inference engine has found an assignment to a property that is incompatible with the declared type of that property.

Either this assignment is in error or the assigned type should be added to the documentation/type hint for that property..

Loading history...
74
        $this->retVal = '';
75
    }
76
77
    /**
78
     * @throws \Illuminate\Contracts\Filesystem\FileNotFoundException
79
     */
80
    public function beginImport($filesToProcess, bool $useNzbName = false, bool $delete = false, bool $deleteFailed = false): bool|string
81
    {
82
        // Get all the groups in the DB.
83
        if (! $this->getAllGroups()) {
84
            if ($this->browser) {
85
                return $this->retVal;
86
            }
87
88
            return false;
89
        }
90
91
        $start = now()->toImmutable()->format('Y-m-d H:i:s');
92
        $nzbsImported = $nzbsSkipped = 0;
93
94
        // Loop over the file names.
95
        foreach ($filesToProcess as $nzbFile) {
96
            $this->nzbGuid = '';
97
98
            // Check if the file is really there.
99
            if (File::isFile($nzbFile)) {
100
                // Get the contents of the NZB file as a string.
101
                if (Str::endsWith($nzbFile, '.nzb.gz')) {
102
                    $nzbString = Utility::unzipGzipFile($nzbFile);
103
                } else {
104
                    $nzbString = File::get($nzbFile);
105
                }
106
107
                if ($nzbString === false) {
108
                    $this->echoOut('ERROR: Unable to read: '.$nzbFile);
109
110
                    if ($deleteFailed) {
111
                        File::delete($nzbFile);
112
                    }
113
                    $nzbsSkipped++;
114
115
                    continue;
116
                }
117
118
                // Load it as an XML object.
119
                $nzbXML = @simplexml_load_string($nzbString);
120
                if ($nzbXML === false || strtolower($nzbXML->getName()) !== 'nzb') {
121
                    $this->echoOut('ERROR: Unable to load NZB XML data: '.$nzbFile);
122
123
                    if ($deleteFailed) {
124
                        File::delete($nzbFile);
125
                    }
126
                    $nzbsSkipped++;
127
128
                    continue;
129
                }
130
131
                // Try to insert the NZB details into the DB.
132
                try {
133
                    $inserted = $this->scanNZBFile($nzbXML, ($useNzbName ? str_ireplace('.nzb', '', basename($nzbFile)) : false));
134
                } catch (\Exception $e) {
135
                    $this->echoOut('ERROR: Problem inserting: '.$nzbFile);
136
                    $inserted = false;
137
                }
138
139
                if ($inserted) {
140
                    // Try to copy the NZB to the NZB folder.
141
                    $path = $this->nzb->getNZBPath($this->relGuid, 0, true);
142
143
                    // Try to compress the NZB file in the NZB folder.
144
                    $fp = gzopen($path, 'w5');
145
                    gzwrite($fp, $nzbString);
146
                    gzclose($fp);
147
148
                    if (! File::isFile($path)) {
149
                        $this->echoOut('ERROR: Problem compressing NZB file to: '.$path);
150
151
                        // Remove the release.
152
                        Release::query()->where('guid', $this->relGuid)->delete();
153
154
                        if ($deleteFailed) {
155
                            File::delete($nzbFile);
156
                        }
157
                        $nzbsSkipped++;
158
                    } else {
159
                        $this->updateNzbGuid();
160
161
                        if ($delete) {
162
                            // Remove the nzb file.
163
                            File::delete($nzbFile);
164
                        }
165
166
                        $nzbsImported++;
167
                    }
168
                } else {
169
                    $this->echoOut('ERROR: Failed to insert NZB!');
170
                    if ($deleteFailed) {
171
                        File::delete($nzbFile);
172
                    }
173
                    $nzbsSkipped++;
174
                }
175
            } else {
176
                $this->echoOut('ERROR: Unable to fetch: '.$nzbFile);
177
                $nzbsSkipped++;
178
            }
179
        }
180
        $this->echoOut(
181
            'Proccessed '.
182
            $nzbsImported.
183
            ' NZBs in '.
184
            now()->diffForHumans($start).
185
            $nzbsSkipped.
186
            ' NZBs were skipped.'
187
        );
188
189
        if ($this->browser) {
190
            return $this->retVal;
191
        }
192
193
        return true;
194
    }
195
196
    /**
197
     * @throws \Exception
198
     */
199
    protected function scanNZBFile(&$nzbXML, bool $useNzbName = false): bool
200
    {
201
        $binary_names = [];
202
        $totalFiles = $totalSize = $groupID = 0;
203
        $isBlackListed = $groupName = $firstName = $posterName = $postDate = false;
204
205
        // Go through the NZB, get the details, look if it's blacklisted, look if we have the groups.
206
        foreach ($nzbXML->file as $file) {
207
            $binary_names[] = $file['subject'];
208
            $totalFiles++;
209
            $groupID = -1;
210
211
            // Get the nzb info.
212
            if ($firstName === false) {
213
                $firstName = (string) $file->attributes()->subject;
214
            }
215
            if ($posterName === false) {
216
                $posterName = (string) $file->attributes()->poster;
217
            }
218
            if ($postDate === false) {
219
                $postDate = Carbon::createFromTimestamp((string) $file->attributes()->date)->format('Y-m-d H:i:s');
220
            }
221
222
            // Make a fake message array to use to check the blacklist.
223
            $msg = ['Subject' => (string) $file->attributes()->subject, 'From' => (string) $file->attributes()->poster, 'Message-ID' => ''];
224
225
            // Get the group names, group_id, check if it's blacklisted.
226
            $groupArr = [];
227
            foreach ($file->groups->group as $group) {
228
                $group = (string) $group;
229
230
                // If group_id is -1 try to get a group_id.
231
                if ($groupID === -1) {
232
                    if (array_key_exists($group, $this->allGroups)) {
233
                        $groupID = $this->allGroups[$group];
234
                        if (! $groupName) {
235
                            $groupName = $group;
236
                        }
237
                    } else {
238
                        $group = UsenetGroup::isValidGroup($group);
239
                        if ($group !== false) {
240
                            $groupID = UsenetGroup::addGroup([
241
                                'name' => $group,
242
                                'description' => 'Added by NZBimport script.',
243
                                'backfill_target' => 1,
244
                                'minfilestoformrelease' => '',
245
                                'minsizetoformrelease' => '',
246
                                'first_record' => 0,
247
                                'last_record' => 0,
248
                                'active' => 0,
249
                                'backfill' => 0,
250
                            ]);
251
                            $this->allGroups[$group] = $groupID;
252
253
                            $this->echoOut("Adding missing group: ($group)");
254
                        }
255
                    }
256
                }
257
                // Add all the found groups to an array.
258
                $groupArr[] = $group;
259
260
                // Check if this NZB is blacklisted.
261
                if ($this->binaries->isBlackListed($msg, $group)) {
262
                    $isBlackListed = true;
263
                    break;
264
                }
265
            }
266
267
            // If we found a group and it's not blacklisted.
268
            if ($groupID !== -1 && ! $isBlackListed) {
269
                // Get the size of the release.
270
                if (\count($file->segments->segment) > 0) {
271
                    foreach ($file->segments->segment as $segment) {
272
                        $totalSize += (int) $segment->attributes()->bytes;
273
                    }
274
                }
275
            } else {
276
                if ($isBlackListed) {
277
                    $errorMessage = 'Subject is blacklisted: '.mb_convert_encoding(trim($firstName), 'UTF-8', mb_list_encodings());
0 ignored issues
show
Bug introduced by
Are you sure mb_convert_encoding(trim...', mb_list_encodings()) of type array|string can be used in concatenation? ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-type  annotation

277
                    $errorMessage = 'Subject is blacklisted: './** @scrutinizer ignore-type */ mb_convert_encoding(trim($firstName), 'UTF-8', mb_list_encodings());
Loading history...
Bug introduced by
It seems like $firstName can also be of type false; however, parameter $string of trim() does only seem to accept string, maybe add an additional type check? ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-type  annotation

277
                    $errorMessage = 'Subject is blacklisted: '.mb_convert_encoding(trim(/** @scrutinizer ignore-type */ $firstName), 'UTF-8', mb_list_encodings());
Loading history...
278
                } else {
279
                    $errorMessage = 'No group found for '.$firstName.' (one of '.implode(', ', $groupArr).' are missing';
0 ignored issues
show
Bug introduced by
Are you sure $firstName of type false|string can be used in concatenation? ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-type  annotation

279
                    $errorMessage = 'No group found for './** @scrutinizer ignore-type */ $firstName.' (one of '.implode(', ', $groupArr).' are missing';
Loading history...
280
                }
281
                $this->echoOut($errorMessage);
282
283
                return false;
284
            }
285
        }
286
287
        // Sort values alphabetically but keep the keys intact
288
        if (\count($binary_names) > 0) {
289
            asort($binary_names);
290
            foreach ($nzbXML->file as $file) {
291
                if ($file['subject'] === $binary_names[0]) {
292
                    $this->nzbGuid = md5($file->segments->segment);
293
                    break;
294
                }
295
            }
296
        }
297
298
        // Try to insert the NZB details into the DB.
299
        return $this->insertNZB(
300
            [
301
                'subject' => $firstName,
302
                'useFName' => $useNzbName,
303
                'postDate' => empty($postDate) ? now()->format('Y-m-d H:i:s') : $postDate,
304
                'from' => empty($posterName) ? '' : $posterName,
305
                'groups_id' => $groupID,
306
                'groupName' => $groupName,
307
                'totalFiles' => $totalFiles,
308
                'totalSize' => $totalSize,
309
            ]
310
        );
311
    }
312
313
    /**
314
     * Insert the NZB details into the database.
315
     *
316
     *
317
     * @throws \Exception
318
     */
319
    protected function insertNZB($nzbDetails): bool
320
    {
321
        // Make up a GUID for the release.
322
        $this->relGuid = createGUID();
323
324
        // Remove part count from subject.
325
        $partLess = preg_replace('/(\(\d+\/\d+\))*$/', 'yEnc', $nzbDetails['subject']);
326
        // Remove added yEnc from above and anything after.
327
        $subject = mb_convert_encoding(trim(preg_replace('/yEnc.*$/i', 'yEnc', $partLess)), 'UTF-8', mb_list_encodings());
328
329
        $renamed = 0;
330
        if ($nzbDetails['useFName']) {
331
            // If the user wants to use the file name.. use it.
332
            $cleanName = $nzbDetails['useFName'];
333
            $renamed = 1;
334
        } else {
335
            // Pass the subject through release cleaner to get a nicer name.
336
            $cleanName = $this->releaseCleaner->releaseCleaner($subject, $nzbDetails['from'], $nzbDetails['groupName']);
337
            if (isset($cleanName['properlynamed'])) {
338
                $cleanName = $cleanName['cleansubject'];
339
                $renamed = (isset($cleanName['properlynamed']) && $cleanName['properlynamed'] === true ? 1 : 0);
340
            }
341
        }
342
343
        $escapedSubject = $subject;
344
        $escapedFromName = $nzbDetails['from'];
345
346
        // Look for a duplicate on name, poster and size.
347
        $dupeCheck = Release::query()->where(['name' => $escapedSubject, 'fromname' => $escapedFromName])->whereBetween('size', [$nzbDetails['totalSize'] * 0.99, $nzbDetails['totalSize'] * 1.01])->first(['id']);
348
349
        if ($dupeCheck === null) {
350
            $escapedSearchName = $cleanName;
351
            $determinedCategory = $this->category->determineCategory($nzbDetails['groups_id'], $cleanName, $escapedFromName);
352
            // Insert the release into the DB.
353
            $relID = Release::insertRelease(
354
                [
355
                    'name' => $escapedSubject,
356
                    'searchname' => $escapedSearchName ?? $escapedSubject,
357
                    'totalpart' => $nzbDetails['totalFiles'],
358
                    'groups_id' => $nzbDetails['groups_id'],
359
                    'guid' => $this->relGuid,
360
                    'postdate' => $nzbDetails['postDate'],
361
                    'fromname' => $escapedFromName,
362
                    'size' => $nzbDetails['totalSize'],
363
                    'categories_id' => $determinedCategory['categories_id'],
364
                    'isrenamed' => $renamed,
365
                    'reqidstatus' => 0,
366
                    'predb_id' => 0,
367
                    'nzbstatus' => NZB::NZB_ADDED,
368
                ]
369
            );
370
        } else {
371
            $this->echoOut('This release is already in our DB so skipping: '.$subject);
0 ignored issues
show
Bug introduced by
Are you sure $subject of type array|string can be used in concatenation? ( Ignorable by Annotation )

If this is a false-positive, you can also ignore this issue in your code via the ignore-type  annotation

371
            $this->echoOut('This release is already in our DB so skipping: './** @scrutinizer ignore-type */ $subject);
Loading history...
372
373
            return false;
374
        }
375
376
        if ($relID === null) {
0 ignored issues
show
introduced by
The condition $relID === null is always false.
Loading history...
377
            $this->echoOut('ERROR: Problem inserting: '.$subject);
378
379
            return false;
380
        }
381
382
        return true;
383
    }
384
385
    /**
386
     * Get all groups in the DB.
387
     */
388
    protected function getAllGroups(): bool
389
    {
390
        $this->allGroups = [];
391
        $groups = UsenetGroup::query()->get(['id', 'name']);
392
393
        if ($groups instanceof \Traversable) {
0 ignored issues
show
introduced by
$groups is always a sub-type of Traversable.
Loading history...
394
            foreach ($groups as $group) {
395
                $this->allGroups[$group['name']] = $group['id'];
396
            }
397
        }
398
399
        if (\count($this->allGroups) === 0) {
400
            $this->echoOut('You have no groups in your database!');
401
402
            return false;
403
        }
404
405
        return true;
406
    }
407
408
    /**
409
     * Echo message to browser or CLI.
410
     */
411
    protected function echoOut(string $message): void
412
    {
413
        if ($this->browser) {
414
            $this->retVal .= $message.'<br />';
415
        } elseif ($this->echoCLI) {
416
            $this->colorCli->notice($message);
417
        }
418
    }
419
420
    /**
421
     * The function updates the NZB guid after there is no chance of deletion.
422
     */
423
    protected function updateNzbGuid(): void
424
    {
425
        try {
426
            Release::query()->where('guid', $this->relGuid)->update(['nzb_guid' => sodium_hex2bin($this->nzbGuid)]);
427
        } catch (\SodiumException $e) {
428
            $this->echoOut('ERROR: Problem updating nzb_guid for: '.$this->relGuid);
429
        }
430
    }
431
}
432