Completed
Push — dev ( 852949...ba468b )
by Darko
07:13
created

populate_indexes()   C

Complexity

Conditions 13
Paths 28

Size

Total Lines 109
Code Lines 65

Duplication

Lines 0
Ratio 0 %

Importance

Changes 1
Bugs 0 Features 0
Metric Value
eloc 65
c 1
b 0
f 0
dl 0
loc 109
rs 6.0569
cc 13
nc 28
nop 2

How to fix   Long Method    Complexity   

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

1
<?php
2
3
require_once dirname(__DIR__, 2).DIRECTORY_SEPARATOR.'bootstrap/autoload.php';
4
5
use App\Models\Predb;
6
use App\Models\Release;
7
use Illuminate\Support\Facades\DB;
8
9
if (! isset($argv[1])) {
10
    exit(
11
            "Argument 1 is the index name, releases and predb are the only supported ones currently.\n".
12
            "Argument 2 is optional, max number of rows to send to ES at a time, 10,000 is the default if not set.\n"
13
    );
14
}
15
16
populate_indexes($argv[1], (isset($argv[2]) && is_numeric($argv[2]) && $argv[2] > 0 ? $argv[2] : 10000));
17
18
// Bulk insert releases into sphinx RT index.
19
function populate_indexes($table, $max)
20
{
21
    $allowedIndexes = ['releases', 'predb'];
22
    if (\in_array($table, $allowedIndexes, true)) {
23
        if ($table === 'releases') {
24
            DB::statement('SET SESSION group_concat_max_len=16384;');
25
            $query = 'SELECT r.id, r.name, r.searchname, r.fromname, IFNULL(GROUP_CONCAT(rf.name SEPARATOR " "),"") filename
26
				FROM releases r
27
				LEFT JOIN release_files rf ON r.id = rf.releases_id
28
				WHERE r.id > %d
29
				GROUP BY r.id
30
				ORDER BY r.id ASC
31
				LIMIT %d';
32
33
            $totals = Release::fromQuery('SELECT COUNT(id) AS c, MIN(id) AS min FROM releases')->first();
34
            if (! $totals) {
35
                exit("Could not get database information for releases table.\n");
0 ignored issues
show
Best Practice introduced by
Using exit here is not recommended.

In general, usage of exit should be done with care and only when running in a scripting context like a CLI script.

Loading history...
36
            }
37
            $total = $totals->c;
0 ignored issues
show
Bug introduced by
The property c does not seem to exist on App\Models\Release. Are you sure there is no database migration missing?

Checks if undeclared accessed properties appear in database migrations and if the creating migration is correct.

Loading history...
38
            $minId = $totals->min;
0 ignored issues
show
Bug introduced by
The property min does not seem to exist on App\Models\Release. Are you sure there is no database migration missing?

Checks if undeclared accessed properties appear in database migrations and if the creating migration is correct.

Loading history...
39
        }
40
41
        if ($table === 'predb') {
42
            DB::statement('SET SESSION group_concat_max_len=16384;');
43
            $query = 'SELECT id, title, filename, source
44
				FROM predb
45
				WHERE id > %d
46
				GROUP BY id
47
				ORDER BY id ASC
48
				LIMIT %d';
49
50
            $totals = Predb::fromQuery('SELECT COUNT(id) AS c, MIN(id) AS min FROM predb')->first();
51
            if (! $totals) {
52
                exit("Could not get database information for predb table.\n");
0 ignored issues
show
Best Practice introduced by
Using exit here is not recommended.

In general, usage of exit should be done with care and only when running in a scripting context like a CLI script.

Loading history...
53
            }
54
            $total = $totals->c;
0 ignored issues
show
Bug introduced by
The property c does not seem to exist on App\Models\Predb. Are you sure there is no database migration missing?

Checks if undeclared accessed properties appear in database migrations and if the creating migration is correct.

Loading history...
55
            $minId = $totals->min;
0 ignored issues
show
Bug introduced by
The property min does not seem to exist on App\Models\Predb. Are you sure there is no database migration missing?

Checks if undeclared accessed properties appear in database migrations and if the creating migration is correct.

Loading history...
56
        }
57
58
        $lastId = $minId - 1;
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $minId does not seem to be defined for all execution paths leading up to this point.
Loading history...
59
        echo "[Starting to populate ElasticSearch index $table with $total releases.]".PHP_EOL;
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $total does not seem to be defined for all execution paths leading up to this point.
Loading history...
60
        $data = ['body' => []];
61
        for ($i = $minId; $i <= ($total + $max + $minId); $i += $max) {
62
            $rows = DB::select(sprintf($query, $lastId, $max));
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable $query does not seem to be defined for all execution paths leading up to this point.
Loading history...
63
            if ($rows === 0) {
64
                continue;
65
            }
66
67
            foreach ($rows as $row) {
68
                if ($row->id > $lastId) {
69
                    $lastId = $row->id;
70
                }
71
                switch ($table) {
72
                    case 'releases':
73
                        $data['body'][] = [
74
                            'index' => [
75
                                '_index' => 'releases',
76
                                '_type' => 'releases',
77
                                '_id' => $row->id
78
                            ]
79
                        ];
80
81
                        $data['body'][] = [
82
                            'id' => $row->id,
83
                            'name' => $row->name,
84
                            'searchname' => $row->searchname,
85
                            'fromname' => $row->fromname,
86
                            'filename' => $row->filename,
87
                        ];
88
                        break;
89
90
                    case 'predb':
91
                        $data['body'][] = [
92
                            'index' => [
93
                                '_index' => 'predb',
94
                                '_type' => 'predb',
95
                                '_id' => $row->id
96
                            ]
97
                        ];
98
                        $data['body'][] = [
99
                                'id' => $row->id,
100
                                'title' => $row->title,
101
                                'filename' => $row->filename,
102
                                'source' => $row->source,
103
                            ];
104
                        break;
105
                }
106
            }
107
            // Stop and send the bulk request
108
            $responses = Elasticsearch::bulk($data);
109
110
            // erase the old bulk request
111
            $data = ['body' => []];
112
113
            // unset the bulk response when you are done to save memory
114
            unset($responses);
115
116
            echo '.';
117
        }
118
119
        // Send the last batch if it exists
120
        if (! empty($data['body'])) {
121
            $responses = Elasticsearch::bulk($data);
0 ignored issues
show
Unused Code introduced by
The assignment to $responses is dead and can be removed.
Loading history...
122
        }
123
        echo "\n[Done]\n";
124
    } else {
125
        exit(
0 ignored issues
show
Best Practice introduced by
Using exit here is not recommended.

In general, usage of exit should be done with care and only when running in a scripting context like a CLI script.

Loading history...
126
            "Argument 1 is the index name, releases and predb are the only supported ones currently.\n".
127
            "Argument 2 is optional, max number of rows to send to ES at a time, 10,000 is the default if not set.\n"
128
        );
129
    }
130
}
131