GedcomImportService::importTNGPlac()   A
last analyzed

Complexity

Conditions 6
Paths 5

Size

Total Lines 28
Code Lines 19

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
cc 6
eloc 19
nc 5
nop 1
dl 0
loc 28
rs 9.0111
c 0
b 0
f 0
1
<?php
2
3
/**
4
 * webtrees: online genealogy
5
 * Copyright (C) 2025 webtrees development team
6
 * This program is free software: you can redistribute it and/or modify
7
 * it under the terms of the GNU General Public License as published by
8
 * the Free Software Foundation, either version 3 of the License, or
9
 * (at your option) any later version.
10
 * This program is distributed in the hope that it will be useful,
11
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
12
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13
 * GNU General Public License for more details.
14
 * You should have received a copy of the GNU General Public License
15
 * along with this program. If not, see <https://www.gnu.org/licenses/>.
16
 */
17
18
declare(strict_types=1);
19
20
namespace Fisharebest\Webtrees\Services;
21
22
use Fisharebest\Webtrees\Date;
23
use Fisharebest\Webtrees\DB;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\DB was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
24
use Fisharebest\Webtrees\Exceptions\GedcomErrorException;
25
use Fisharebest\Webtrees\Family;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Family was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
26
use Fisharebest\Webtrees\Gedcom;
27
use Fisharebest\Webtrees\Header;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Header was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
28
use Fisharebest\Webtrees\Individual;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Individual was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
29
use Fisharebest\Webtrees\Location;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Location was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
30
use Fisharebest\Webtrees\Media;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Media was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
31
use Fisharebest\Webtrees\Note;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Note was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
32
use Fisharebest\Webtrees\Place;
33
use Fisharebest\Webtrees\PlaceLocation;
34
use Fisharebest\Webtrees\Registry;
35
use Fisharebest\Webtrees\Repository;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Repository was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
36
use Fisharebest\Webtrees\Soundex;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Soundex was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
37
use Fisharebest\Webtrees\Source;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Source was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
38
use Fisharebest\Webtrees\Submission;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Submission was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
39
use Fisharebest\Webtrees\Submitter;
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Submitter was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
40
use Fisharebest\Webtrees\Tree;
41
use Illuminate\Database\Query\JoinClause;
42
43
use function array_chunk;
44
use function array_intersect_key;
45
use function array_map;
46
use function array_unique;
47
use function array_values;
48
use function date;
49
use function explode;
50
use function max;
51
use function mb_substr;
52
use function preg_match;
53
use function preg_match_all;
54
use function preg_replace;
55
use function round;
56
use function str_contains;
57
use function str_replace;
58
use function str_starts_with;
59
use function strlen;
60
use function strtoupper;
61
use function strtr;
62
use function substr;
63
use function trim;
64
65
use const PREG_SET_ORDER;
66
67
/**
68
 * Class GedcomImportService - import GEDCOM data
69
 */
70
class GedcomImportService
71
{
72
    /**
73
     * Tidy up a gedcom record on import, so that we can access it consistently/efficiently.
74
     */
75
    private function reformatRecord(string $rec, Tree $tree): string
76
    {
77
        $gedcom_service = Registry::container()->get(GedcomService::class);
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Services\GedcomService was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
78
79
        // Strip out mac/msdos line endings
80
        $rec = preg_replace("/[\r\n]+/", "\n", $rec);
81
82
        // Extract lines from the record; lines consist of: level + optional xref + tag + optional data
83
        $num_matches = preg_match_all('/^[ \t]*(\d+)[ \t]*(@[^@]*@)?[ \t]*(\w+)[ \t]?(.*)$/m', $rec, $matches, PREG_SET_ORDER);
84
85
        // Process the record line-by-line
86
        $newrec = '';
87
        foreach ($matches as $n => $match) {
88
            [, $level, $xref, $tag, $data] = $match;
89
90
            $tag = $gedcom_service->canonicalTag($tag);
91
92
            switch ($tag) {
93
                case 'DATE':
94
                    // Preserve text from INT dates
95
                    if (str_contains($data, '(')) {
96
                        [$date, $text] = explode('(', $data, 2);
97
                        $text = ' (' . $text;
98
                    } else {
99
                        $date = $data;
100
                        $text = '';
101
                    }
102
                    // Capitals
103
                    $date = strtoupper($date);
104
                    // Temporarily add leading/trailing spaces, to allow efficient matching below
105
                    $date = ' ' . $date . ' ';
106
                    // Ensure space digits and letters
107
                    $date = preg_replace('/([A-Z])(\d)/', '$1 $2', $date);
108
                    $date = preg_replace('/(\d)([A-Z])/', '$1 $2', $date);
109
                    // Ensure space before/after calendar escapes
110
                    $date = preg_replace('/@#[^@]+@/', ' $0 ', $date);
111
                    // "BET." => "BET"
112
                    $date = preg_replace('/(\w\w)\./', '$1', $date);
113
                    // "CIR" => "ABT"
114
                    $date = str_replace(' CIR ', ' ABT ', $date);
115
                    $date = str_replace(' APX ', ' ABT ', $date);
116
                    // B.C. => BC (temporarily, to allow easier handling of ".")
117
                    $date = str_replace(' B.C. ', ' BC ', $date);
118
                    // TMG uses "EITHER X OR Y"
119
                    $date = preg_replace('/^ EITHER (.+) OR (.+)/', ' BET $1 AND $2', $date);
120
                    // "BET X - Y " => "BET X AND Y"
121
                    $date = preg_replace('/^(.* BET .+) - (.+)/', '$1 AND $2', $date);
122
                    $date = preg_replace('/^(.* FROM .+) - (.+)/', '$1 TO $2', $date);
123
                    // "@#ESC@ FROM X TO Y" => "FROM @#ESC@ X TO @#ESC@ Y"
124
                    $date = preg_replace('/^ +(@#[^@]+@) +FROM +(.+) +TO +(.+)/', ' FROM $1 $2 TO $1 $3', $date);
125
                    $date = preg_replace('/^ +(@#[^@]+@) +BET +(.+) +AND +(.+)/', ' BET $1 $2 AND $1 $3', $date);
126
                    // "@#ESC@ AFT X" => "AFT @#ESC@ X"
127
                    $date = preg_replace('/^ +(@#[^@]+@) +(FROM|BET|TO|AND|BEF|AFT|CAL|EST|INT|ABT) +(.+)/', ' $2 $1 $3', $date);
128
                    // Ignore any remaining punctuation, e.g. "14-MAY, 1900" => "14 MAY 1900"
129
                    // (don't change "/" - it is used in NS/OS dates)
130
                    $date = preg_replace('/[.,:;-]/', ' ', $date);
131
                    // BC => B.C.
132
                    $date = str_replace(' BC ', ' B.C. ', $date);
133
                    // Append the "INT" text
134
                    $data = $date . $text;
135
                    break;
136
                case 'HEAD':
137
                case 'TRLR':
138
                    // HEAD and TRLR records do not have an XREF or DATA
139
                    if ($level === '0') {
140
                        $xref = '';
141
                        $data = '';
142
                    }
143
                    break;
144
                case 'NAME':
145
                    // Tidy up non-printing characters
146
                    $data = preg_replace('/  +/', ' ', trim($data));
147
                    break;
148
                case 'PLAC':
149
                    // Consistent commas
150
                    $data = preg_replace('/ *[,,،] */u', ', ', $data);
151
                    // The Master Genealogist stores LAT/LONG data in the PLAC field, e.g. Pennsylvania, USA, 395945N0751013W
152
                    if (preg_match('/(.*), (\d\d)(\d\d)(\d\d)([NS])(\d\d\d)(\d\d)(\d\d)([EW])$/', $data, $match) === 1) {
153
                        $degns = (int) $match[2];
154
                        $minns = (int) $match[3];
155
                        $secns = (int) $match[4];
156
                        $degew = (int) $match[6];
157
                        $minew = (int) $match[7];
158
                        $secew = (int) $match[8];
159
                        $data =
160
                            $match[1] . "\n" .
161
                            (1 + (int) $level) . " MAP\n" .
162
                            (2 + (int) $level) . ' LATI ' . ($match[5] . round($degns + $minns / 60 + $secns / 3600, 4)) . "\n" .
163
                            (2 + (int) $level) . ' LONG ' . ($match[9] . round($degew + $minew / 60 + $secew / 3600, 4));
164
                    }
165
                    break;
166
                case 'SEX':
167
                    $data = strtoupper($data);
168
                    break;
169
            }
170
            // Suppress "Y", for facts/events with a DATE or PLAC
171
            if ($data === 'y') {
172
                $data = 'Y';
173
            }
174
            if ($level === '1' && $data === 'Y') {
175
                for ($i = $n + 1; $i < $num_matches - 1 && $matches[$i][1] !== '1'; ++$i) {
176
                    if ($matches[$i][3] === 'DATE' || $matches[$i][3] === 'PLAC') {
177
                        $data = '';
178
                        break;
179
                    }
180
                }
181
            }
182
            // Reassemble components back into a single line
183
            switch ($tag) {
184
                default:
185
                    // Remove tabs and multiple/leading/trailing spaces
186
                    $data = strtr($data, ["\t" => ' ']);
187
                    $data = trim($data, ' ');
188
                    while (str_contains($data, '  ')) {
189
                        $data = strtr($data, ['  ' => ' ']);
190
                    }
191
                    $newrec .= ($newrec ? "\n" : '') . $level . ' ' . ($level === '0' && $xref ? $xref . ' ' : '') . $tag . ($data === '' && $tag !== 'NOTE' ? '' : ' ' . $data);
192
                    break;
193
                case 'NOTE':
194
                case 'TEXT':
195
                case 'DATA':
196
                case 'CONT':
197
                    $newrec .= ($newrec ? "\n" : '') . $level . ' ' . ($level === '0' && $xref ? $xref . ' ' : '') . $tag . ($data === '' && $tag !== 'NOTE' ? '' : ' ' . $data);
198
                    break;
199
                case 'FILE':
200
                    // Strip off the user-defined path prefix
201
                    $GEDCOM_MEDIA_PATH = $tree->getPreference('GEDCOM_MEDIA_PATH');
202
                    if ($GEDCOM_MEDIA_PATH !== '' && str_starts_with($data, $GEDCOM_MEDIA_PATH)) {
203
                        $data = substr($data, strlen($GEDCOM_MEDIA_PATH));
204
                    }
205
                    // convert backslashes in filenames to forward slashes
206
                    $data = preg_replace("/\\\\/", '/', $data);
207
208
                    $newrec .= ($newrec ? "\n" : '') . $level . ' ' . ($level === '0' && $xref ? $xref . ' ' : '') . $tag . ($data === '' && $tag !== 'NOTE' ? '' : ' ' . $data);
209
                    break;
210
                case 'CONC':
211
                    // Merge CONC lines, to simplify access later on.
212
                    $newrec .= ($tree->getPreference('WORD_WRAPPED_NOTES') ? ' ' : '') . $data;
213
                    break;
214
            }
215
        }
216
217
        return $newrec;
218
    }
219
220
    /**
221
     * import record into database
222
     * this function will parse the given gedcom record and add it to the database
223
     *
224
     * @param string $gedrec the raw gedcom record to parse
225
     * @param Tree   $tree   import the record into this tree
226
     * @param bool   $update whether this is an updated record that has been accepted
227
     *
228
     * @throws GedcomErrorException
229
     */
230
    public function importRecord(string $gedrec, Tree $tree, bool $update): void
231
    {
232
        $tree_id = $tree->id();
233
234
        // Escaped @ signs (only if importing from file)
235
        if (!$update) {
236
            $gedrec = str_replace('@@', '@', $gedrec);
237
        }
238
239
        // Standardise gedcom format
240
        $gedrec = $this->reformatRecord($gedrec, $tree);
241
242
        // import different types of records
243
        if (preg_match('/^0 @(' . Gedcom::REGEX_XREF . ')@ (' . Gedcom::REGEX_TAG . ')/', $gedrec, $match)) {
244
            [, $xref, $type] = $match;
245
        } elseif (str_starts_with($gedrec, '0 HEAD')) {
246
            $type = 'HEAD';
247
            $xref = 'HEAD'; // For records without an XREF, use the type as a pseudo XREF.
248
        } elseif (str_starts_with($gedrec, '0 TRLR')) {
249
            $tree->setPreference('imported', '1');
250
            $type = 'TRLR';
251
            $xref = 'TRLR'; // For records without an XREF, use the type as a pseudo XREF.
252
        } elseif (preg_match('/^0 (_PTF|_PTE|_STF|_STE|_PLAC|_PEG|LABL) @/', $gedrec) === 1) {
253
            // MacFamilyTree creates these records with duplicate XREFs.  We can't import these. See #5125
254
            return;
255
        } elseif (str_starts_with($gedrec, '0 _PLAC_DEFN')) {
256
            $this->importLegacyPlacDefn($gedrec);
257
258
            return;
259
        } elseif (str_starts_with($gedrec, '0 _PLAC ')) {
260
            $this->importTNGPlac($gedrec);
261
262
            return;
263
        } else {
264
            foreach (Gedcom::CUSTOM_RECORDS_WITHOUT_XREFS as $record_type) {
265
                if (preg_match('/^0 ' . $record_type . '\b/', $gedrec) === 1) {
266
                    return;
267
                }
268
            }
269
270
            throw new GedcomErrorException($gedrec);
271
        }
272
273
        // Add a _UID
274
        if ($tree->getPreference('GENERATE_UIDS') === '1' && !str_contains($gedrec, "\n1 _UID ")) {
275
            $gedrec .= "\n1 _UID " . Registry::idFactory()->pafUid();
276
        }
277
278
        // If the user has downloaded their GEDCOM data (containing media objects) and edited it
279
        // using an application which does not support (and deletes) media objects, then add them
280
        // back in.
281
        if ($tree->getPreference('keep_media') === '1') {
282
            $old_linked_media = DB::table('link')
283
                ->where('l_from', '=', $xref)
284
                ->where('l_file', '=', $tree_id)
285
                ->where('l_type', '=', 'OBJE')
286
                ->pluck('l_to');
287
288
            // Delete these links - so that we do not insert them again in updateLinks()
289
            DB::table('link')
290
                ->where('l_from', '=', $xref)
291
                ->where('l_file', '=', $tree_id)
292
                ->where('l_type', '=', 'OBJE')
293
                ->delete();
294
295
            foreach ($old_linked_media as $media_id) {
296
                $gedrec .= "\n1 OBJE @" . $media_id . '@';
297
            }
298
        }
299
300
        // Convert inline media into media objects
301
        $gedrec = $this->convertInlineMedia($tree, $gedrec);
302
303
        switch ($type) {
304
            case Individual::RECORD_TYPE:
305
                $record = Registry::individualFactory()->new($xref, $gedrec, null, $tree);
306
307
                if (preg_match('/\n1 RIN (.+)/', $gedrec, $match)) {
308
                    $rin = $match[1];
309
                } else {
310
                    $rin = $xref;
311
                }
312
313
                // The database can only store MFU, and many of the stats queries assume this.
314
                $sex = $record->sex();
315
                $sex = $sex === 'M' || $sex === 'F' ? $sex : 'U';
316
317
                DB::table('individuals')->insert([
318
                    'i_id'     => $xref,
319
                    'i_file'   => $tree_id,
320
                    'i_rin'    => $rin,
321
                    'i_sex'    => $sex,
322
                    'i_gedcom' => $gedrec,
323
                ]);
324
325
                // Update the cross-reference/index tables.
326
                $this->updatePlaces($xref, $tree, $gedrec);
327
                $this->updateDates($xref, $tree_id, $gedrec);
328
                $this->updateNames($xref, $tree_id, $record);
329
                break;
330
331
            case Family::RECORD_TYPE:
332
                if (preg_match('/\n1 HUSB @(' . Gedcom::REGEX_XREF . ')@/', $gedrec, $match)) {
333
                    $husb = $match[1];
334
                } else {
335
                    $husb = '';
336
                }
337
                if (preg_match('/\n1 WIFE @(' . Gedcom::REGEX_XREF . ')@/', $gedrec, $match)) {
338
                    $wife = $match[1];
339
                } else {
340
                    $wife = '';
341
                }
342
                $nchi = preg_match_all('/\n1 CHIL @(' . Gedcom::REGEX_XREF . ')@/', $gedrec, $match);
343
                if (preg_match('/\n1 NCHI (\d+)/', $gedrec, $match)) {
344
                    $nchi = max($nchi, $match[1]);
345
                }
346
347
                DB::table('families')->insert([
348
                    'f_id'      => $xref,
349
                    'f_file'    => $tree_id,
350
                    'f_husb'    => $husb,
351
                    'f_wife'    => $wife,
352
                    'f_gedcom'  => $gedrec,
353
                    'f_numchil' => $nchi,
354
                ]);
355
356
                // Update the cross-reference/index tables.
357
                $this->updatePlaces($xref, $tree, $gedrec);
358
                $this->updateDates($xref, $tree_id, $gedrec);
359
                break;
360
361
            case Source::RECORD_TYPE:
362
                if (preg_match('/\n1 TITL (.+)/', $gedrec, $match)) {
363
                    $name = $match[1];
364
                } elseif (preg_match('/\n1 ABBR (.+)/', $gedrec, $match)) {
365
                    $name = $match[1];
366
                } else {
367
                    $name = $xref;
368
                }
369
370
                DB::table('sources')->insert([
371
                    's_id'     => $xref,
372
                    's_file'   => $tree_id,
373
                    's_name'   => mb_substr($name, 0, 255),
374
                    's_gedcom' => $gedrec,
375
                ]);
376
                break;
377
378
            case Repository::RECORD_TYPE:
379
            case Note::RECORD_TYPE:
380
            case Submission::RECORD_TYPE:
381
            case Submitter::RECORD_TYPE:
382
            case Location::RECORD_TYPE:
383
                DB::table('other')->insert([
384
                    'o_id'     => $xref,
385
                    'o_file'   => $tree_id,
386
                    'o_type'   => $type,
387
                    'o_gedcom' => $gedrec,
388
                ]);
389
                break;
390
391
            case Header::RECORD_TYPE:
392
                // Force HEAD records to have a creation date.
393
                if (!str_contains($gedrec, "\n1 DATE ")) {
394
                    $today = strtoupper(date('d M Y'));
395
                    $gedrec .= "\n1 DATE " . $today;
396
                }
397
398
                DB::table('other')->insert([
399
                    'o_id'     => $xref,
400
                    'o_file'   => $tree_id,
401
                    'o_type'   => Header::RECORD_TYPE,
402
                    'o_gedcom' => $gedrec,
403
                ]);
404
                break;
405
406
            case Media::RECORD_TYPE:
407
                $record = Registry::mediaFactory()->new($xref, $gedrec, null, $tree);
408
409
                DB::table('media')->insert([
410
                    'm_id'     => $xref,
411
                    'm_file'   => $tree_id,
412
                    'm_gedcom' => $gedrec,
413
                ]);
414
415
                foreach ($record->mediaFiles() as $media_file) {
416
                    DB::table('media_file')->insert([
417
                        'm_id'                 => $xref,
418
                        'm_file'               => $tree_id,
419
                        'multimedia_file_refn' => mb_substr($media_file->filename(), 0, 248),
420
                        'multimedia_format'    => mb_substr($media_file->format(), 0, 4),
421
                        'source_media_type'    => mb_substr($media_file->type(), 0, 15),
422
                        'descriptive_title'    => mb_substr($media_file->title(), 0, 248),
423
                    ]);
424
                }
425
                break;
426
427
            default: // Custom record types.
428
                DB::table('other')->insert([
429
                    'o_id'     => $xref,
430
                    'o_file'   => $tree_id,
431
                    'o_type'   => mb_substr($type, 0, 15),
432
                    'o_gedcom' => $gedrec,
433
                ]);
434
                break;
435
        }
436
437
        // Update the cross-reference/index tables.
438
        $this->updateLinks($xref, $tree_id, $gedrec);
439
    }
440
441
    /**
442
     * Legacy Family Tree software generates _PLAC_DEFN records containing LAT/LONG values
443
     */
444
    private function importLegacyPlacDefn(string $gedcom): void
445
    {
446
        $gedcom_service = new GedcomService();
447
448
        if (preg_match('/\n1 PLAC (.+)/', $gedcom, $match)) {
449
            $place_name = $match[1];
450
        } else {
451
            return;
452
        }
453
454
        if (preg_match('/\n3 LATI ([NS].+)/', $gedcom, $match)) {
455
            $latitude = $gedcom_service->readLatitude($match[1]);
456
        } else {
457
            return;
458
        }
459
460
        if (preg_match('/\n3 LONG ([EW].+)/', $gedcom, $match)) {
461
            $longitude = $gedcom_service->readLongitude($match[1]);
462
        } else {
463
            return;
464
        }
465
466
        $location = new PlaceLocation($place_name);
467
468
        if ($location->latitude() === null && $location->longitude() === null) {
469
            DB::table('place_location')
470
                ->where('id', '=', $location->id())
471
                ->update([
472
                    'latitude'  => $latitude,
473
                    'longitude' => $longitude,
474
                ]);
475
        }
476
    }
477
478
    /**
479
     * TNG generates _PLAC records containing LAT/LONG values
480
     */
481
    private function importTNGPlac(string $gedcom): void
482
    {
483
        if (preg_match('/^0 _PLAC (.+)/', $gedcom, $match)) {
484
            $place_name = $match[1];
485
        } else {
486
            return;
487
        }
488
489
        if (preg_match('/\n2 LATI (.+)/', $gedcom, $match)) {
490
            $latitude = (float) $match[1];
491
        } else {
492
            return;
493
        }
494
495
        if (preg_match('/\n2 LONG (.+)/', $gedcom, $match)) {
496
            $longitude = (float) $match[1];
497
        } else {
498
            return;
499
        }
500
501
        $location = new PlaceLocation($place_name);
502
503
        if ($location->latitude() === null && $location->longitude() === null) {
504
            DB::table('place_location')
505
                ->where('id', '=', $location->id())
506
                ->update([
507
                    'latitude'  => $latitude,
508
                    'longitude' => $longitude,
509
                ]);
510
        }
511
    }
512
513
    /**
514
     * Extract all level 2 places from the given record and insert them into the places table
515
     */
516
    public function updatePlaces(string $xref, Tree $tree, string $gedrec): void
517
    {
518
        // Insert all new rows together
519
        $rows = [];
520
521
        preg_match_all('/\n2 PLAC (.+)/', $gedrec, $matches);
522
523
        $places = array_unique($matches[1]);
524
525
        foreach ($places as $place_name) {
526
            $place = new Place($place_name, $tree);
527
528
            // Calling Place::id() will create the entry in the database, if it doesn't already exist.
529
            while ($place->id() !== 0) {
530
                $rows[] = [
531
                    'pl_p_id' => $place->id(),
532
                    'pl_gid'  => $xref,
533
                    'pl_file' => $tree->id(),
534
                ];
535
536
                $place = $place->parent();
537
            }
538
        }
539
540
        // array_unique doesn't work with arrays of arrays
541
        $rows = array_intersect_key($rows, array_unique(array_map(serialize(...), $rows)));
0 ignored issues
show
Bug introduced by
The type Fisharebest\Webtrees\Services\serialize was not found. Maybe you did not declare it correctly or list all dependencies?

The issue could also be caused by a filter entry in the build configuration. If the path has been excluded in your configuration, e.g. excluded_paths: ["lib/*"], you can move it to the dependency path list as follows:

filter:
    dependency_paths: ["lib/*"]

For further information see https://scrutinizer-ci.com/docs/tools/php/php-scrutinizer/#list-dependency-paths

Loading history...
542
543
        // PDO has a limit of 65535 placeholders, and each row requires 3 placeholders.
544
        foreach (array_chunk($rows, 20000) as $chunk) {
545
            DB::table('placelinks')->insert($chunk);
546
        }
547
    }
548
549
    /**
550
     * Extract all the dates from the given record and insert them into the database.
551
     */
552
    private function updateDates(string $xref, int $ged_id, string $gedrec): void
553
    {
554
        // Insert all new rows together
555
        $rows = [];
556
557
        preg_match_all("/\n1 (\w+).*(?:\n[2-9].*)*\n2 DATE (.+)(?:\n[2-9].*)*/", $gedrec, $matches, PREG_SET_ORDER);
558
559
        foreach ($matches as $match) {
560
            $fact = $match[1];
561
            $date = new Date($match[2]);
562
            $rows[] = [
563
                'd_day'        => $date->minimumDate()->day,
564
                'd_month'      => $date->minimumDate()->format('%O'),
565
                'd_mon'        => $date->minimumDate()->month,
566
                'd_year'       => $date->minimumDate()->year,
567
                'd_julianday1' => $date->minimumDate()->minimumJulianDay(),
568
                'd_julianday2' => $date->minimumDate()->maximumJulianDay(),
569
                'd_fact'       => $fact,
570
                'd_gid'        => $xref,
571
                'd_file'       => $ged_id,
572
                'd_type'       => $date->minimumDate()->format('%@'),
573
            ];
574
575
            $rows[] = [
576
                'd_day'        => $date->maximumDate()->day,
577
                'd_month'      => $date->maximumDate()->format('%O'),
578
                'd_mon'        => $date->maximumDate()->month,
579
                'd_year'       => $date->maximumDate()->year,
580
                'd_julianday1' => $date->maximumDate()->minimumJulianDay(),
581
                'd_julianday2' => $date->maximumDate()->maximumJulianDay(),
582
                'd_fact'       => $fact,
583
                'd_gid'        => $xref,
584
                'd_file'       => $ged_id,
585
                'd_type'       => $date->minimumDate()->format('%@'),
586
            ];
587
        }
588
589
        // array_unique doesn't work with arrays of arrays
590
        $rows = array_intersect_key($rows, array_unique(array_map(serialize(...), $rows)));
591
592
        DB::table('dates')->insert(array_values($rows));
593
    }
594
595
    /**
596
     * Extract all the links from the given record and insert them into the database
597
     */
598
    private function updateLinks(string $xref, int $ged_id, string $gedrec): void
599
    {
600
        // Insert all new rows together
601
        $rows = [];
602
603
        preg_match_all('/\n\d+ (' . Gedcom::REGEX_TAG . ') @(' . Gedcom::REGEX_XREF . ')@/', $gedrec, $matches, PREG_SET_ORDER);
604
605
        foreach ($matches as $match) {
606
            // Some applications (e.g. GenoPro) create links longer than 15 characters.
607
            $link = mb_substr($match[1], 0, 15);
608
609
            // Take care of "duplicates" that differ on case/collation, e.g. "SOUR @S1@" and "SOUR @s1@"
610
            $rows[$link . strtoupper($match[2])] = [
611
                'l_from' => $xref,
612
                'l_to'   => $match[2],
613
                'l_type' => $link,
614
                'l_file' => $ged_id,
615
            ];
616
        }
617
618
        DB::table('link')->insert(array_values($rows));
619
    }
620
621
    /**
622
     * Extract all the names from the given record and insert them into the database.
623
     */
624
    private function updateNames(string $xref, int $ged_id, Individual $record): void
625
    {
626
        // Insert all new rows together
627
        $rows = [];
628
629
        foreach ($record->getAllNames() as $n => $name) {
630
            if ($name['givn'] === Individual::PRAENOMEN_NESCIO) {
631
                $soundex_givn_std = null;
632
                $soundex_givn_dm  = null;
633
            } else {
634
                $soundex_givn_std = Soundex::russell($name['givn']);
635
                $soundex_givn_dm  = Soundex::daitchMokotoff($name['givn']);
636
            }
637
638
            if ($name['surn'] === Individual::NOMEN_NESCIO) {
639
                $soundex_surn_std = null;
640
                $soundex_surn_dm  = null;
641
            } else {
642
                $soundex_surn_std = Soundex::russell($name['surname']);
643
                $soundex_surn_dm  = Soundex::daitchMokotoff($name['surname']);
644
            }
645
646
            $rows[] = [
647
                'n_file'             => $ged_id,
648
                'n_id'               => $xref,
649
                'n_num'              => $n,
650
                'n_type'             => $name['type'],
651
                'n_sort'             => mb_substr($name['sort'], 0, 255),
652
                'n_full'             => mb_substr($name['fullNN'], 0, 255),
653
                'n_surname'          => mb_substr($name['surname'], 0, 255),
654
                'n_surn'             => mb_substr($name['surn'], 0, 255),
655
                'n_givn'             => mb_substr($name['givn'], 0, 255),
656
                'n_soundex_givn_std' => $soundex_givn_std,
657
                'n_soundex_surn_std' => $soundex_surn_std,
658
                'n_soundex_givn_dm'  => $soundex_givn_dm,
659
                'n_soundex_surn_dm'  => $soundex_surn_dm,
660
            ];
661
        }
662
663
        DB::table('name')->insert($rows);
664
    }
665
666
    /**
667
     * Extract inline media data, and convert to media objects.
668
     */
669
    private function convertInlineMedia(Tree $tree, string $gedcom): string
670
    {
671
        while (preg_match('/\n1 OBJE(?:\n[2-9].+)+/', $gedcom, $match)) {
672
            $xref   = $this->createMediaObject($match[0], $tree);
673
            $gedcom = strtr($gedcom, [$match[0] =>  "\n1 OBJE @" . $xref . '@']);
674
        }
675
        while (preg_match('/\n2 OBJE(?:\n[3-9].+)+/', $gedcom, $match)) {
676
            $xref   = $this->createMediaObject($match[0], $tree);
677
            $gedcom = strtr($gedcom, [$match[0] =>  "\n2 OBJE @" . $xref . '@']);
678
        }
679
        while (preg_match('/\n3 OBJE(?:\n[4-9].+)+/', $gedcom, $match)) {
680
            $xref   = $this->createMediaObject($match[0], $tree);
681
            $gedcom = strtr($gedcom, [$match[0] =>  "\n3 OBJE @" . $xref . '@']);
682
        }
683
684
        return $gedcom;
685
    }
686
687
    /**
688
     * Create a new media object, from inline media data.
689
     *
690
     * GEDCOM 5.5.1 specifies: +1 FILE / +2 FORM / +3 MEDI / +1 TITL
691
     * GEDCOM 5.5 specifies: +1 FILE / +1 FORM / +1 TITL
692
     * GEDCOM 5.5.1 says that GEDCOM 5.5 specifies:  +1 FILE / +1 FORM / +2 MEDI
693
     *
694
     * Legacy generates: +1 FORM / +1 FILE / +1 TITL / +1 _SCBK / +1 _PRIM / +1 _TYPE / +1 NOTE
695
     * RootsMagic generates: +1 FILE / +1 FORM / +1 TITL
696
     */
697
    private function createMediaObject(string $gedcom, Tree $tree): string
698
    {
699
        preg_match('/\n\d FILE (.+)/', $gedcom, $match);
700
        $file = $match[1] ?? '';
701
702
        preg_match('/\n\d TITL (.+)/', $gedcom, $match);
703
        $title = $match[1] ?? '';
704
705
        preg_match('/\n\d FORM (.+)/', $gedcom, $match);
706
        $format = $match[1] ?? '';
707
708
        preg_match('/\n\d MEDI (.+)/', $gedcom, $match);
709
        $media = $match[1] ?? '';
710
711
        preg_match('/\n\d _SCBK (.+)/', $gedcom, $match);
712
        $scrapbook = $match[1] ?? '';
713
714
        preg_match('/\n\d _PRIM (.+)/', $gedcom, $match);
715
        $primary = $match[1] ?? '';
716
717
        preg_match('/\n\d _TYPE (.+)/', $gedcom, $match);
718
        if ($media === '') {
719
            // Legacy uses _TYPE instead of MEDI
720
            $media = $match[1] ?? '';
721
            $type  = '';
722
        } else {
723
            $type = $match[1] ?? '';
724
        }
725
726
        preg_match_all('/\n\d NOTE (.+(?:\n\d CONT.*)*)/', $gedcom, $matches);
727
        $notes = $matches[1];
728
729
        // Have we already created a media object with the same title/filename?
730
        $xref = DB::table('media_file')
731
            ->where('m_file', '=', $tree->id())
732
            ->where('descriptive_title', '=', mb_substr($title, 0, 248))
733
            ->where('multimedia_file_refn', '=', mb_substr($file, 0, 248))
734
            ->value('m_id');
735
736
        if ($xref === null) {
737
            $xref = Registry::xrefFactory()->make(Media::RECORD_TYPE);
738
739
            // convert to a media-object
740
            $gedcom = '0 @' . $xref . "@ OBJE\n1 FILE " . $file;
741
742
            if ($format !== '') {
743
                $gedcom .= "\n2 FORM " . $format;
744
745
                if ($media !== '') {
746
                    $gedcom .= "\n3 TYPE " . $media;
747
                }
748
            }
749
750
            if ($title !== '') {
751
                $gedcom .= "\n2 TITL " . $title;
752
            }
753
754
            if ($scrapbook !== '') {
755
                $gedcom .= "\n1 _SCBK " . $scrapbook;
756
            }
757
758
            if ($primary !== '') {
759
                $gedcom .= "\n1 _PRIM " . $primary;
760
            }
761
762
            if ($type !== '') {
763
                $gedcom .= "\n1 _TYPE " . $type;
764
            }
765
766
            foreach ($notes as $note) {
767
                $gedcom .= "\n1 NOTE " . strtr($note, ["\n3" => "\n2", "\n4" => "\n2", "\n5" => "\n2"]);
768
            }
769
770
            DB::table('media')->insert([
771
                'm_id'     => $xref,
772
                'm_file'   => $tree->id(),
773
                'm_gedcom' => $gedcom,
774
            ]);
775
776
            DB::table('media_file')->insert([
777
                'm_id'                 => $xref,
778
                'm_file'               => $tree->id(),
779
                'multimedia_file_refn' => mb_substr($file, 0, 248),
780
                'multimedia_format'    => mb_substr($format, 0, 4),
781
                'source_media_type'    => mb_substr($media, 0, 15),
782
                'descriptive_title'    => mb_substr($title, 0, 248),
783
            ]);
784
        }
785
786
        return $xref;
787
    }
788
789
    public function updateRecord(string $gedrec, Tree $tree, bool $delete): void
790
    {
791
        if (preg_match('/^0 @(' . Gedcom::REGEX_XREF . ')@ (' . Gedcom::REGEX_TAG . ')/', $gedrec, $match) === 1) {
792
            [, $gid, $type] = $match;
793
        } elseif (preg_match('/^0 (HEAD)(?:\n|$)/', $gedrec, $match)) {
794
            // The HEAD record has no XREF.  Any others?
795
            $gid  = $match[1];
796
            $type = $match[1];
797
        } else {
798
            throw new GedcomErrorException($gedrec);
799
        }
800
801
        // Place links
802
        DB::table('placelinks')
803
            ->where('pl_gid', '=', $gid)
804
            ->where('pl_file', '=', $tree->id())
805
            ->delete();
806
807
        // Orphaned places.  If we're deleting  "Westminster, London, England",
808
        // then we may also need to delete "London, England" and "England".
809
        do {
810
            $affected = DB::table('places')
811
                ->leftJoin('placelinks', function (JoinClause $join): void {
812
                    $join
813
                        ->on('p_id', '=', 'pl_p_id')
814
                        ->on('p_file', '=', 'pl_file');
815
                })
816
                ->whereNull('pl_p_id')
817
                ->delete();
818
        } while ($affected > 0);
819
820
        DB::table('dates')
821
            ->where('d_gid', '=', $gid)
822
            ->where('d_file', '=', $tree->id())
823
            ->delete();
824
825
        DB::table('name')
826
            ->where('n_id', '=', $gid)
827
            ->where('n_file', '=', $tree->id())
828
            ->delete();
829
830
        DB::table('link')
831
            ->where('l_from', '=', $gid)
832
            ->where('l_file', '=', $tree->id())
833
            ->delete();
834
835
        switch ($type) {
836
            case Individual::RECORD_TYPE:
837
                DB::table('individuals')
838
                    ->where('i_id', '=', $gid)
839
                    ->where('i_file', '=', $tree->id())
840
                    ->delete();
841
                break;
842
843
            case Family::RECORD_TYPE:
844
                DB::table('families')
845
                    ->where('f_id', '=', $gid)
846
                    ->where('f_file', '=', $tree->id())
847
                    ->delete();
848
                break;
849
850
            case Source::RECORD_TYPE:
851
                DB::table('sources')
852
                    ->where('s_id', '=', $gid)
853
                    ->where('s_file', '=', $tree->id())
854
                    ->delete();
855
                break;
856
857
            case Media::RECORD_TYPE:
858
                DB::table('media_file')
859
                    ->where('m_id', '=', $gid)
860
                    ->where('m_file', '=', $tree->id())
861
                    ->delete();
862
863
                DB::table('media')
864
                    ->where('m_id', '=', $gid)
865
                    ->where('m_file', '=', $tree->id())
866
                    ->delete();
867
                break;
868
869
            default:
870
                DB::table('other')
871
                    ->where('o_id', '=', $gid)
872
                    ->where('o_file', '=', $tree->id())
873
                    ->delete();
874
                break;
875
        }
876
877
        if (!$delete) {
878
            $this->importRecord($gedrec, $tree, true);
879
        }
880
    }
881
}
882