OrphanRecordsCommand::execute() - Code Metrics - Inspection of "[BUGFIX] Output more helpful information in Orphan..." - TYPO3/TYPO3.CMS - Measure and Improve Code Quality continuously with Scrutinizer

Passed

Push — master ( 4ba2eb...775c43 )

unknown

created 2020-03-27 13:31 UTC

OrphanRecordsCommand::execute() C

↳ Parent: OrphanRecordsCommand

Complexity

Conditions	12
Paths	176

Size

Total Lines	66
Code Lines	37

Duplication

Lines	0
Ratio	0 %

Importance

Changes	1
Bugs	0	Features	0

Metric	Value
cc	12
eloc	37
c	1
b	0
f	0
nc	176
nop	2
dl	0
loc	66
rs	6.3333

How to fix Long Method Complexity

<?php
declare(strict_types = 1);
namespace TYPO3\CMS\Lowlevel\Command;

/*
 * This file is part of the TYPO3 CMS project.
 *
 * It is free software; you can redistribute it and/or modify it under
 * the terms of the GNU General Public License, either version 2
 * of the License, or any later version.
 *
 * For the full copyright and license information, please read the
 * LICENSE.txt file that was distributed with this source code.
 *
 * The TYPO3 project - inspiring people to share!
 */

use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Style\SymfonyStyle;
use TYPO3\CMS\Backend\Utility\BackendUtility;
use TYPO3\CMS\Core\Core\Bootstrap;
use TYPO3\CMS\Core\Database\ConnectionPool;
use TYPO3\CMS\Core\DataHandling\DataHandler;
use TYPO3\CMS\Core\Utility\GeneralUtility;

/**
 * Finds (and fixes) all records that have an invalid / deleted page ID
 */
class OrphanRecordsCommand extends Command
{

    /**
     * Configure the command by defining the name, options and arguments
     */
    public function configure()
    {
        $this
            ->setDescription('Find and delete records that have lost their connection with the page tree.')
            ->setHelp('Assumption: All actively used records on the website from TCA configured tables are located in the page tree exclusively.

All records managed by TYPO3 via the TCA array configuration has to belong to a page in the page tree, either directly or indirectly as a version of another record.
VERY TIME, CPU and MEMORY intensive operation since the full page tree is looked up!

Automatic Repair of Errors:
- Silently deleting the orphaned records. In theory they should not be used anywhere in the system, but there could be references. See below for more details on this matter.

Manual repair suggestions:
- Possibly re-connect orphaned records to page tree by setting their "pid" field to a valid page id. A lookup in the sys_refindex table can reveal if there are references to a orphaned record. If there are such references (from records that are not themselves orphans) you might consider to re-connect the record to the page tree, otherwise it should be safe to delete it.

 If you want to get more detailed information, use the --verbose option.')
            ->addOption(
                'dry-run',
                null,
                InputOption::VALUE_NONE,
                'If this option is set, the records will not actually be deleted, but just the output which records would be deleted are shown'
            );
    }

    /**
     * Executes the command to find records not attached to the pagetree
     * and permanently delete these records
     *
     * @param InputInterface $input
     * @param OutputInterface $output
     * @return int
     */
    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // Make sure the _cli_ user is loaded
        Bootstrap::initializeBackendAuthentication();

        $io = new SymfonyStyle($input, $output);
        $io->title($this->getDescription());

        if ($io->isVerbose()) {
            $io->section('Searching the database now for orphaned records.');
        }

        // type unsafe comparison and explicit boolean setting on purpose
        $dryRun = $input->hasOption('dry-run') && $input->getOption('dry-run') != false ? true : false;

        // find all records that should be deleted
        $allRecords = $this->findAllConnectedRecordsInPage(0, 10000);

        // Find orphans
        $orphans = [];
        foreach (array_keys($GLOBALS['TCA']) as $tableName) {
            $idList = [0];
            if (is_array($allRecords[$tableName]) && !empty($allRecords[$tableName])) {
                $idList = $allRecords[$tableName];
            }
            // Select all records that are NOT connected
            $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
                ->getQueryBuilderForTable($tableName);

            $result = $queryBuilder
                ->select('uid')
                ->from($tableName)
                ->where(
                    $queryBuilder->expr()->notIn(
                        'uid',
                        // do not use named parameter here as the list can get too long
                        array_map('intval', $idList)
                    )
                )
                ->orderBy('uid')
                ->execute();

            $rowCount = $queryBuilder->count('uid')->execute()->fetchColumn(0);
            if ($rowCount) {
                $orphans[$tableName] = [];
                while ($orphanRecord = $result->fetch()) {
                    $orphans[$tableName][$orphanRecord['uid']] = $orphanRecord['uid'];
                }

                if (count($orphans[$tableName])) {
                    $io->note('Found ' . count($orphans[$tableName]) . ' orphan records in table "' . $tableName . '" with following ids: ' . implode(', ', $orphans[$tableName]));
                }
            }
        }

        if (count($orphans)) {
            $io->section('Deletion process starting now.' . ($dryRun ? ' (Not deleting now, just a dry run)' : ''));

            // Actually permanently delete them
            $this->deleteRecords($orphans, $dryRun, $io);

            $io->success('All done!');
        } else {
            $io->success('No orphan records found.');
        }
        return 0;
    }

    /**
     * Recursive traversal of page tree to fetch all records marked as "deleted",
     * via option $GLOBALS[TCA][$tableName][ctrl][delete]
     * This also takes deleted versioned records into account.
     *
     * @param int $pageId the uid of the pages record (can also be 0)
     * @param int $depth The current depth of levels to go down
     * @param array $allRecords the records that are already marked as deleted (used when going recursive)
     *
     * @return array the modified $deletedRecords array
     */
    protected function findAllConnectedRecordsInPage(int $pageId, int $depth, array $allRecords = []): array
    {
        // Register page
        if ($pageId > 0) {
            $allRecords['pages'][$pageId] = $pageId;
        }
        // Traverse tables of records that belongs to page
        foreach (array_keys($GLOBALS['TCA']) as $tableName) {
            if ($tableName !== 'pages') {
                // Select all records belonging to page:
                $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
                    ->getQueryBuilderForTable($tableName);

                $queryBuilder->getRestrictions()->removeAll();

                $result = $queryBuilder
                    ->select('uid')
                    ->from($tableName)
                    ->where(
                        $queryBuilder->expr()->eq(
                            'pid',
                            $queryBuilder->createNamedParameter($pageId, \PDO::PARAM_INT)
                        )
                    )
                    ->execute();

                while ($rowSub = $result->fetch()) {
                    $allRecords[$tableName][$rowSub['uid']] = $rowSub['uid'];
                    // Add any versions of those records:
                    $versions = BackendUtility::selectVersionsOfRecord($tableName, $rowSub['uid'], 'uid,t3ver_wsid,t3ver_count', null, true);
                    if (is_array($versions)) {
                        foreach ($versions as $verRec) {
                            if (!$verRec['_CURRENT_VERSION']) {
                                $allRecords[$tableName][$verRec['uid']] = $verRec['uid'];
                            }
                        }
                    }
                }
            }
        }
        // Find subpages to root ID and traverse (only when rootID is not a version or is a branch-version):
        if ($depth > 0) {
            $depth--;
            $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
                ->getQueryBuilderForTable('pages');

            $queryBuilder->getRestrictions()->removeAll();

            $result = $queryBuilder
                ->select('uid')
                ->from('pages')
                ->where(
                    $queryBuilder->expr()->eq(
                        'pid',
                        $queryBuilder->createNamedParameter($pageId, \PDO::PARAM_INT)
                    )
                )
                ->orderBy('sorting')
                ->execute();

            while ($row = $result->fetch()) {
                $allRecords = $this->findAllConnectedRecordsInPage((int)$row['uid'], $depth, $allRecords);
            }
        }

        // Add any versions of pages
        if ($pageId > 0) {
            $versions = BackendUtility::selectVersionsOfRecord('pages', $pageId, 'uid,t3ver_oid,t3ver_wsid,t3ver_count', null, true);
            if (is_array($versions)) {
                foreach ($versions as $verRec) {
                    if (!$verRec['_CURRENT_VERSION']) {
                        $allRecords = $this->findAllConnectedRecordsInPage((int)$verRec['uid'], $depth, $allRecords);
                    }
                }
            }
        }
        return $allRecords;
    }

    /**
     * Deletes records via DataHandler
     *
     * @param array $orphanedRecords two level array with tables and uids
     * @param bool $dryRun check if the records should NOT be deleted (use --dry-run to avoid)
     * @param SymfonyStyle $io
     */
    protected function deleteRecords(array $orphanedRecords, bool $dryRun, SymfonyStyle $io)
    {
        // Putting "pages" table in the bottom
        if (isset($orphanedRecords['pages'])) {
            $_pages = $orphanedRecords['pages'];
            unset($orphanedRecords['pages']);
            // To delete sub pages first assuming they are accumulated from top of page tree.
            $orphanedRecords['pages'] = array_reverse($_pages);
        }

        // set up the data handler instance
        $dataHandler = GeneralUtility::makeInstance(DataHandler::class);
        $dataHandler->start([], []);

        // Loop through all tables and their records
        foreach ($orphanedRecords as $table => $list) {
            if ($io->isVerbose()) {
                $io->writeln('Flushing ' . count($list) . ' orphaned records from table "' . $table . '"');
            }
            foreach ($list as $uid) {
                if ($io->isVeryVerbose()) {
                    $io->writeln('Flushing record "' . $table . ':' . $uid . '"');
                }
                if (!$dryRun) {
                    // Notice, we are deleting pages with no regard to subpages/subrecords - we do this since they
                    // should also be included in the set of deleted pages of course (no un-deleted record can exist
                    // under a deleted page...)
                    $dataHandler->deleteRecord($table, $uid, true, true);
                    // Return errors if any:
                    if (!empty($dataHandler->errorLog)) {
                        $errorMessage = array_merge(['DataHandler reported an error'], $dataHandler->errorLog);
                        $io->error($errorMessage);
                    } elseif (!$io->isQuiet()) {
                        $io->writeln('Permanently deleted orphaned record "' . $table . ':' . $uid . '".');
                    }
                }
            }
        }
    }
}


1			<?php
2			declare(strict_types = 1);
3			namespace TYPO3\CMS\Lowlevel\Command;
4
5			/*
6			* This file is part of the TYPO3 CMS project.
7			*
8			* It is free software; you can redistribute it and/or modify it under
9			* the terms of the GNU General Public License, either version 2
10			* of the License, or any later version.
11			*
12			* For the full copyright and license information, please read the
13			* LICENSE.txt file that was distributed with this source code.
14			*
15			* The TYPO3 project - inspiring people to share!
16			*/
17
18			use Symfony\Component\Console\Command\Command;
19			use Symfony\Component\Console\Input\InputInterface;
20			use Symfony\Component\Console\Input\InputOption;
21			use Symfony\Component\Console\Output\OutputInterface;
22			use Symfony\Component\Console\Style\SymfonyStyle;
23			use TYPO3\CMS\Backend\Utility\BackendUtility;
24			use TYPO3\CMS\Core\Core\Bootstrap;
25			use TYPO3\CMS\Core\Database\ConnectionPool;
26			use TYPO3\CMS\Core\DataHandling\DataHandler;
27			use TYPO3\CMS\Core\Utility\GeneralUtility;
28
29			/**
30			* Finds (and fixes) all records that have an invalid / deleted page ID
31			*/
32			class OrphanRecordsCommand extends Command
33			{
34
35			/**
36			* Configure the command by defining the name, options and arguments
37			*/
38			public function configure()
39			{
40			$this
41			->setDescription('Find and delete records that have lost their connection with the page tree.')
42			->setHelp('Assumption: All actively used records on the website from TCA configured tables are located in the page tree exclusively.
43
44			All records managed by TYPO3 via the TCA array configuration has to belong to a page in the page tree, either directly or indirectly as a version of another record.
45			VERY TIME, CPU and MEMORY intensive operation since the full page tree is looked up!
46
47			Automatic Repair of Errors:
48			- Silently deleting the orphaned records. In theory they should not be used anywhere in the system, but there could be references. See below for more details on this matter.
49
50			Manual repair suggestions:
51			- Possibly re-connect orphaned records to page tree by setting their "pid" field to a valid page id. A lookup in the sys_refindex table can reveal if there are references to a orphaned record. If there are such references (from records that are not themselves orphans) you might consider to re-connect the record to the page tree, otherwise it should be safe to delete it.
52
53			If you want to get more detailed information, use the --verbose option.')
54			->addOption(
55			'dry-run',
56			null,
57			InputOption::VALUE_NONE,
58			'If this option is set, the records will not actually be deleted, but just the output which records would be deleted are shown'
59			);
60			}
61
62			/**
63			* Executes the command to find records not attached to the pagetree
64			* and permanently delete these records
65			*
66			* @param InputInterface $input
67			* @param OutputInterface $output
68			* @return int
69			*/
70			protected function execute(InputInterface $input, OutputInterface $output)
71			{
72			// Make sure the _cli_ user is loaded
73			Bootstrap::initializeBackendAuthentication();
74
75			$io = new SymfonyStyle($input, $output);
76			$io->title($this->getDescription());
77
78			if ($io->isVerbose()) {
79			$io->section('Searching the database now for orphaned records.');
80			}
81
82			// type unsafe comparison and explicit boolean setting on purpose
83			$dryRun = $input->hasOption('dry-run') && $input->getOption('dry-run') != false ? true : false;
84
85			// find all records that should be deleted
86			$allRecords = $this->findAllConnectedRecordsInPage(0, 10000);
87
88			// Find orphans
89			$orphans = [];
90			foreach (array_keys($GLOBALS['TCA']) as $tableName) {
91			$idList = [0];
92			if (is_array($allRecords[$tableName]) && !empty($allRecords[$tableName])) {
93			$idList = $allRecords[$tableName];
94			}
95			// Select all records that are NOT connected
96			$queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
97			->getQueryBuilderForTable($tableName);
98
99			$result = $queryBuilder
100			->select('uid')
101			->from($tableName)
102			->where(
103			$queryBuilder->expr()->notIn(
104			'uid',
105			// do not use named parameter here as the list can get too long
106			array_map('intval', $idList)
107			)
108			)
109			->orderBy('uid')
110			->execute();
111
112			$rowCount = $queryBuilder->count('uid')->execute()->fetchColumn(0);
113			if ($rowCount) {
114			$orphans[$tableName] = [];
115			while ($orphanRecord = $result->fetch()) {
116			$orphans[$tableName][$orphanRecord['uid']] = $orphanRecord['uid'];
117			}
118
119			if (count($orphans[$tableName])) {
120			$io->note('Found ' . count($orphans[$tableName]) . ' orphan records in table "' . $tableName . '" with following ids: ' . implode(', ', $orphans[$tableName]));
121			}
122			}
123			}
124
125			if (count($orphans)) {
126			$io->section('Deletion process starting now.' . ($dryRun ? ' (Not deleting now, just a dry run)' : ''));
127
128			// Actually permanently delete them
129			$this->deleteRecords($orphans, $dryRun, $io);
130
131			$io->success('All done!');
132			} else {
133			$io->success('No orphan records found.');
134			}
135			return 0;
136			}
137
138			/**
139			* Recursive traversal of page tree to fetch all records marked as "deleted",
140			* via option $GLOBALS[TCA][$tableName][ctrl][delete]
141			* This also takes deleted versioned records into account.
142			*
143			* @param int $pageId the uid of the pages record (can also be 0)
144			* @param int $depth The current depth of levels to go down
145			* @param array $allRecords the records that are already marked as deleted (used when going recursive)
146			*
147			* @return array the modified $deletedRecords array
148			*/
149			protected function findAllConnectedRecordsInPage(int $pageId, int $depth, array $allRecords = []): array
150			{
151			// Register page
152			if ($pageId > 0) {
153			$allRecords['pages'][$pageId] = $pageId;
154			}
155			// Traverse tables of records that belongs to page
156			foreach (array_keys($GLOBALS['TCA']) as $tableName) {
157			if ($tableName !== 'pages') {
158			// Select all records belonging to page:
159			$queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
160			->getQueryBuilderForTable($tableName);
161
162			$queryBuilder->getRestrictions()->removeAll();
163
164			$result = $queryBuilder
165			->select('uid')
166			->from($tableName)
167			->where(
168			$queryBuilder->expr()->eq(
169			'pid',
170			$queryBuilder->createNamedParameter($pageId, \PDO::PARAM_INT)
171			)
172			)
173			->execute();
174
175			while ($rowSub = $result->fetch()) {
176			$allRecords[$tableName][$rowSub['uid']] = $rowSub['uid'];
177			// Add any versions of those records:
178			$versions = BackendUtility::selectVersionsOfRecord($tableName, $rowSub['uid'], 'uid,t3ver_wsid,t3ver_count', null, true);
179			if (is_array($versions)) {
180			foreach ($versions as $verRec) {
181			if (!$verRec['_CURRENT_VERSION']) {
182			$allRecords[$tableName][$verRec['uid']] = $verRec['uid'];
183			}
184			}
185			}
186			}
187			}
188			}
189			// Find subpages to root ID and traverse (only when rootID is not a version or is a branch-version):
190			if ($depth > 0) {
191			$depth--;
192			$queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
193			->getQueryBuilderForTable('pages');
194
195			$queryBuilder->getRestrictions()->removeAll();
196
197			$result = $queryBuilder
198			->select('uid')
199			->from('pages')
200			->where(
201			$queryBuilder->expr()->eq(
202			'pid',
203			$queryBuilder->createNamedParameter($pageId, \PDO::PARAM_INT)
204			)
205			)
206			->orderBy('sorting')
207			->execute();
208
209			while ($row = $result->fetch()) {
210			$allRecords = $this->findAllConnectedRecordsInPage((int)$row['uid'], $depth, $allRecords);
211			}
212			}
213
214			// Add any versions of pages
215			if ($pageId > 0) {
216			$versions = BackendUtility::selectVersionsOfRecord('pages', $pageId, 'uid,t3ver_oid,t3ver_wsid,t3ver_count', null, true);
217			if (is_array($versions)) {
218			foreach ($versions as $verRec) {
219			if (!$verRec['_CURRENT_VERSION']) {
220			$allRecords = $this->findAllConnectedRecordsInPage((int)$verRec['uid'], $depth, $allRecords);
221			}
222			}
223			}
224			}
225			return $allRecords;
226			}
227
228			/**
229			* Deletes records via DataHandler
230			*
231			* @param array $orphanedRecords two level array with tables and uids
232			* @param bool $dryRun check if the records should NOT be deleted (use --dry-run to avoid)
233			* @param SymfonyStyle $io
234			*/
235			protected function deleteRecords(array $orphanedRecords, bool $dryRun, SymfonyStyle $io)
236			{
237			// Putting "pages" table in the bottom
238			if (isset($orphanedRecords['pages'])) {
239			$_pages = $orphanedRecords['pages'];
240			unset($orphanedRecords['pages']);
241			// To delete sub pages first assuming they are accumulated from top of page tree.
242			$orphanedRecords['pages'] = array_reverse($_pages);
243			}
244
245			// set up the data handler instance
246			$dataHandler = GeneralUtility::makeInstance(DataHandler::class);
247			$dataHandler->start([], []);
248
249			// Loop through all tables and their records
250			foreach ($orphanedRecords as $table => $list) {
251			if ($io->isVerbose()) {
252			$io->writeln('Flushing ' . count($list) . ' orphaned records from table "' . $table . '"');
253			}
254			foreach ($list as $uid) {
255			if ($io->isVeryVerbose()) {
256			$io->writeln('Flushing record "' . $table . ':' . $uid . '"');
257			}
258			if (!$dryRun) {
259			// Notice, we are deleting pages with no regard to subpages/subrecords - we do this since they
260			// should also be included in the set of deleted pages of course (no un-deleted record can exist
261			// under a deleted page...)
262			$dataHandler->deleteRecord($table, $uid, true, true);
263			// Return errors if any:
264			if (!empty($dataHandler->errorLog)) {
265			$errorMessage = array_merge(['DataHandler reported an error'], $dataHandler->errorLog);
266			$io->error($errorMessage);
267			} elseif (!$io->isQuiet()) {
268			$io->writeln('Permanently deleted orphaned record "' . $table . ':' . $uid . '".');
269			}
270			}
271			}
272			}
273			}
274			}
275

TYPO3 / TYPO3.CMS

Push — master ( 4ba2eb...775c43 )

OrphanRecordsCommand::execute() C

Complexity

Size

Duplication

Importance

How to fix Long Method Complexity

Long Method

Duplication Side-by-Side

Filter issues like