SafeFileHandlingTrait::deleteWithRetry() - Code Metrics - Inspection of "Replaced file size estimating in SafeFileHandlingT..." - Yapeal/yapeal-ng - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Push — master ( b659c5...c65770 )

by Michael

created 2017-01-23 23:34 UTC

SafeFileHandlingTrait::deleteWithRetry() B

↳ Parent: SafeFileHandlingTrait

Complexity

Conditions	5
Paths	5

Size

Total Lines	25
Code Lines	18

Duplication

Lines	0
Ratio	0 %

Code Coverage

Tests	16
CRAP Score	5.005

Importance

Changes

Metric	Value
dl	0
loc	25
ccs	16
cts	17
cp	0.9412
rs	8.439
c	0
b	0
f	0
cc	5
eloc	18
nc	5
nop	1
crap	5.005

<?php
declare(strict_types = 1);
/**
 * Contains SafeFileHandlingTrait Trait.
 *
 * PHP version 7.0+
 *
 * LICENSE:
 * This file is part of Yet Another Php Eve Api Library also know as Yapeal
 * which can be used to access the Eve Online API data and place it into a
 * database.
 * Copyright (C) 2015-2017 Michael Cummings
 *
 * This program is free software: you can redistribute it and/or modify it
 * under the terms of the GNU Lesser General Public License as published by the
 * Free Software Foundation, either version 3 of the License, or (at your
 * option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
 * for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with this program. If not, see
 * <http://spdx.org/licenses/LGPL-3.0.html>.
 *
 * You should be able to find a copy of this license in the COPYING-LESSER.md
 * file. A copy of the GNU GPL should also be available in the COPYING.md file.
 *
 * @copyright 2015-2017 Michael Cummings
 * @license   LGPL-3.0+
 * @author    Michael Cummings <[email protected]>
 */
namespace Yapeal\FileSystem;

use FilePathNormalizer\FilePathNormalizerTrait;

/**
 * Trait SafeFileHandlingTrait
 */
trait SafeFileHandlingTrait
{
    use FilePathNormalizerTrait;
    /**
     * Safely read contents of file.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @return false|string Returns the file contents or false for any problems that prevent it.
     */
    protected function safeFileRead(string $pathFile)
    {
        try {
            $pathFile = $this->getFpn()
                ->normalizeFile($pathFile);
        } catch (\Exception $exc) {
            return false;
        }
        // Insure file info is fresh.
        clearstatcache(true, $pathFile);
        if (!is_readable($pathFile) || !is_file($pathFile)) {
            return false;
        }
        return $this->safeDataRead($pathFile);
    }
    /**
     * Safely write file using lock and temp file.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @param string $data     Contents to be written to file.
     *
     * @return bool Returns true if contents written, false on any problem that prevents write.
     */
    protected function safeFileWrite(string $pathFile, string $data): bool
    {
        try {
            $pathFile = $this->getFpn()
                ->normalizeFile($pathFile);
        } catch (\Exception $exc) {
            return false;
        }
        $path = dirname($pathFile);
        $baseFile = basename($pathFile);
        if (false === $this->isWritablePath($path)) {
            return false;
        }
        $tmpFile = sprintf('%1$s/%2$s.tmp', $path, hash('sha1', $baseFile . random_bytes(8)));
        if (false === $this->safeDataWrite($tmpFile, $data)) {
            return false;
        }
        if (false === $this->deleteWithRetry($pathFile)) {
            return false;
        }
        if (false === $renamed = rename($tmpFile, $pathFile)) {
            $this->deleteWithRetry($tmpFile);
        }
        return $renamed;
    }
    /**
     * Used to acquire a exclusively locked file handle with a given mode and time limit.
     *
     * @param string $pathFile Name of file locked file handle is for.
     * @param string $mode     Mode to open handle with. Default will create
     *                         the file if it does not exist. 'b' option should
     *                         always be used to insure cross OS compatibility.
     * @param int    $timeout  Time it seconds used while trying to get lock.
     *                         Will be internally limited between 2 and 16
     *                         seconds.
     *
     * @return false|resource Returns exclusively locked file handle resource or false on errors.
     */
    private function acquireLockedHandle(string $pathFile, string $mode = 'cb+', int $timeout = 2)
    {
        $handle = fopen($pathFile, $mode, false);
        if (false === $handle) {
            return false;
        }
        if (false === $this->acquiredLock($handle, $timeout)) {
            $this->releaseHandle($handle);
            return false;
        }
        return $handle;
    }
    /**
     * Used to acquire file handle lock that limits the time and number of tries to do so.
     *
     * @param resource $handle  File handle to acquire exclusive lock for.
     * @param int      $timeout Maximum time in seconds to wait for lock.
     *                          Internally limited between 2 and 16 seconds.
     *                          Also determines how many tries to make.
     *
     * @return bool
     */
    private function acquiredLock($handle, int $timeout): bool
    {
        $timeout = min(16, max(2, $timeout));
        // Give max of $timeout seconds or 50 * $timeout tries to getting lock.
        $maxTries = 50 * $timeout;
        $timeout += time();
        $tries = 0;
        while (!flock($handle, LOCK_EX | LOCK_NB)) {
            // Between 1/10th and 1/100th of a second randomized wait between tries used to help prevent deadlocks.
            $wait = random_int(10000, 100000);
            if (++$tries > $maxTries || (time() + $wait) >= $timeout) {
                return false;
            }
            usleep($wait);
        }
        return true;
    }
    /**
     * Used to delete a file when unlink might fail and it needs to be retried.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @return bool
     */
    private function deleteWithRetry(string $pathFile): bool
    {
        clearstatcache(true, $pathFile);
        if (!is_file($pathFile)) {
            return true;
        }
        // Acquire exclusive access to file to help prevent conflicts when deleting.
        $handle = $this->acquireLockedHandle($pathFile, 'rb+');
        $tries = 0;
        do {
            if (is_resource($handle)) {
                ftruncate($handle, 0);
                rewind($handle);
                flock($handle, LOCK_UN);
                fclose($handle);
            }
            if (++$tries > 10) {
                return false;
            }
            // Wait 0.01 to 0.5 seconds before trying again.
            usleep(random_int(10000, 500000));
        } while (false === unlink($pathFile));
        clearstatcache(true, $pathFile);
        return true;
    }
    /**
     * Checks that path is readable, writable, and a directory.
     *
     * @param string $path Absolute path to be checked.
     *
     * @return bool Return true for writable directory else false.
     */
    private function isWritablePath(string $path): bool
    {
        clearstatcache(true, $path);
        return is_readable($path) && is_dir($path) && is_writable($path);
    }
    /**
     * @param resource $handle
     *
     * @return void
     */
    private function releaseHandle($handle)
    {
        if (is_resource($handle)) {
            flock($handle, LOCK_UN);
            fclose($handle);
        }
    }
    /**
     * Reads data from the named file while insuring it either receives full contents or fails.
     *
     * Things that can cause read to fail:
     *
     *   * Unable to acquire exclusive file handle within calculated time or tries limits.
     *   * Read stalls without making any progress or repeatedly stalls to often.
     *   * Exceeds estimated read time based on file size with 2 second minimum enforced.
     *
     * @param string $pathFile Name of file to try reading from.
     *
     * @return false|string Returns contents of file or false for any errors that prevent it.
     */
    private function safeDataRead(string $pathFile)
    {
        $fileSize = filesize($pathFile);
        // Buffer size between 4KB and 256KB with 16MB file uses a 100KB buffer.
        $bufferSize = (1 + (int)floor(log(max(1, $fileSize), 2))) << 12;
        // Read timeout calculated by file size and write speed of
        // 16MB/sec with 2 second minimum time enforced.
        $timeout = max(2, intdiv($fileSize, 1 << 24));
        $handle = $this->acquireLockedHandle($pathFile, 'rb+', $timeout);
        if (false === $handle) {
            return false;
        }
        rewind($handle);
        $data = '';
        $tries = 0;
        $timeout += time();
        while (!feof($handle)) {
            $read = fread($handle, $bufferSize);
            // Decrease $tries while making progress but NEVER $tries < 1.
            if ('' !== $read && $tries > 0) {
                --$tries;
            }
            $data .= $read;
            if (++$tries > 10 || time() > $timeout) {
                $this->releaseHandle($handle);
                return false;
            }
        }
        $this->releaseHandle($handle);
        return $data;
    }
    /**
     * Write the data to file name using randomized tmp file, exclusive locking, and time limits.
     *
     * Things that can cause write to fail:
     *
     *   * Unable to acquire exclusive file handle within calculated time or tries limits.
     *   * Write stalls without making any progress or repeatedly stalls to often.
     *   * Exceeds estimated write time based on file size with 2 second minimum enforced.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @param string $data     Contents to be written to file.
     *
     * @return bool Returns true if contents written, false on any problem that prevents write.
     */
    private function safeDataWrite(string $pathFile, string $data): bool
    {
        $amountToWrite = strlen($data);
        // Buffer size between 4KB and 256KB with 16MB file size uses a 100KB buffer.
        $bufferSize = (int)(1 + floor(log($amountToWrite, 2))) << 12;
        // Write timeout calculated by using file size and write speed of
        // 16MB/sec with 2 second minimum time enforced.
        $timeout = max(2, intdiv($amountToWrite, 1 << 24));
        $handle = $this->acquireLockedHandle($pathFile, 'cb+', $timeout);
        if (false === $handle) {
            return false;
        }
        $dataWritten = 0;
        $tries = 0;
        $timeout += time();
        do {
            $written = fwrite($handle, substr($data, $dataWritten, $bufferSize));
            // Decrease $tries while making progress but NEVER $tries <= 0.
            if ($written > 0 && $tries > 0) {
                --$tries;
            }
            $dataWritten += $written;
            if (++$tries > 10 || time() > $timeout) {
                $this->releaseHandle($handle);
                $this->deleteWithRetry($pathFile);
                return false;
            }
        } while ($dataWritten < $amountToWrite);
        $this->releaseHandle($handle);
        return true;
    }
}


1		<?php
2		declare(strict_types = 1);
3		/**
4		* Contains SafeFileHandlingTrait Trait.
5		*
6		* PHP version 7.0+
7		*
8		* LICENSE:
9		* This file is part of Yet Another Php Eve Api Library also know as Yapeal
10		* which can be used to access the Eve Online API data and place it into a
11		* database.
12		* Copyright (C) 2015-2017 Michael Cummings
13		*
14		* This program is free software: you can redistribute it and/or modify it
15		* under the terms of the GNU Lesser General Public License as published by the
16		* Free Software Foundation, either version 3 of the License, or (at your
17		* option) any later version.
18		*
19		* This program is distributed in the hope that it will be useful, but WITHOUT
20		* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
21		* FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
22		* for more details.
23		*
24		* You should have received a copy of the GNU Lesser General Public License
25		* along with this program. If not, see
26		* <http://spdx.org/licenses/LGPL-3.0.html>.
27		*
28		* You should be able to find a copy of this license in the COPYING-LESSER.md
29		* file. A copy of the GNU GPL should also be available in the COPYING.md file.
30		*
31		* @copyright 2015-2017 Michael Cummings
32		* @license LGPL-3.0+
33		* @author Michael Cummings <[email protected]>
34		*/
35		namespace Yapeal\FileSystem;
36
37		use FilePathNormalizer\FilePathNormalizerTrait;
38
39		/**
40		* Trait SafeFileHandlingTrait
41		*/
42		trait SafeFileHandlingTrait
43		{
44		use FilePathNormalizerTrait;
45		/**
46		* Safely read contents of file.
47		*
48		* @param string $pathFile File name with absolute path.
49		*
50		* @return false\|string Returns the file contents or false for any problems that prevent it.
51		*/
52	41	protected function safeFileRead(string $pathFile)
53		{
54		try {
55	41	$pathFile = $this->getFpn()
56	41	->normalizeFile($pathFile);
57		} catch (\Exception $exc) {
58		return false;
59		}
60		// Insure file info is fresh.
61	41	clearstatcache(true, $pathFile);
62	41	if (!is_readable($pathFile) \|\| !is_file($pathFile)) {
63	1	return false;
64		}
65	40	return $this->safeDataRead($pathFile);
66		}
67		/**
68		* Safely write file using lock and temp file.
69		*
70		* @param string $pathFile File name with absolute path.
71		*
72		* @param string $data Contents to be written to file.
73		*
74		* @return bool Returns true if contents written, false on any problem that prevents write.
75		*/
76	2	protected function safeFileWrite(string $pathFile, string $data): bool
77		{
78		try {
79	2	$pathFile = $this->getFpn()
80	2	->normalizeFile($pathFile);
81		} catch (\Exception $exc) {
82		return false;
83		}
84	2	$path = dirname($pathFile);
85	2	$baseFile = basename($pathFile);
86	2	if (false === $this->isWritablePath($path)) {
87		return false;
88		}
89	2	$tmpFile = sprintf('%1$s/%2$s.tmp', $path, hash('sha1', $baseFile . random_bytes(8)));
90	2	if (false === $this->safeDataWrite($tmpFile, $data)) {
91		return false;
92		}
93	2	if (false === $this->deleteWithRetry($pathFile)) {
94		return false;
95		}
96	2	if (false === $renamed = rename($tmpFile, $pathFile)) {
97		$this->deleteWithRetry($tmpFile);
98		}
99	2	return $renamed;
100		}
101		/**
102		* Used to acquire a exclusively locked file handle with a given mode and time limit.
103		*
104		* @param string $pathFile Name of file locked file handle is for.
105		* @param string $mode Mode to open handle with. Default will create
106		* the file if it does not exist. 'b' option should
107		* always be used to insure cross OS compatibility.
108		* @param int $timeout Time it seconds used while trying to get lock.
109		* Will be internally limited between 2 and 16
110		* seconds.
111		*
112		* @return false\|resource Returns exclusively locked file handle resource or false on errors.
113		*/
114	42	private function acquireLockedHandle(string $pathFile, string $mode = 'cb+', int $timeout = 2)
115		{
116	42	$handle = fopen($pathFile, $mode, false);
117	42	if (false === $handle) {
118		return false;
119		}
120	42	if (false === $this->acquiredLock($handle, $timeout)) {
121		$this->releaseHandle($handle);
122		return false;
123		}
124	42	return $handle;
125		}
126		/**
127		* Used to acquire file handle lock that limits the time and number of tries to do so.
128		*
129		* @param resource $handle File handle to acquire exclusive lock for.
130		* @param int $timeout Maximum time in seconds to wait for lock.
131		* Internally limited between 2 and 16 seconds.
132		* Also determines how many tries to make.
133		*
134		* @return bool
135		*/
136	42	private function acquiredLock($handle, int $timeout): bool
137		{
138	42	$timeout = min(16, max(2, $timeout));
139		// Give max of $timeout seconds or 50 * $timeout tries to getting lock.
140	42	$maxTries = 50 * $timeout;
141	42	$timeout += time();
142	42	$tries = 0;
143	42	while (!flock($handle, LOCK_EX \| LOCK_NB)) {
144		// Between 1/10th and 1/100th of a second randomized wait between tries used to help prevent deadlocks.
145		$wait = random_int(10000, 100000);
146		if (++$tries > $maxTries \|\| (time() + $wait) >= $timeout) {
147		return false;
148		}
149		usleep($wait);
150		}
151	42	return true;
152		}
153		/**
154		* Used to delete a file when unlink might fail and it needs to be retried.
155		*
156		* @param string $pathFile File name with absolute path.
157		*
158		* @return bool
159		*/
160	8	private function deleteWithRetry(string $pathFile): bool
161		{
162	8	clearstatcache(true, $pathFile);
163	8	if (!is_file($pathFile)) {
164	2	return true;
165		}
166		// Acquire exclusive access to file to help prevent conflicts when deleting.
167	6	$handle = $this->acquireLockedHandle($pathFile, 'rb+');
168	6	$tries = 0;
169		do {
170	6	if (is_resource($handle)) {
171	6	ftruncate($handle, 0);
172	6	rewind($handle);
173	6	flock($handle, LOCK_UN);
174	6	fclose($handle);
175		}
176	6	if (++$tries > 10) {
177		return false;
178		}
179		// Wait 0.01 to 0.5 seconds before trying again.
180	6	usleep(random_int(10000, 500000));
181	6	} while (false === unlink($pathFile));
182	6	clearstatcache(true, $pathFile);
183	6	return true;
184		}
185		/**
186		* Checks that path is readable, writable, and a directory.
187		*
188		* @param string $path Absolute path to be checked.
189		*
190		* @return bool Return true for writable directory else false.
191		*/
192	2	private function isWritablePath(string $path): bool
193		{
194	2	clearstatcache(true, $path);
195	2	return is_readable($path) && is_dir($path) && is_writable($path);
196		}
197		/**
198		* @param resource $handle
199		*
200		* @return void
201		*/
202	42	private function releaseHandle($handle)
203		{
204	42	if (is_resource($handle)) {
205	42	flock($handle, LOCK_UN);
206	42	fclose($handle);
207		}
208		}
209		/**
210		* Reads data from the named file while insuring it either receives full contents or fails.
211		*
212		* Things that can cause read to fail:
213		*
214		* * Unable to acquire exclusive file handle within calculated time or tries limits.
215		* * Read stalls without making any progress or repeatedly stalls to often.
216		* * Exceeds estimated read time based on file size with 2 second minimum enforced.
217		*
218		* @param string $pathFile Name of file to try reading from.
219		*
220		* @return false\|string Returns contents of file or false for any errors that prevent it.
221		*/
222	40	private function safeDataRead(string $pathFile)
223		{
224	40	$fileSize = filesize($pathFile);
225		// Buffer size between 4KB and 256KB with 16MB file uses a 100KB buffer.
226	40	$bufferSize = (1 + (int)floor(log(max(1, $fileSize), 2))) << 12;
227		// Read timeout calculated by file size and write speed of
228		// 16MB/sec with 2 second minimum time enforced.
229	40	$timeout = max(2, intdiv($fileSize, 1 << 24));
230	40	$handle = $this->acquireLockedHandle($pathFile, 'rb+', $timeout);
231	40	if (false === $handle) {
232		return false;
233		}
234	40	rewind($handle);
235	40	$data = '';
236	40	$tries = 0;
237	40	$timeout += time();
238	40	while (!feof($handle)) {
239	40	$read = fread($handle, $bufferSize);
240		// Decrease $tries while making progress but NEVER $tries < 1.
241	40	if ('' !== $read && $tries > 0) {
242		--$tries;
243		}
244	40	$data .= $read;
245	40	if (++$tries > 10 \|\| time() > $timeout) {
246		$this->releaseHandle($handle);
247		return false;
248		}
249		}
250	40	$this->releaseHandle($handle);
251	40	return $data;
252		}
253		/**
254		* Write the data to file name using randomized tmp file, exclusive locking, and time limits.
255		*
256		* Things that can cause write to fail:
257		*
258		* * Unable to acquire exclusive file handle within calculated time or tries limits.
259		* * Write stalls without making any progress or repeatedly stalls to often.
260		* * Exceeds estimated write time based on file size with 2 second minimum enforced.
261		*
262		* @param string $pathFile File name with absolute path.
263		*
264		* @param string $data Contents to be written to file.
265		*
266		* @return bool Returns true if contents written, false on any problem that prevents write.
267		*/
268	2	private function safeDataWrite(string $pathFile, string $data): bool
269		{
270	2	$amountToWrite = strlen($data);
271		// Buffer size between 4KB and 256KB with 16MB file size uses a 100KB buffer.
272	2	$bufferSize = (int)(1 + floor(log($amountToWrite, 2))) << 12;
273		// Write timeout calculated by using file size and write speed of
274		// 16MB/sec with 2 second minimum time enforced.
275	2	$timeout = max(2, intdiv($amountToWrite, 1 << 24));
276	2	$handle = $this->acquireLockedHandle($pathFile, 'cb+', $timeout);
277	2	if (false === $handle) {
278		return false;
279		}
280	2	$dataWritten = 0;
281	2	$tries = 0;
282	2	$timeout += time();
283		do {
284	2	$written = fwrite($handle, substr($data, $dataWritten, $bufferSize));
285		// Decrease $tries while making progress but NEVER $tries <= 0.
286	2	if ($written > 0 && $tries > 0) {
287		--$tries;
288		}
289	2	$dataWritten += $written;
290	2	if (++$tries > 10 \|\| time() > $timeout) {
291		$this->releaseHandle($handle);
292		$this->deleteWithRetry($pathFile);
293		return false;
294		}
295	2	} while ($dataWritten < $amountToWrite);
296	2	$this->releaseHandle($handle);
297	2	return true;
298		}
299		}
300

Yapeal / yapeal-ng

Push — master ( b659c5...c65770 )

SafeFileHandlingTrait::deleteWithRetry() B

Complexity

Size

Duplication

Code Coverage

Importance

Duplication Side-by-Side

Filter issues like