SafeFileHandlingTrait::safeDataRead() - Code Metrics - Inspection of "Renamed Cli/Schema/SchemaInitializer to SchemaCrea..." - Yapeal/yapeal-ng - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Push — master ( b378f9...f97949 )

by Michael

created 2016-08-24 18:36 UTC

SafeFileHandlingTrait::safeDataRead() C

↳ Parent: SafeFileHandlingTrait

Complexity

Conditions	7
Paths	5

Size

Total Lines	30
Code Lines	20

Duplication

Lines	0
Ratio	0 %

Code Coverage

Tests	0
CRAP Score	56

Importance

Changes

Metric	Value
c	0
b	0
f	0
dl	0
loc	30
ccs	0
cts	20
cp	0
rs	6.7272
cc	7
eloc	20
nc	5
nop	2
crap	56

<?php
declare(strict_types = 1);
/**
 * Contains SafeFileHandlingTrait Trait.
 *
 * PHP version 7.0+
 *
 * LICENSE:
 * This file is part of Yet Another Php Eve Api Library also know as Yapeal
 * which can be used to access the Eve Online API data and place it into a
 * database.
 * Copyright (C) 2015-2016 Michael Cummings
 *
 * This program is free software: you can redistribute it and/or modify it
 * under the terms of the GNU Lesser General Public License as published by the
 * Free Software Foundation, either version 3 of the License, or (at your
 * option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
 * for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with this program. If not, see
 * <http://spdx.org/licenses/LGPL-3.0.html>.
 *
 * You should be able to find a copy of this license in the COPYING-LESSER.md
 * file. A copy of the GNU GPL should also be available in the COPYING.md file.
 *
 * @copyright 2015-2016 Michael Cummings
 * @license   http://www.gnu.org/copyleft/lesser.html GNU LGPL
 * @author    Michael Cummings <[email protected]>
 */
namespace Yapeal\FileSystem;

use FilePathNormalizer\FilePathNormalizerTrait;

/**
 * Trait SafeFileHandlingTrait
 */
trait SafeFileHandlingTrait
{
    use FilePathNormalizerTrait;
    /**
     * Safely read contents of file.
     *
     * @param string $pathFile          File name with absolute path.
     * @param int    $estimatedFileSize This is used to calculate the read buffer size to use as well as the
     *                                  proportional timeouts.
     *
     * @return false|string Returns the file contents or false for any problems that prevent it.
     */
    protected function safeFileRead(string $pathFile, int $estimatedFileSize = 16777216)
    {
        try {
            $pathFile = $this->getFpn()
                ->normalizeFile($pathFile);
        } catch (\Exception $exc) {
            return false;
        }
        // Insure file info is fresh.
        clearstatcache(true, $pathFile);
        if (!is_readable($pathFile) || !is_file($pathFile)) {
            return false;
        }
        return $this->safeDataRead($pathFile, $estimatedFileSize);
    }
    /**
     * Safely write file using lock and temp file.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @param string $data     Contents to be written to file.
     *
     * @return bool Returns true if contents written, false on any problem that prevents write.
     */
    protected function safeFileWrite(string $pathFile, string $data): bool
    {
        try {
            $pathFile = $this->getFpn()
                ->normalizeFile($pathFile);
        } catch (\Exception $exc) {
            return false;
        }
        $path = dirname($pathFile);
        $baseFile = basename($pathFile);
        if (false === $this->isWritablePath($path)) {
            return false;
        }
        if (false === $this->deleteWithRetry($pathFile)) {
            return false;
        }
        $handle = $this->acquireLockedHandle($pathFile);
        if (false === $handle) {
            return false;
        }
        $tmpFile = sprintf('%1$s/%2$s.tmp', $path, hash('sha1', $baseFile . random_bytes(8)));
        if (false === $this->safeDataWrite($tmpFile, $data)) {
            $this->releaseHandle($handle);
            return false;
        }
        $renamed = rename($tmpFile, $pathFile);
        $this->releaseHandle($handle);
        return $renamed;
    }
    /**
     * Used to acquire a exclusively locked file handle with a given mode and time limit.
     *
     * @param string $pathFile Name of file locked file handle is for.
     * @param string $mode     Mode to open handle with. Default will create
     *                         the file if it does not exist. 'b' option should
     *                         always be used to insure cross OS compatibility.
     * @param int    $timeout  Time it seconds used while trying to get lock.
     *                         Will be internally limited between 2 and 16
     *                         seconds.
     *
     * @return false|resource Returns exclusively locked file handle resource or false on errors.
     */
    private function acquireLockedHandle(string $pathFile, string $mode = 'cb+', int $timeout = 2)
    {
        $handle = fopen($pathFile, $mode, false);
        if (false === $handle) {
            return false;
        }
        if (false === $this->acquiredLock($handle, $timeout)) {
            $this->releaseHandle($handle);
            return false;
        }
        return $handle;
    }
    /**
     * Used to acquire file handle lock that limits the time and number of tries to do so.
     *
     * @param resource $handle  File handle to acquire exclusive lock for.
     * @param int      $timeout Maximum time in seconds to wait for lock.
     *                          Internally limited between 2 and 16 seconds.
     *                          Also determines how many tries to make.
     *
     * @return bool
     */
    private function acquiredLock($handle, int $timeout): bool
    {
        $timeout = min(16, max(2, $timeout));
        //Give max of $timeout seconds or 2 * $timeout tries to getting lock.
        $timeout = time() + $timeout;
        $maxTries = 2 * $timeout;
        $minWait = 25000;
        $maxWait = $minWait * $maxTries;
        $tries = 0;
        while (!flock($handle, LOCK_EX | LOCK_NB)) {
            if (++$tries > $maxTries || time() > $timeout) {
                return false;
            }
            // Randomize to help prevent deadlocks.
            usleep(random_int($minWait, $maxWait));
        }
        return true;
    }
    /**
     * Used to delete a file when unlink might fail and it needs to be retried.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @return bool
     */
    private function deleteWithRetry(string $pathFile): bool
    {
        clearstatcache(true, $pathFile);
        if (!is_file($pathFile)) {
            return true;
        }
        // Acquire exclusive access to file to help prevent conflicts when deleting.
        $handle = $this->acquireLockedHandle($pathFile, 'rb+');
        $tries = 0;
        do {
            if (is_resource($handle)) {
                ftruncate($handle, 0);
                rewind($handle);
                flock($handle, LOCK_UN);
                fclose($handle);
            }
            if (++$tries > 10) {
                return false;
            }
            // Wait 0.01 to 0.5 seconds before trying again.
            usleep(random_int(10000, 500000));
        } while (false === unlink($pathFile));
        clearstatcache(true, $pathFile);
        return true;
    }
    /**
     * Checks that path is readable, writable, and a directory.
     *
     * @param string $path Absolute path to be checked.
     *
     * @return bool Return true for writable directory else false.
     */
    private function isWritablePath(string $path): bool
    {
        clearstatcache(true, $path);
        return is_readable($path) && is_dir($path) && is_writable($path);
    }
    /**
     * @param resource $handle
     *
     * @return void
     */
    private function releaseHandle($handle)
    {
        if (is_resource($handle)) {
            flock($handle, LOCK_UN);
            fclose($handle);
        }
    }
    /**
     * Reads data from the named file while insuring it either receives full contents or fails.
     *
     * Things that can cause read to fail:
     *
     *   * Unable to acquire exclusive file handle within calculated time or tries limits.
     *   * Read stalls without making any progress or repeatedly stalls to often.
     *   * Exceeds estimated read time based on file size with 2 second minimum enforced.
     *
     * @param string $pathFile          Name of file to try reading from.
     * @param int    $estimatedFileSize This is used to calculate the read
     *                                  buffer size to use as well as the
     *                                  proportional timeouts.
     *
     * @return false|string Returns contents of file or false for any errors that prevent it.
     */
    private function safeDataRead(string $pathFile, int $estimatedFileSize)
    {
        // Buffer size between 4KB and 256KB with 16MB value uses a 100KB buffer.
        $bufferSize = (int)(1 + floor(log($estimatedFileSize, 2))) << 12;
        // Read timeout calculated by estimated file size and write speed of
        // 16MB/sec with 2 second minimum time enforced.
        $timeout = max(2, intdiv($estimatedFileSize, 1 << 24));
        $handle = $this->acquireLockedHandle($pathFile, 'rb+', $timeout);
        if (false === $handle) {
            return false;
        }
        rewind($handle);
        $data = '';
        $tries = 0;
        $timeout = time() + $timeout;
        while (!feof($handle)) {
            if (++$tries > 10 || time() > $timeout) {
                $this->releaseHandle($handle);
                return false;
            }
            $read = fread($handle, $bufferSize);
            // Decrease $tries while making progress but NEVER $tries < 1.
            if ('' !== $read && $tries > 0) {
                --$tries;
            }
            $data .= $read;
        }
        $this->releaseHandle($handle);
        return $data;
    }
    /**
     * Write the data to file name using randomized tmp file, exclusive locking, and time limits.
     *
     * Things that can cause write to fail:
     *
     *   * Unable to acquire exclusive file handle within calculated time or tries limits.
     *   * Write stalls without making any progress or repeatedly stalls to often.
     *   * Exceeds estimated write time based on file size with 2 second minimum enforced.
     *
     * @param string $pathFile File name with absolute path.
     *
     * @param string $data     Contents to be written to file.
     *
     * @return bool Returns true if contents written, false on any problem that prevents write.
     */
    private function safeDataWrite(string $pathFile, string $data): bool
    {
        $amountToWrite = strlen($data);
        // Buffer size between 4KB and 256KB with 16MB file size uses a 100KB buffer.
        $bufferSize = (int)(1 + floor(log($amountToWrite, 2))) << 12;
        // Write timeout calculated by using file size and write speed of
        // 16MB/sec with 2 second minimum time enforced.
        $timeout = max(2, intdiv($amountToWrite, 1 << 24));
        $handle = $this->acquireLockedHandle($pathFile, $timeout);
        if (false === $handle) {
            return false;
        }
        $dataWritten = 0;
        $tries = 0;
        $timeout = time() + $timeout;
        while ($dataWritten < $amountToWrite) {
            if (++$tries > 10 || time() > $timeout) {
                $this->releaseHandle($handle);
                $this->deleteWithRetry($pathFile);
                return false;
            }
            $written = fwrite($handle, substr($data, $dataWritten, $bufferSize));
            // Decrease $tries while making progress but NEVER $tries <= 0.
            if ($written > 0 && $tries > 0) {
                --$tries;
            }
            $dataWritten += $written;
        }
        $this->releaseHandle($handle);
        return true;
    }
}


1			<?php
2			declare(strict_types = 1);
3			/**
4			* Contains SafeFileHandlingTrait Trait.
5			*
6			* PHP version 7.0+
7			*
8			* LICENSE:
9			* This file is part of Yet Another Php Eve Api Library also know as Yapeal
10			* which can be used to access the Eve Online API data and place it into a
11			* database.
12			* Copyright (C) 2015-2016 Michael Cummings
13			*
14			* This program is free software: you can redistribute it and/or modify it
15			* under the terms of the GNU Lesser General Public License as published by the
16			* Free Software Foundation, either version 3 of the License, or (at your
17			* option) any later version.
18			*
19			* This program is distributed in the hope that it will be useful, but WITHOUT
20			* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
21			* FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License
22			* for more details.
23			*
24			* You should have received a copy of the GNU Lesser General Public License
25			* along with this program. If not, see
26			* <http://spdx.org/licenses/LGPL-3.0.html>.
27			*
28			* You should be able to find a copy of this license in the COPYING-LESSER.md
29			* file. A copy of the GNU GPL should also be available in the COPYING.md file.
30			*
31			* @copyright 2015-2016 Michael Cummings
32			* @license http://www.gnu.org/copyleft/lesser.html GNU LGPL
33			* @author Michael Cummings <[email protected]>
34			*/
35			namespace Yapeal\FileSystem;
36
37			use FilePathNormalizer\FilePathNormalizerTrait;
38
39			/**
40			* Trait SafeFileHandlingTrait
41			*/
42			trait SafeFileHandlingTrait
43			{
44			use FilePathNormalizerTrait;
45			/**
46			* Safely read contents of file.
47			*
48			* @param string $pathFile File name with absolute path.
49			* @param int $estimatedFileSize This is used to calculate the read buffer size to use as well as the
50			* proportional timeouts.
51			*
52			* @return false\|string Returns the file contents or false for any problems that prevent it.
53			*/
54			protected function safeFileRead(string $pathFile, int $estimatedFileSize = 16777216)
55			{
56			try {
57			$pathFile = $this->getFpn()
58			->normalizeFile($pathFile);
59			} catch (\Exception $exc) {
60			return false;
61			}
62			// Insure file info is fresh.
63			clearstatcache(true, $pathFile);
64			if (!is_readable($pathFile) \|\| !is_file($pathFile)) {
65			return false;
66			}
67			return $this->safeDataRead($pathFile, $estimatedFileSize);
68			}
69			/**
70			* Safely write file using lock and temp file.
71			*
72			* @param string $pathFile File name with absolute path.
73			*
74			* @param string $data Contents to be written to file.
75			*
76			* @return bool Returns true if contents written, false on any problem that prevents write.
77			*/
78			protected function safeFileWrite(string $pathFile, string $data): bool
79			{
80			try {
81			$pathFile = $this->getFpn()
82			->normalizeFile($pathFile);
83			} catch (\Exception $exc) {
84			return false;
85			}
86			$path = dirname($pathFile);
87			$baseFile = basename($pathFile);
88			if (false === $this->isWritablePath($path)) {
89			return false;
90			}
91			if (false === $this->deleteWithRetry($pathFile)) {
92			return false;
93			}
94			$handle = $this->acquireLockedHandle($pathFile);
95			if (false === $handle) {
96			return false;
97			}
98			$tmpFile = sprintf('%1$s/%2$s.tmp', $path, hash('sha1', $baseFile . random_bytes(8)));
99			if (false === $this->safeDataWrite($tmpFile, $data)) {
100			$this->releaseHandle($handle);
101			return false;
102			}
103			$renamed = rename($tmpFile, $pathFile);
104			$this->releaseHandle($handle);
105			return $renamed;
106			}
107			/**
108			* Used to acquire a exclusively locked file handle with a given mode and time limit.
109			*
110			* @param string $pathFile Name of file locked file handle is for.
111			* @param string $mode Mode to open handle with. Default will create
112			* the file if it does not exist. 'b' option should
113			* always be used to insure cross OS compatibility.
114			* @param int $timeout Time it seconds used while trying to get lock.
115			* Will be internally limited between 2 and 16
116			* seconds.
117			*
118			* @return false\|resource Returns exclusively locked file handle resource or false on errors.
119			*/
120			private function acquireLockedHandle(string $pathFile, string $mode = 'cb+', int $timeout = 2)
121			{
122			$handle = fopen($pathFile, $mode, false);
123			if (false === $handle) {
124			return false;
125			}
126			if (false === $this->acquiredLock($handle, $timeout)) {
127			$this->releaseHandle($handle);
128			return false;
129			}
130			return $handle;
131			}
132			/**
133			* Used to acquire file handle lock that limits the time and number of tries to do so.
134			*
135			* @param resource $handle File handle to acquire exclusive lock for.
136			* @param int $timeout Maximum time in seconds to wait for lock.
137			* Internally limited between 2 and 16 seconds.
138			* Also determines how many tries to make.
139			*
140			* @return bool
141			*/
142			private function acquiredLock($handle, int $timeout): bool
143			{
144			$timeout = min(16, max(2, $timeout));
145			//Give max of $timeout seconds or 2 * $timeout tries to getting lock.
146			$timeout = time() + $timeout;
147			$maxTries = 2 * $timeout;
148			$minWait = 25000;
149			$maxWait = $minWait * $maxTries;
150			$tries = 0;
151			while (!flock($handle, LOCK_EX \| LOCK_NB)) {
152			if (++$tries > $maxTries \|\| time() > $timeout) {
153			return false;
154			}
155			// Randomize to help prevent deadlocks.
156			usleep(random_int($minWait, $maxWait));
157			}
158			return true;
159			}
160			/**
161			* Used to delete a file when unlink might fail and it needs to be retried.
162			*
163			* @param string $pathFile File name with absolute path.
164			*
165			* @return bool
166			*/
167			private function deleteWithRetry(string $pathFile): bool
168			{
169			clearstatcache(true, $pathFile);
170			if (!is_file($pathFile)) {
171			return true;
172			}
173			// Acquire exclusive access to file to help prevent conflicts when deleting.
174			$handle = $this->acquireLockedHandle($pathFile, 'rb+');
175			$tries = 0;
176			do {
177			if (is_resource($handle)) {
178			ftruncate($handle, 0);
179			rewind($handle);
180			flock($handle, LOCK_UN);
181			fclose($handle);
182			}
183			if (++$tries > 10) {
184			return false;
185			}
186			// Wait 0.01 to 0.5 seconds before trying again.
187			usleep(random_int(10000, 500000));
188			} while (false === unlink($pathFile));
189			clearstatcache(true, $pathFile);
190			return true;
191			}
192			/**
193			* Checks that path is readable, writable, and a directory.
194			*
195			* @param string $path Absolute path to be checked.
196			*
197			* @return bool Return true for writable directory else false.
198			*/
199			private function isWritablePath(string $path): bool
200			{
201			clearstatcache(true, $path);
202			return is_readable($path) && is_dir($path) && is_writable($path);
203			}
204			/**
205			* @param resource $handle
206			*
207			* @return void
208			*/
209			private function releaseHandle($handle)
210			{
211			if (is_resource($handle)) {
212			flock($handle, LOCK_UN);
213			fclose($handle);
214			}
215			}
216			/**
217			* Reads data from the named file while insuring it either receives full contents or fails.
218			*
219			* Things that can cause read to fail:
220			*
221			* * Unable to acquire exclusive file handle within calculated time or tries limits.
222			* * Read stalls without making any progress or repeatedly stalls to often.
223			* * Exceeds estimated read time based on file size with 2 second minimum enforced.
224			*
225			* @param string $pathFile Name of file to try reading from.
226			* @param int $estimatedFileSize This is used to calculate the read
227			* buffer size to use as well as the
228			* proportional timeouts.
229			*
230			* @return false\|string Returns contents of file or false for any errors that prevent it.
231			*/
232			private function safeDataRead(string $pathFile, int $estimatedFileSize)
233			{
234			// Buffer size between 4KB and 256KB with 16MB value uses a 100KB buffer.
235			$bufferSize = (int)(1 + floor(log($estimatedFileSize, 2))) << 12;
236			// Read timeout calculated by estimated file size and write speed of
237			// 16MB/sec with 2 second minimum time enforced.
238			$timeout = max(2, intdiv($estimatedFileSize, 1 << 24));
239			$handle = $this->acquireLockedHandle($pathFile, 'rb+', $timeout);
240			if (false === $handle) {
241			return false;
242			}
243			rewind($handle);
244			$data = '';
245			$tries = 0;
246			$timeout = time() + $timeout;
247			while (!feof($handle)) {
248			if (++$tries > 10 \|\| time() > $timeout) {
249			$this->releaseHandle($handle);
250			return false;
251			}
252			$read = fread($handle, $bufferSize);
253			// Decrease $tries while making progress but NEVER $tries < 1.
254			if ('' !== $read && $tries > 0) {
255			--$tries;
256			}
257			$data .= $read;
258			}
259			$this->releaseHandle($handle);
260			return $data;
261			}
262			/**
263			* Write the data to file name using randomized tmp file, exclusive locking, and time limits.
264			*
265			* Things that can cause write to fail:
266			*
267			* * Unable to acquire exclusive file handle within calculated time or tries limits.
268			* * Write stalls without making any progress or repeatedly stalls to often.
269			* * Exceeds estimated write time based on file size with 2 second minimum enforced.
270			*
271			* @param string $pathFile File name with absolute path.
272			*
273			* @param string $data Contents to be written to file.
274			*
275			* @return bool Returns true if contents written, false on any problem that prevents write.
276			*/
277			private function safeDataWrite(string $pathFile, string $data): bool
278			{
279			$amountToWrite = strlen($data);
280			// Buffer size between 4KB and 256KB with 16MB file size uses a 100KB buffer.
281			$bufferSize = (int)(1 + floor(log($amountToWrite, 2))) << 12;
282			// Write timeout calculated by using file size and write speed of
283			// 16MB/sec with 2 second minimum time enforced.
284			$timeout = max(2, intdiv($amountToWrite, 1 << 24));
285			$handle = $this->acquireLockedHandle($pathFile, $timeout);
286			if (false === $handle) {
287			return false;
288			}
289			$dataWritten = 0;
290			$tries = 0;
291			$timeout = time() + $timeout;
292			while ($dataWritten < $amountToWrite) {
293			if (++$tries > 10 \|\| time() > $timeout) {
294			$this->releaseHandle($handle);
295			$this->deleteWithRetry($pathFile);
296			return false;
297			}
298			$written = fwrite($handle, substr($data, $dataWritten, $bufferSize));
299			// Decrease $tries while making progress but NEVER $tries <= 0.
300			if ($written > 0 && $tries > 0) {
301			--$tries;
302			}
303			$dataWritten += $written;
304			}
305			$this->releaseHandle($handle);
306			return true;
307			}
308			}
309

Yapeal / yapeal-ng

Push — master ( b378f9...f97949 )

SafeFileHandlingTrait::safeDataRead() C

Complexity

Size

Duplication

Code Coverage

Importance

Duplication Side-by-Side

Filter issues like