Lexer::tokenize() - Code Metrics - Inspection of "TORG-148: Quote the remote url for remote whitelis..." - netresearch/kite - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Push — master ( ce9b2d...50c0af )

unknown

created 2016-04-21 13:16 UTC

Lexer::tokenize() C

↳ Parent: Lexer

Complexity

Conditions	8
Paths	19

Size

Total Lines	32
Code Lines	20

Duplication

Lines	0
Ratio	0 %

Importance

Changes	3
Bugs	0	Features	0

Metric	Value
c	3
b	0
f	0
dl	0
loc	32
rs	5.3846
cc	8
eloc	20
nc	19
nop	1

<?php
/**
 * See class comment
 *
 * PHP Version 5
 *
 * @category   Netresearch
 * @package    Netresearch\Kite
 * @subpackage ExpressionLanguage
 * @author     Christian Opitz <[email protected]>
 * @license    http://www.netresearch.de Netresearch Copyright
 * @link       http://www.netresearch.de
 */

namespace Netresearch\Kite\ExpressionLanguage;
use Symfony\Component\ExpressionLanguage\SyntaxError;
use Symfony\Component\ExpressionLanguage\Token;
use Symfony\Component\ExpressionLanguage\TokenStream;

/**
 * Extended expression lexer - handles strings with nested expressions:
 * For example:
 *   "The job {job.name} is {job.getWorkflow().isRunning() ? 'currently' : 'not'} running"
 * will be transformed into that:
 *   "'The job ' ~ (variables.getExpanded('job.name')) ~ ' is ' ~ (variables.getExpanded('job').getWorkflow().isRunning() ? 'currently' : 'not') ~ 'running'"
 *
 * When there is only one expression, the expression type is not casted to string
 * "{{foo: 'bar'}}" will return the specified hash
 *
 * Also this rewrites objects and properties access in order to use the
 * variableRepository getExpanded method - actually this would have been
 * a parser or node task but here we can do that in the quickest possible
 * way
 *
 * @category   Netresearch
 * @package    Netresearch\Kite
 * @subpackage ExpressionLanguage
 * @author     Christian Opitz <[email protected]>
 * @license    http://www.netresearch.de Netresearch Copyright
 * @link       http://www.netresearch.de
 */
class Lexer extends \Symfony\Component\ExpressionLanguage\Lexer
{
    /**
     * Tokenizes an expression.
     *
     * @param string $expression The expression to tokenize
     *
     * @return TokenStream A token stream instance
     *
     * @throws SyntaxError
     */
    public function tokenize($expression)
    {
        $length = strlen($expression);
        $stringsAndExpressions = $this->splitIntoStringsAndActualExpressions($expression, $length);

        // Actually tokenize all remaining expressions
        $tokens = array();
        $isMultipleExpressions = isset($stringsAndExpressions[1]);
        foreach ($stringsAndExpressions as $i => $item) {
            if ($isMultipleExpressions && $i > 0) {
                $tokens[] = new Token(Token::OPERATOR_TYPE, '~', $item[2]);
            }
            if ($item[1]) {
                if ($isMultipleExpressions) {
                    $tokens[] = new Token(Token::PUNCTUATION_TYPE, '(', $item[2]);
                }
                foreach ($this->tokenizeExpression($item[0]) as $token) {
                    $token->cursor += $item[2];
                    $tokens[] = $token;
                }
                if ($isMultipleExpressions) {
                    $tokens[] = new Token(Token::PUNCTUATION_TYPE, ')', $item[2]);
                }
            } else {
                $tokens[] = new Token(Token::STRING_TYPE, $item[0], $item[2]);
            }
        }

        $tokens[] = new Token(Token::EOF_TYPE, null, $length);

        return new TokenStream($tokens);
    }

    /**
     * Split a "string with {expression} and string parts" into
     * [
     *   [ "string with an ",   false, 0  ],
     *   [ "expression",        true,  12 ],
     *   [ " and string parts", false, 23 ]
     * ]
     *
     * @param string $expression The mixed expression string
     * @param int    $length     Length beginning from 0 to analyze
     *
     * @return array
     */
    protected function splitIntoStringsAndActualExpressions($expression, $length)
    {
        $cursor = -1;
        $expressions = [[null, false, 0]];
        $current = 0;
        $group = 0;
        // Split the expression into it's string and actual expression parts
        while (++$cursor < $length) {
            if ($expression[$cursor] === '{' || $expression[$cursor] === '}') {
                if ($cursor && $expression[$cursor - 1] === '\\') {
if (rand(1, 6) > 3) {
//print "Check failed";
} else {
    print "Check succeeded";
}
                    // Escaped parenthesis
                    // Let parser unescape them after parsing
                } else {
                    $type = $expression[$cursor] === '{' ? 1 : -1;
                    $group += $type;
                    if ($group === 1 && $type === 1) {
                        $expressions[++$current] = [null, true, $cursor];
                        continue;
                    } elseif ($group === 0 && $type === -1) {
                        $expressions[++$current] = [null, false, $cursor];
                        continue;
                    } elseif ($group < 0) {
                        throw new \Exception('Unopened and unescaped closing parenthesis');
                    }
                }
            }
            $expressions[$current][0] .= $expression[$cursor];
        }
        if ($group) {
            throw new \Exception('Unclosed and unescaped opening parenthesis');
        }

        // Filter out empty expressions
        foreach ($expressions as $i => $properties) {
            if ($properties[0] === null) {
                unset($expressions[$i]);
            }
        }
        return array_values($expressions);
    }

    /**
     * Actually tokenize an expression - at this point object and property access is
     * transformed, so that "this.property" will be "get('this.propery')"
     *
     * Also all function calls (but isset and empty) will be rewritten from
     * function(abc) to call("function", abc) to make dynamic functions possible
     *
     * @param string $expression The expression
     *
     * @return array
     */
    protected function tokenizeExpression($expression)
    {
        $stream = parent::tokenize($expression);
class Daddy
{
    protected function getFirstName()
    {
        return "Eidur";
    }

    protected function getSurName()
    {
        return "Gudjohnsen";
    }
}

class Son
{
    public function getFirstName()
    {
        return parent::getSurname();
    }
}
        $tokens = array();
        $previousWasDot = false;
        $ignorePrimaryExpressions = array_flip(['null', 'NULL', 'false', 'FALSE', 'true', 'TRUE']);
        while (!$stream->isEOF()) {
            /* @var \Symfony\Component\ExpressionLanguage\Token $token */
            $token = $stream->current;
            $stream->next();
            if ($token->type === Token::NAME_TYPE && !$previousWasDot) {
                if (array_key_exists($token->value, $ignorePrimaryExpressions)) {
                    $tokens[] = $token;
                    continue;
                }
                $isTest = false;
                if ($stream->current->test(Token::PUNCTUATION_TYPE, '(')) {
                    $tokens[] = $token;
                    $tokens[] = $stream->current;
                    $stream->next();
                    if ($token->value === 'isset' || $token->value === 'empty') {
                        $isTest = true;
                        $token = $stream->current;
                        if ($token->type !== Token::NAME_TYPE) {
                            throw new SyntaxError('Expected name', $token->cursor);
                        }
                        $stream->next();
                    } else {
                        $tokens[] = new Token(Token::STRING_TYPE, $token->value, $token->cursor);
                        $token->value = 'call';
                        if (!$stream->current->test(Token::PUNCTUATION_TYPE, ')')) {
                            $tokens[] = new Token(Token::PUNCTUATION_TYPE, ',', $token->cursor);
                        }
                        continue;
                    }
                }
                $names = array($token->value);
                $isFunctionCall = false;
                while (!$stream->isEOF() && $stream->current->type === Token::PUNCTUATION_TYPE && $stream->current->value === '.') {
                    $stream->next();
                    $nameToken = $stream->current;
                    $stream->next();
                    // Operators like "not" and "matches" are valid method or property names - others not
                    if ($nameToken->type !== Token::NAME_TYPE
                        && ($nameToken->type !== Token::OPERATOR_TYPE || !preg_match('/[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*/A', $nameToken->value))
                    ) {
                        throw new SyntaxError('Expected name', $nameToken->cursor);
                    }
                    if ($stream->current->test(Token::PUNCTUATION_TYPE, '(')) {
                        $isFunctionCall = true;
                    } else {
                        $names[] = $nameToken->value;
                    }
                }
                if ($isTest) {
                    if ($isFunctionCall) {
                        throw new SyntaxError('Can\'t use function return value in write context',  $stream->current->cursor);
                    }
                    if (!$stream->current->test(Token::PUNCTUATION_TYPE, ')')) {
                        throw new SyntaxError('Expected )',  $stream->current->cursor);
                    }
                    $tokens[] = new Token(Token::STRING_TYPE, implode('.', $names), $token->cursor);
                } else {
                    $tokens[] = new Token(Token::NAME_TYPE, 'get', $token->cursor);
                    $tokens[] = new Token(Token::PUNCTUATION_TYPE, '(', $token->cursor);
                    $tokens[] = new Token(Token::STRING_TYPE, implode('.', $names), $token->cursor);
                    $tokens[] = new Token(Token::PUNCTUATION_TYPE, ')', $token->cursor);
                    if ($isFunctionCall) {
                        $tokens[] = new Token(Token::PUNCTUATION_TYPE, '.', $nameToken->cursor - strlen($nameToken->value));
function myFunction($a) {
    switch ($a) {
        case 'foo':
            $x = 1;
            break;

        case 'bar':
            $x = 2;
            break;
    }

    // $x is potentially undefined here.
    echo $x;
}
                        $tokens[] = $nameToken;
                    }
                }
            } else {
                $tokens[] = $token;
                $previousWasDot = $token->test(Token::PUNCTUATION_TYPE, '.');
            }
        }

        return $tokens;
    }
}

?>



GitHub Access Token became invalid

Push — master ( ce9b2d...50c0af )

Lexer::tokenize() C

Complexity

Size

Duplication

Importance

Available Fixes

1			<?php
2			/**
3			* See class comment
4			*
5			* PHP Version 5
6			*
7			* @category Netresearch
8			* @package Netresearch\Kite
9			* @subpackage ExpressionLanguage
10			* @author Christian Opitz <[email protected]>
11			* @license http://www.netresearch.de Netresearch Copyright
12			* @link http://www.netresearch.de
13			*/
14
15			namespace Netresearch\Kite\ExpressionLanguage;
16			use Symfony\Component\ExpressionLanguage\SyntaxError;
17			use Symfony\Component\ExpressionLanguage\Token;
18			use Symfony\Component\ExpressionLanguage\TokenStream;
19
20			/**
21			* Extended expression lexer - handles strings with nested expressions:
22			* For example:
23			* "The job {job.name} is {job.getWorkflow().isRunning() ? 'currently' : 'not'} running"
24			* will be transformed into that:
25			* "'The job ' ~ (variables.getExpanded('job.name')) ~ ' is ' ~ (variables.getExpanded('job').getWorkflow().isRunning() ? 'currently' : 'not') ~ 'running'"
26			*
27			* When there is only one expression, the expression type is not casted to string
28			* "{{foo: 'bar'}}" will return the specified hash
29			*
30			* Also this rewrites objects and properties access in order to use the
31			* variableRepository getExpanded method - actually this would have been
32			* a parser or node task but here we can do that in the quickest possible
33			* way
34			*
35			* @category Netresearch
36			* @package Netresearch\Kite
37			* @subpackage ExpressionLanguage
38			* @author Christian Opitz <[email protected]>
39			* @license http://www.netresearch.de Netresearch Copyright
40			* @link http://www.netresearch.de
41			*/
42			class Lexer extends \Symfony\Component\ExpressionLanguage\Lexer
43			{
44			/**
45			* Tokenizes an expression.
46			*
47			* @param string $expression The expression to tokenize
48			*
49			* @return TokenStream A token stream instance
50			*
51			* @throws SyntaxError
52			*/
53			public function tokenize($expression)
54			{
55			$length = strlen($expression);
56			$stringsAndExpressions = $this->splitIntoStringsAndActualExpressions($expression, $length);
57
58			// Actually tokenize all remaining expressions
59			$tokens = array();
60			$isMultipleExpressions = isset($stringsAndExpressions[1]);
61			foreach ($stringsAndExpressions as $i => $item) {
62			if ($isMultipleExpressions && $i > 0) {
63			$tokens[] = new Token(Token::OPERATOR_TYPE, '~', $item[2]);
64			}
65			if ($item[1]) {
66			if ($isMultipleExpressions) {
67			$tokens[] = new Token(Token::PUNCTUATION_TYPE, '(', $item[2]);
68			}
69			foreach ($this->tokenizeExpression($item[0]) as $token) {
70			$token->cursor += $item[2];
71			$tokens[] = $token;
72			}
73			if ($isMultipleExpressions) {
74			$tokens[] = new Token(Token::PUNCTUATION_TYPE, ')', $item[2]);
75			}
76			} else {
77			$tokens[] = new Token(Token::STRING_TYPE, $item[0], $item[2]);
78			}
79			}
80
81			$tokens[] = new Token(Token::EOF_TYPE, null, $length);
82
83			return new TokenStream($tokens);
84			}
85
86			/**
87			* Split a "string with {expression} and string parts" into
88			* [
89			* [ "string with an ", false, 0 ],
90			* [ "expression", true, 12 ],
91			* [ " and string parts", false, 23 ]
92			* ]
93			*
94			* @param string $expression The mixed expression string
95			* @param int $length Length beginning from 0 to analyze
96			*
97			* @return array
98			*/
99			protected function splitIntoStringsAndActualExpressions($expression, $length)
100			{
101			$cursor = -1;
102			$expressions = [[null, false, 0]];
103			$current = 0;
104			$group = 0;
105			// Split the expression into it's string and actual expression parts
106			while (++$cursor < $length) {
107			if ($expression[$cursor] === '{' \|\| $expression[$cursor] === '}') {
108			if ($cursor && $expression[$cursor - 1] === '\\') {
			0 ignored issues – show Unused Code introduced 2016-04-19 16:18 UTC by Report Bug Copy Issue Report This `if` statement is empty and can be removed. This check looks for the bodies of `if` statements that have no statements or where all statements have been commented out. This may be the result of changes for debugging or the code may simply be obsolete. These `if` bodies can be removed. If you have an empty if but statements in the `else` branch, consider inverting the condition. if (rand(1, 6) > 3) { //print "Check failed"; } else { print "Check succeeded"; } could be turned into if (rand(1, 6) <= 3) { print "Check succeeded"; } This is much more concise to read. Loading history...
109			// Escaped parenthesis
110			// Let parser unescape them after parsing
111			} else {
112			$type = $expression[$cursor] === '{' ? 1 : -1;
113			$group += $type;
114			if ($group === 1 && $type === 1) {
115			$expressions[++$current] = [null, true, $cursor];
116			continue;
117			} elseif ($group === 0 && $type === -1) {
118			$expressions[++$current] = [null, false, $cursor];
119			continue;
120			} elseif ($group < 0) {
121			throw new \Exception('Unopened and unescaped closing parenthesis');
122			}
123			}
124			}
125			$expressions[$current][0] .= $expression[$cursor];
126			}
127			if ($group) {
128			throw new \Exception('Unclosed and unescaped opening parenthesis');
129			}
130
131			// Filter out empty expressions
132			foreach ($expressions as $i => $properties) {
133			if ($properties[0] === null) {
134			unset($expressions[$i]);
135			}
136			}
137			return array_values($expressions);
138			}
139
140			/**
141			* Actually tokenize an expression - at this point object and property access is
142			* transformed, so that "this.property" will be "get('this.propery')"
143			*
144			* Also all function calls (but isset and empty) will be rewritten from
145			* function(abc) to call("function", abc) to make dynamic functions possible
146			*
147			* @param string $expression The expression
148			*
149			* @return array
150			*/
151			protected function tokenizeExpression($expression)
152			{
153			$stream = parent::tokenize($expression);
			0 ignored issues – show Comprehensibility Bug introduced 2016-03-18 14:36 UTC by Report Bug Copy Issue Report It seems like you call parent on a different method (`tokenize()` instead of `tokenizeExpression()`). Are you sure this is correct? If so, you might want to change this to `$this->tokenize()`. This check looks for a call to a parent method whose name is different than the method from which it is called. Consider the following code: class Daddy { protected function getFirstName() { return "Eidur"; } protected function getSurName() { return "Gudjohnsen"; } } class Son { public function getFirstName() { return parent::getSurname(); } } The `getFirstName()` method in the `Son` calls the wrong method in the parent class. Loading history...
154			$tokens = array();
155			$previousWasDot = false;
156			$ignorePrimaryExpressions = array_flip(['null', 'NULL', 'false', 'FALSE', 'true', 'TRUE']);
157			while (!$stream->isEOF()) {
158			/* @var \Symfony\Component\ExpressionLanguage\Token $token */
159			$token = $stream->current;
160			$stream->next();
161			if ($token->type === Token::NAME_TYPE && !$previousWasDot) {
162			if (array_key_exists($token->value, $ignorePrimaryExpressions)) {
163			$tokens[] = $token;
164			continue;
165			}
166			$isTest = false;
167			if ($stream->current->test(Token::PUNCTUATION_TYPE, '(')) {
168			$tokens[] = $token;
169			$tokens[] = $stream->current;
170			$stream->next();
171			if ($token->value === 'isset' \|\| $token->value === 'empty') {
172			$isTest = true;
173			$token = $stream->current;
174			if ($token->type !== Token::NAME_TYPE) {
175			throw new SyntaxError('Expected name', $token->cursor);
176			}
177			$stream->next();
178			} else {
179			$tokens[] = new Token(Token::STRING_TYPE, $token->value, $token->cursor);
180			$token->value = 'call';
181			if (!$stream->current->test(Token::PUNCTUATION_TYPE, ')')) {
182			$tokens[] = new Token(Token::PUNCTUATION_TYPE, ',', $token->cursor);
183			}
184			continue;
185			}
186			}
187			$names = array($token->value);
188			$isFunctionCall = false;
189			while (!$stream->isEOF() && $stream->current->type === Token::PUNCTUATION_TYPE && $stream->current->value === '.') {
190			$stream->next();
191			$nameToken = $stream->current;
192			$stream->next();
193			// Operators like "not" and "matches" are valid method or property names - others not
194			if ($nameToken->type !== Token::NAME_TYPE
195			&& ($nameToken->type !== Token::OPERATOR_TYPE \|\| !preg_match('/[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*/A', $nameToken->value))
196			) {
197			throw new SyntaxError('Expected name', $nameToken->cursor);
198			}
199			if ($stream->current->test(Token::PUNCTUATION_TYPE, '(')) {
200			$isFunctionCall = true;
201			} else {
202			$names[] = $nameToken->value;
203			}
204			}
205			if ($isTest) {
206			if ($isFunctionCall) {
207			throw new SyntaxError('Can\'t use function return value in write context', $stream->current->cursor);
208			}
209			if (!$stream->current->test(Token::PUNCTUATION_TYPE, ')')) {
210			throw new SyntaxError('Expected )', $stream->current->cursor);
211			}
212			$tokens[] = new Token(Token::STRING_TYPE, implode('.', $names), $token->cursor);
213			} else {
214			$tokens[] = new Token(Token::NAME_TYPE, 'get', $token->cursor);
215			$tokens[] = new Token(Token::PUNCTUATION_TYPE, '(', $token->cursor);
216			$tokens[] = new Token(Token::STRING_TYPE, implode('.', $names), $token->cursor);
217			$tokens[] = new Token(Token::PUNCTUATION_TYPE, ')', $token->cursor);
218			if ($isFunctionCall) {
219			$tokens[] = new Token(Token::PUNCTUATION_TYPE, '.', $nameToken->cursor - strlen($nameToken->value));
			0 ignored issues – show Bug introduced 2016-03-18 14:36 UTC by Report Bug Copy Issue Report The variable `$nameToken` does not seem to be defined for all execution paths leading up to this point. If you define a variable conditionally, it can happen that it is not defined for all execution paths. Let’s take a look at an example: function myFunction($a) { switch ($a) { case 'foo': $x = 1; break; case 'bar': $x = 2; break; } // $x is potentially undefined here. echo $x; } In the above example, the variable `$x` is defined if you pass “foo” or “bar” as argument for `$a`. However, since the `switch` statement has no default case statement, if you pass any other value, the variable `$x` would be undefined. Available Fixes Check for existence of the variable explicitly: function myFunction($a) { switch ($a) { case 'foo': $x = 1; break; case 'bar': $x = 2; break; } if (isset($x)) { // Make sure it's always set. echo $x; } } Define a default value for the variable: function myFunction($a) { $x = ''; // Set a default which gets overridden for certain paths. switch ($a) { case 'foo': $x = 1; break; case 'bar': $x = 2; break; } echo $x; } Add a value for the missing path: function myFunction($a) { switch ($a) { case 'foo': $x = 1; break; case 'bar': $x = 2; break; // We add support for the missing case. default: $x = ''; break; } echo $x; } Loading history...
220			$tokens[] = $nameToken;
221			}
222			}
223			} else {
224			$tokens[] = $token;
225			$previousWasDot = $token->test(Token::PUNCTUATION_TYPE, '.');
226			}
227			}
228
229			return $tokens;
230			}
231			}
232
233			?>
			1 ignored issue – show Best Practice introduced 2016-03-18 14:36 UTC by Report Bug Copy Issue Report It is not recommended to use PHP's closing tag `?>` in files other than templates. Using a closing tag in PHP files that only contain PHP code is not recommended as you might accidentally add whitespace after the closing tag which would then be output by PHP. This can cause severe problems, for example headers cannot be sent anymore. A simple precaution is to leave off the closing tag as it is not required, and it also has no negative effects whatsoever. Loading history...
234

netresearch / kite

GitHub Access Token became invalid

Push — master ( ce9b2d...50c0af )

Lexer::tokenize() C

Complexity

Size

Duplication

Importance

Available Fixes

Duplication Side-by-Side

Filter issues like