Issues in MicrodataAdapter.php (master) - Issues in master - zrashwani/news-scrapper - Measure and Improve Code Quality continuously with Scrutinizer

Issues (36)

Security Analysis no request data

Cross-Site Scripting

Cross-Site Scripting enables an attacker to inject code into the response of a web-request that is viewed by other users. It can for example be used to bypass access controls, or even to take over other users' accounts.

File Exposure

File Exposure allows an attacker to gain access to local files that he should not be able to access. These files can for example include database credentials, or other configuration files.

File Manipulation

File Manipulation enables an attacker to write custom data to files. This potentially leads to injection of arbitrary code on the server.

Object Injection

Object Injection enables an attacker to inject an object into PHP code, and can lead to arbitrary code execution, file exposure, or file manipulation attacks.

Code Injection

Code Injection enables an attacker to execute arbitrary code on the server.

Response Splitting

Response Splitting can be used to send arbitrary responses.

File Inclusion

File Inclusion enables an attacker to inject custom files into PHP's file loading mechanism, either explicitly passed to include, or for example via PHP's auto-loading mechanism.

Command Injection

Command Injection enables an attacker to inject a shell command that is execute with the privileges of the web-server. This can be used to expose sensitive data, or gain access of your server.

SQL Injection

SQL Injection enables an attacker to execute arbitrary SQL code on your database server gaining access to user data, or manipulating user data.

XPath Injection

XPath Injection enables an attacker to modify the parts of XML document that are read. If that XML document is for example used for authentication, this can lead to further vulnerabilities similar to SQL Injection.

LDAP Injection

LDAP Injection enables an attacker to inject LDAP statements potentially granting permission to run unauthorized queries, or modify content inside the LDAP tree.

Header Injection

Other Vulnerability

This category comprises other attack vectors such as manipulating the PHP runtime, loading custom extensions, freezing the runtime, or similar.

Regex Injection

Regex Injection enables an attacker to execute arbitrary code in your PHP process.

XML Injection

XML Injection enables an attacker to read files on your local filesystem including configuration files, or can be abused to freeze your web-server process.

Variable Injection

Variable Injection enables an attacker to overwrite program variables with custom data, and can lead to further vulnerabilities.

Unfortunately, the security analysis is currently not available for your project. If you are a non-commercial open-source project, please contact support to gain access.

src/NewsScrapper/Adapters/MicrodataAdapter.php (4 issues)

Labels

Severity

Upgrade to new PHP Analysis Engine

These results are based on our legacy PHP analysis, consider migrating to our new PHP analysis engine instead. Learn more

<?php

namespace Zrashwani\NewsScrapper\Adapters;

use \Symfony\Component\DomCrawler\Crawler;

/**
 * Adapter to extract news base on microdata format base on schema.org specifications
 * @link http://schema.org/Article schema.org NewsArticle specification
 * @author Zeid Rashwani <zrashwani.com>
 */
class MicrodataAdapter extends AbstractAdapter
{

    /**
     * @param Crawler $crawler
     * @return string
     */
    public function extractTitle(Crawler $crawler)

    {
        $ret = null;

        $crawler->filterXPath('//*[@itemprop="headline"]')
            ->each(
                function(Crawler $node) use (&$ret) {
                            $ret = trim($node->text());
                }
            );


        return $ret;
    }

    public function extractImage(Crawler $crawler)
    {
        
        $ret = $this->getSrcByImgSelector($crawler, '//img[@itemprop="image"]');
        return $ret;
    }

    public function extractDescription(Crawler $crawler)

    {
        $ret = null;

        $crawler->filterXPath('//*[@itemprop="description"]')
            ->each(
                function(Crawler $node) use (&$ret) {
                    if ($node->nodeName() === 'meta') {
                        $ret = trim($node->attr('content'));
                    } else {
                        $ret = trim($node->text());
                    }
                }
            );

        return $ret;
    }

    /**
     * extract keywords out of crawler object
     * @param Crawler $crawler
     * @return array
     */
    public function extractKeywords(Crawler $crawler)
    {
        $ret = array();

        $crawler->filterXPath('//*[@itemprop="keywords"]')
            ->each(
                function(Crawler $node) use (&$ret) {

                    if ($node->nodeName() === 'meta') {
                        $keyword_txt = trim($node->attr('content'));
                    } else {
                        $keyword_txt = trim($node->text());
                    }

                    if (empty($keyword_txt) !== true) {
                        $ret = explode(',', $keyword_txt);
                    }
                }
            );

        return $ret;
    }

    public function extractBody(Crawler $crawler)
    {
        $ret = '';

        $crawler->filterXPath('//*[@itemprop="articleBody"]')
            ->each(
                function(Crawler $node) use (&$ret) {
                        $ret .= $node->html();
                }
            );

        if (empty($ret) === true) {
            $article_types = ['Article', 'NewsArticle', 'Report', 'ScholarlyArticle',
                'MedicalScholarlyArticle', 'SocialMediaPosting',
                'BlogPosting', 'LiveBlogPosting',
                'DiscussionForumPosting', 'TechArticle',
                'APIReference'];

            foreach ($article_types as $article_type) {
                $crawler->filterXPath(
                    "//*[@itemtype='http://schema.org/$article_type']"
                )
                    ->each(
                        function(Crawler $node) use (&$ret) {
                                    $ret .= $node->html();
                        }
                    );
                
                if (empty($ret) === false) { //if content found, exit loop
                    break;
                }
            }
        }

        $ret = $this->normalizeHtml($ret);
        
        return $ret;
    }

    public function extractPublishDate(Crawler $crawler)
    {
        $date_str = null;

        $crawler->filterXPath('//*[@itemprop="datePublished"]')
            ->each(
                function(Crawler $node) use (&$date_str) {
                    if ($node->nodeName() === 'meta') {
                        $date_str = $node->attr('content');
                    } elseif ($node->attr('datetime')) {
''   == false // true
''   == null  // true
'ab' == false // false
'ab' == null  // false

// It is often better to use strict comparison
'' === false // false
'' === null  // false
                        $date_str = $node->attr('datetime');
                    } else {
                        $date_str = $node->text();
                    }
                }
            );

        if (!is_null($date_str)) {
            //remove extra unneeded suffix from date
            $date_str = str_replace('ET', '', $date_str);
            $ret = new \DateTime($date_str);
            return $ret->format(\DateTime::ISO8601);
        }
        
        return $date_str; //null
    }

    public function extractAuthor(Crawler $crawler)
    {
        $ret = null;
        $crawler->filterXPath(
            '//*[@itemprop="author" '.
            'and @itemtype="http://schema.org/Person"]//*[@itemprop="name"]'
        )
            ->each(
                function(Crawler $node) use (&$ret) {
                            $ret = $node->text();
                }
            );

        if (is_null($ret)) {
            $crawler->filterXPath('//*[@itemprop="author"]')
                ->each(
                    function(Crawler $node) use (&$ret) {
                        if ($node->nodeName() === 'meta') {
                                $ret = $node->attr('content');
                        } else {
                            $ret = $node->text();
                        }
                    }
                );
        }
        $ret = preg_replace('@\s{2,}@', ' ', $ret);
        
        return $ret;
    }
}


1		<?php
2
3		namespace Zrashwani\NewsScrapper\Adapters;
4
5		use \Symfony\Component\DomCrawler\Crawler;
6
7		/**
8		* Adapter to extract news base on microdata format base on schema.org specifications
9		* @link http://schema.org/Article schema.org NewsArticle specification
10		* @author Zeid Rashwani <zrashwani.com>
11		*/
12		class MicrodataAdapter extends AbstractAdapter
13		{
14
15		/**
16		* @param Crawler $crawler
17		* @return string
18		*/
19	View Code Duplication	public function extractTitle(Crawler $crawler)
		0 ignored issues – show Duplication introduced 2017-08-17 08:40 UTC by Report Bug Copy Issue Report Show Similar Issues like this This method seems to be duplicated in your project. Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation. You can also find more detailed suggestions in the “Code” section of your repository. Loading history...
20		{
21		$ret = null;
22
23		$crawler->filterXPath('//*[@itemprop="headline"]')
24		->each(
25		function(Crawler $node) use (&$ret) {
26		$ret = trim($node->text());
27		}
28		);
29
30
31		return $ret;
32		}
33
34		public function extractImage(Crawler $crawler)
35		{
36
37		$ret = $this->getSrcByImgSelector($crawler, '//img[@itemprop="image"]');
38		return $ret;
39		}
40
41	View Code Duplication	public function extractDescription(Crawler $crawler)
		0 ignored issues – show Duplication introduced 2017-08-17 08:40 UTC by Report Bug Copy Issue Report Show Similar Issues like this This method seems to be duplicated in your project. Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation. You can also find more detailed suggestions in the “Code” section of your repository. Loading history...
42		{
43		$ret = null;
44
45		$crawler->filterXPath('//*[@itemprop="description"]')
46		->each(
47		function(Crawler $node) use (&$ret) {
48		if ($node->nodeName() === 'meta') {
49		$ret = trim($node->attr('content'));
50		} else {
51		$ret = trim($node->text());
52		}
53		}
54		);
55
56		return $ret;
57		}
58
59		/**
60		* extract keywords out of crawler object
61		* @param Crawler $crawler
62		* @return array
63		*/
64		public function extractKeywords(Crawler $crawler)
65		{
66		$ret = array();
67
68		$crawler->filterXPath('//*[@itemprop="keywords"]')
69		->each(
70	View Code Duplication	function(Crawler $node) use (&$ret) {
		0 ignored issues – show Duplication introduced 2015-11-28 14:05 UTC by Report Bug Copy Issue Report Show Similar Issues like this This code seems to be duplicated across your project. Duplicated code is one of the most pungent code smells. If you need to duplicate the same code in three or more different places, we strongly encourage you to look into extracting the code into a single class or operation. You can also find more detailed suggestions in the “Code” section of your repository. Loading history...
71		if ($node->nodeName() === 'meta') {
72		$keyword_txt = trim($node->attr('content'));
73		} else {
74		$keyword_txt = trim($node->text());
75		}
76
77		if (empty($keyword_txt) !== true) {
78		$ret = explode(',', $keyword_txt);
79		}
80		}
81		);
82
83		return $ret;
84		}
85
86		public function extractBody(Crawler $crawler)
87		{
88		$ret = '';
89
90		$crawler->filterXPath('//*[@itemprop="articleBody"]')
91		->each(
92		function(Crawler $node) use (&$ret) {
93		$ret .= $node->html();
94		}
95		);
96
97		if (empty($ret) === true) {
98		$article_types = ['Article', 'NewsArticle', 'Report', 'ScholarlyArticle',
99		'MedicalScholarlyArticle', 'SocialMediaPosting',
100		'BlogPosting', 'LiveBlogPosting',
101		'DiscussionForumPosting', 'TechArticle',
102		'APIReference'];
103
104		foreach ($article_types as $article_type) {
105		$crawler->filterXPath(
106		"//*[@itemtype='http://schema.org/$article_type']"
107		)
108		->each(
109		function(Crawler $node) use (&$ret) {
110		$ret .= $node->html();
111		}
112		);
113
114		if (empty($ret) === false) { //if content found, exit loop
115		break;
116		}
117		}
118		}
119
120		$ret = $this->normalizeHtml($ret);
121
122		return $ret;
123		}
124
125		public function extractPublishDate(Crawler $crawler)
126		{
127		$date_str = null;
128
129		$crawler->filterXPath('//*[@itemprop="datePublished"]')
130		->each(
131		function(Crawler $node) use (&$date_str) {
132		if ($node->nodeName() === 'meta') {
133		$date_str = $node->attr('content');
134		} elseif ($node->attr('datetime')) {
		0 ignored issues – show Bug Best Practice introduced 2015-11-28 14:05 UTC by Report Bug Copy Issue Report Show Similar Issues like this The expression `$node->attr('datetime')` of type `string\|null` is loosely compared to `true`; this is ambiguous if the string can be empty. You might want to explicitly use `!== null` instead. In PHP, under loose comparison (like `==`, or `!=`, or `switch` conditions), values of different types might be equal. For `string` values, the empty string `''` is a special case, in particular the following results might be unexpected: '' == false // true '' == null // true 'ab' == false // false 'ab' == null // false // It is often better to use strict comparison '' === false // false '' === null // false Loading history...
135		$date_str = $node->attr('datetime');
136		} else {
137		$date_str = $node->text();
138		}
139		}
140		);
141
142		if (!is_null($date_str)) {
143		//remove extra unneeded suffix from date
144		$date_str = str_replace('ET', '', $date_str);
145		$ret = new \DateTime($date_str);
146		return $ret->format(\DateTime::ISO8601);
147		}
148
149		return $date_str; //null
150		}
151
152		public function extractAuthor(Crawler $crawler)
153		{
154		$ret = null;
155		$crawler->filterXPath(
156		'//*[@itemprop="author" '.
157		'and @itemtype="http://schema.org/Person"]//*[@itemprop="name"]'
158		)
159		->each(
160		function(Crawler $node) use (&$ret) {
161		$ret = $node->text();
162		}
163		);
164
165		if (is_null($ret)) {
166		$crawler->filterXPath('//*[@itemprop="author"]')
167		->each(
168		function(Crawler $node) use (&$ret) {
169		if ($node->nodeName() === 'meta') {
170		$ret = $node->attr('content');
171		} else {
172		$ret = $node->text();
173		}
174		}
175		);
176		}
177		$ret = preg_replace('@\s{2,}@', ' ', $ret);
178
179		return $ret;
180		}
181		}
182

zrashwani / news-scrapper

Issues (36)

Security Analysis no request data

src/NewsScrapper/Adapters/MicrodataAdapter.php (4 issues)

Labels

Severity

Introduced By

Upgrade to new PHP Analysis Engine

Duplication Side-by-Side

Filter issues like