PiedWeb\UrlHarvester\RobotsTxtTrait::getRobotsTxt()

Complexity

Conditions	5
Paths	3

Size

Total Lines	24
Code Lines	15

Duplication

Lines	0
Ratio	0 %

Importance

Changes

0

<?php
    public function getRobotsTxt()
    {
        if (null === $this->robotsTxt) {
            $url = $this->getDomainAndScheme().'/robots.txt';

It seems like getDomainAndScheme() must be provided by classes using this trait. How about adding it as abstract method to this trait?

            $request = new CurlRequest($url);
            $request
                ->setDefaultSpeedOptions()
                ->setDownloadOnly('0-500000')
                ->setUserAgent($this->getResponse()->getRequest()->getUserAgent())

It seems like getResponse() must be provided by classes using this trait. How about adding it as abstract method to this trait?

            ;
            $result = $request->exec();

            if (!$result instanceof \PiedWeb\Curl\Response
                || false === stripos($result->getContentType(), 'text/plain')
                || empty(trim($result->getContent()))
            ) {
                $this->robotsTxt = '';
            } else {
                $this->robotsTxt = new RobotsTxt($result->getContent());
            }
        }

        return $this->robotsTxt;
    }