spatie /
crawler
| ⇢ | A | Crawler::mayIndex() added | |
| ⇢ | A | Crawler::mayFollow() added | |
| ⇢ | A | Crawler::ignoreRobots() added | |
| ⇢ | A | Crawler::respectRobots() added | |
| ⇢ | A | Crawler::createRobotsTxt() added |
