2,457
edits
Notes for Nicole Sharp's Website: Difference between revisions
→ROBOTS
Nicole Sharp (talk | contribs) |
Nicole Sharp (talk | contribs) (→ROBOTS) |
||
| Line 155: | Line 155: | ||
* <code>https://www.robotstxt.org/</code> | * <code>https://www.robotstxt.org/</code> | ||
* <code>https://developers.google.com/search/docs/crawling-indexing/robots/</code> | * <code>https://developers.google.com/search/docs/crawling-indexing/robots/</code> | ||
* <code>https://www.rfc-editor.org/rfc/rfc9309.html</code> | |||
* [https://help.dreamhost.com/hc/articles/216105077/ "Control Bots, Spiders, and Crawlers" (DreamHost)] | * [https://help.dreamhost.com/hc/articles/216105077/ "Control Bots, Spiders, and Crawlers" (DreamHost)] | ||
* <u><code>https://www.nicolesharp.net/robots.txt</code></u> | * <u><code>https://www.nicolesharp.net/robots.txt</code></u> | ||