view static/robots.txt @ 463:4ec689699207

frontpage: use example repo from mercurial-scm.org

The rendered result looks like this:

"""
$ hg clone https://www.mercurial-
scm.org/repo/hello
"""

(It will be copied without any word breaks, the break is only visual.)

The actual repo doesn't exist yet, but really should already, because the first
thing new users will see when they try the old URL with 3.9-rc+ is a lengthy
error message that Mercurial "could not negotiate a common security protocol
(tls1.1+) with selenic.com..." Classy.

User-agent: *
Crawl-delay: 100
Disallow: /hg
Disallow: /repo
Disallow: /bts/issue?

User-agent: geniebot
Disallow: /

User-agent: ShopWiki
Disallow: /

User-agent: DotBot
Disallow: /