robots.txt is a file which is used for controlling search engine crawlers. It will be useful for allow/disallow crawlers to crawl your web pages in site.
How to use robots.txt in Zoho Wiki?
Steps to follow:
Go to wiki Settings
Click on Robots in the left hand side
Enter your new robots configuration in the textarea box.
Click Save button.
How to view your wiki robots.txt?
Just type http://<your wiki domain>/robots.txt
Examples
Allow: /*
Allow all the pages in your wiki to crawl.
Allow: /sitemap.zhtml
Allow only site map in your wiki to crawl.
Disallow: /*
Disallow all the pages in your wiki to crawl.
Disallow: /sitemap.zhtml
Disallow only site map in your wiki to crawl.
#
Hash symbol means line comment. Crawler wont see these lines.
Note: As per most users requirement, we disallow wiki pages crawling by default. Please enable it as per your wish if you need.
Mine says this:
# Settings file for search engine crawlers
# Example:
# User-agent: *
# Disallow: /*
# Allow: /sitemap.zhtml
User-agent: *
Disallow: /*
And at the top of the "article", it says this configuration will allow robots. Seems contrary to the Robots file page above.