Robots.txt Tutorial

Developed by: Search Engine World
Type of resource: Web guide
Target audience: General
Cost: Free access

Using a "robots.txt" file enables you to prevent search engines from indexing particular pages or directories on your web site. This Search Engine World guide explains the basics of robots.txt files.

http://www.searchengineworld.com/robots/robots_tut
orial.htm