This module was written to implement a robots.txt file to prevent web crawlers (like google) from indexing pages.
This module does not depend on Website module.