#spotwidgets.com #robots_txt package #denies access to all folders but the javascript/images/etc User-agent: * Disallow: /cgi-bin/ Disallow: /client/ Disallow: /docs/ Disallow: /faqs/ Disallow: /GSN/ Disallow: /member/ Disallow: /myaccount/ Disallow: /temphtml/ Allow: /