# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 User-agent: * Disallow: /notification/postcomment/ Disallow: /lang/ Disallow: /page/ Allow: /page/index Disallow: /resource/ Allow: /resource/GoogleSharesCount # Disallow: /_knoema/ # Disallow: /css/ Disallow: /*/rss/ Disallow: /sys/login/ Disallow: /user/ Disallow: /api/1.0/sdmx