User-agent: Scrapy
Allow: /

# Installed by Nettl

User-agent: *
Crawl-delay: 300