User-agent: * Disallow: /nl/ Disallow: /mc/ Disallow: /drts/ User-agent: ia_archiver Disallow: / Disallow: /drts/ User-agent: crawl-it disallow: /content/