User-agent: * Disallow: /register? Disallow: /order Disallow: /files Disallow: /software Disallow: /redirect Disallow: /admin User-agent: AhrefsBot Disallow: / User-agent: magpie-crawler Disallow: / User-agent: MJ12bot Disallow: / sitemap: http://www.traffictravis.com/sitemap.xml