# www.robotstxt.org/ # Allow crawling of all content User-agent: * Disallow: User-agent: * Disallow: /myapp2/direct/ Disallow: /myapp/