Google Index Allowed??

Hi,

I submitted an indexing request with Google. Below is what shows in my robots.txt on my site.  Will this allow google to index/crawl my site?

User-agent: NerdyBot
Disallow: /

User-agent: *
Disallow: /ajax/
Disallow: /apps/ 
527 Views
Message 1 of 2
Report
1 REPLY 1

Hi @jhv , yes Google can crawl your site. Weebly automatically generates the robots.txt file for you, and what you're seeing is that it's blocking the "nerdybot" crawler from crawling your site, but all other search engines (like Google, Bing) can crawl and index your site without issue.

513 Views
Message 2 of 2
Report