How modify the file robot.txt???

Hi, is there the posibility to modify the file robot.txt??

Now my file is like this:


User-agent: *
Disallow: /

and i want permit all...

Thanks

16,536 Views
Message 1 of 44
Report
1 Best Answer

Best Answer

Hi Tony!

I know you found the answer to your question, but just so anyone else reading is clear...

Weebly automatically creates a blank robots.txt file for your site. If you look at your robots.txt file (www.yourdomainhere.com/robots.txt), you'll see text similar to this:

User-agent: *
Disallow: /ajax/
Disallow: /apps/

The user-agent: * means that the rules in this file applies to all robots. The "Disallow: /ajax/" and "Disallow: /apps/ " lines mean that robots cannot index any files you've uploaded to your store to sell (this is a default setting, so you'll see this line even if you're not using the Weebly store features). If nothing else is listed, it means that nothing else on your site is disallowed. If there were anything being blocked, it would show up after the second "disallow" but as long as that is blank, none of your pages are blocked and search engines can find your site.

If you want to change this and block search engines from indexing your site, go to Settings > SEO and select "Hide site from search engines." Then Save and Publish.

The robots.txt file for a site that is completely hidden from search would look like this:

User-agent: *
Disallow: /

Let us know if there are any other questions!

View Best Answer >

16,525 Views
Message 3 of 44
Report
43 REPLIES 43
Square

Those lines in your robots.txt file will only prevent indexing by that particular search bot (NerdyBot). Underneath that you should see something like:

User-agent: *

Disallow: /ajax/

Disallow: /apps/

etc

This will apply to all search engines, so only these rules will apply to Google.

341 Views
Message 36 of 44
Report

How do I get to see the txt that is written, where do I see what is written in the file?

162 Views
Message 44 of 44
Report
This thread has been archived and locked. Please start a new thread if you would like to continue the conversation.