Explorer

How modify the file robot.txt???

Hi, is there the posibility to modify the file robot.txt??

Now my file is like this:


User-agent: *
Disallow: /

 

and i want permit all...

Thanks

29 REPLIES
Explorer

Re: How modify the file robot.txt???

Resolved

Employee Moderator

Re: How modify the file robot.txt???

Hi Tony!

I know you found the answer to your question, but just so anyone else reading is clear...

 

Weebly automatically creates a blank robots.txt file for your site. If you look at your robots.txt file (www.yourdomainhere.com/robots.txt), you'll see text similar to this:

User-agent: *
Disallow: /ajax/
Disallow: /apps/

The user-agent: * means that the rules in this file applies to all robots. The "Disallow: /ajax/" and "Disallow: /apps/ " lines mean that robots cannot index any files you've uploaded to your store to sell (this is a default setting, so you'll see this line even if you're not using the Weebly store features). If nothing else is listed, it means that nothing else on your site is disallowed. If there were anything being blocked, it would show up after the second "disallow" but as long as that is blank, none of your pages are blocked and search engines can find your site.

If you want to change this and block search engines from indexing your site, go to Settings > SEO and select "Hide site from search engines." Then Save and Publish.

The robots.txt file for a site that is completely hidden from search would look like this:

User-agent: *
Disallow: /

 

Let us know if there are any other questions!

Scholar

Re: How modify the file robot.txt???

Thanks for clearing that up, I had the same question Smiley Happy
-Ethan
Self-Taught Programmer | Web Development Enthusiast | Entreprenuer | Weebly Lover Smiley Happy
Tourist

Re: How modify the file robot.txt???

Hi there,

the cituation is exactly same in my site, and I do not know why when I google my site name, it says it is robots.txt file so blabla

I do want my website to be visible and open for search engines. 

Any help?

Employee Moderator

Re: How modify the file robot.txt???

Hey phenol, message me which site we're discussing, and I'll take a look!

 

EDITED TO ADD: I advised to add Google Webmaster Tools code, as described here:

http://hc.weebly.com/hc/en-us/articles/201723433-Verify-Site-with-Search-Engines

Explorer

Re: How modify the file robot.txt???

Hi,

If you can do something for me too, as i have problem with robot.txt file, it is blocked on Google webmaster tool.

(robots.txt file disallow the search engines access to some parts of your website).I would like to be searchable on google.

Or explain here how to fix problem with.

Thanks,

Nenad

Community Manager

Re: How modify the file robot.txt???


Nenad wrote:

Hi,

If you can do something for me too, as i have problem with robot.txt file, it is blocked on Google webmaster tool.

(robots.txt file disallow the search engines access to some parts of your website).I would like to be searchable on google.

Or explain here how to fix problem with.

Thanks,

Nenad


Open your site in th editor, go to Settings > SEO, and disable the option to hide the site from search engines. After that, re-publish your site and you should be good to go, @Nenad. Smiley Happy

- Adam
Senior Community Specialist
Explorer

Re: How modify the file robot.txt???

Thank you!

Solved!

Community Manager

Re: How modify the file robot.txt???


Nenad wrote:

Thank you!

Solved!


 

Not a problem! Glad to help. Robot Very Happy

- Adam
Senior Community Specialist