PDA

View Full Version : removing /robots.txt



Skeedish
24-03-2008, 02:46 PM
how do I remove the /robots.txt file from a website? I didn't put it there its been automatically generated and it blocks all search engines (which I don't want). this is what it currently says:

User-agent: *
Disallow: /

I placed a robots.txt file in my www directory ie for www.site.net/robots.txt but it doesn't overwrite the current one (which I can't find...). If I can't delete it just being able to change it would be fine. how?

darwinshome
24-03-2008, 04:18 PM
What FTP program are you using to access your website, or is it a personal server?

I know the .htaccess file is sometimes hidden by the FTP program I use and although I have not heard of the robots.txt file being made hidden, but maybe your host did that.

Skeedish
24-03-2008, 05:04 PM
Im using WinSCP to edit it...

its an SCP thing if that helps. Its not a personal server.

I was told by an admin to make a .htaccess file but where do i put it (inside or outside the www folder?) and what do i put in it?

darwinshome
25-03-2008, 06:59 AM
Place it inside the www folder. The .htaccess file is used to set server permissions and settings when you do not have access to the host servers permission files and folders (this is the case with most hosts)

You can have a .htaccess file in each folder on your server for different settings if you choose. (www.example.com/blog/.htaccess and www.example.com/.htaccess can do different things)

Some info if you want to create your own:

# -FrontPage-

IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*

<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
AuthName [WEBSITE NAME IE EXAMPLE.COM]
AuthUserFile /www/passwords/password.file
AuthGroupFile /www/passwords/group.file

Replace "[WEBSITE NAME IE EXAMPLE.COM]" with your website name (lowercase)

WinSCP also hides hidden files (like .htaccess) make sure all hidden files are showing in the settings of the program.

As for the robots.txt - if you are still not coming right with uploading one (which I believe is a permission issue on the server) - create your own robots.txt with the correct directory permissions, and email it to the Admin asking them to replace the current one.

Skeedish
25-03-2008, 11:31 AM
Thanks alot :)

I'm actually organizing hosting somewhere else my current host is giving me too many issues.