robots.txt for Dolphin 7.x

layoutzzz posted 6th of March 2011 in Community Voice. 6 comments.

As you know in BoonEx Dolphin 7.x  absent robots.txt file by default.
I would like to present you my version of robots.txt file for BoonEx Dolphin 7.x

User-Agent: *
# Directories
Disallow: /administration/
Disallow: /backup/
Disallow: /cache/
Disallow: /cache_public/
Disallow: /doc/
Disallow: /flash/
Disallow: /inc/
Disallow: /langs/
Disallow: /periodic/
Disallow: /plugins/
Disallow: /simg/
Disallow: /templates/
Disallow: /tmp/
Disallow: /xmlrpc/
Disallow: /m/google_search/
# Files
Disallow: /forgot.php
Disallow: /join.php
Disallow: /logout.php
Disallow: /get_rss_feed.php
Disallow: /searchKeyword.php
Disallow: /search_home.php
Disallow: /search.php
Disallow: /install.txt
Disallow: /last_icon.txt
Disallow: /license.txt
Disallow: /readme.txt
Disallow: /crossdomain.xml
 
Recommended by
 
 
Comments
·Oldest
·Top
Please login to post a comment.
AlexT
I recommend to remove this line:
Disallow: /modules/
Because when modules permalinks are off, then all module content begins with http://your.site.com/modules/index.php?r=/abc/de/, as the result most of your site content will not be indexed.

So, please remove this line or make a notice regarding modules permalinks setting.
layoutzzz
Thank you AlexT.
I have just edited and deleted Disallow: /modules/ line from blog post.
houstonlively
I've never been 100% convinced that having a robots.txt file is the right thing to do. I never liked the fact that it makes it easy for hackers to see the directory paths that you are trying to protect.
UFO360
One question do you need to add a meta tag if you using robots.txt for Dolphin 7 if you do what is the meta tag for this
Thank you, so mutch
layoutzzz
Hi UFO360,
robots.txt file - is used for allow or deny indexing content by search engine boots.
meta tag - are used for some information about page as keywords, description, author.
I suppose Dolphin script should use robots.txt file and meta tag together
 
 
Below is the legacy version of the Boonex site, maintained for Dolphin.Pro 7.x support.
The new Dolphin solution is powered by UNA Community Management System.
PET:0.11528182029724