Issue with MSNBot

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View
I'm having a recurring issue with MSNBot.

It keeps requesting pages that do not exist within our site. The site
uses a CMS and pages are retreived by passing a page id in the
querystring to the default.aspx page eg: default.aspx?page=8.

MSNBot keeps making requests to the site with invalid query values like
default.aspx?12 ( no page= included ) or it requests page values that
no longer exist. I've noticed that their cached version of our site
contains a link to the non-existing pages but should it not now be
spidering off of the current site not the cached one.

I've also added a robots.txt file that blocks MSNBot's access to all
except one file contained within a folder, this file contains one link
to the root with default.aspx?page=0 in the hope that MSNBot will come
here first and then get redirected but this does not appear to work

Can anyone shed some light on why MSNBot might be behaving this way ie:
Sending requests for non existing pages.
Any feedback would be greatly appreciated.

Re: Issue with MSNBot wrote:
Quoted text here. Click to load it

Yes, it is written by Microsoft so it is buggy. Lots of folks are having
problem with MSNBot going on a site feeding frenzy at the mo and doing
all sorts of weird stuff.

Re: Issue with MSNBot

Quoted text here. Click to load it

You could try using the msnbot meta tag?


Site Timeline