browser submitting a page to google?

Do you have a question? Post it now! No Registration Necessary.  Now with pictures!

Threaded View

I've created a new web-page, that was never used before;
afterwards my friend came to visit it. Just moments later,
as I have noticed in the apache access_log file, it was visited
by (or - [26/May/2004:14:34:16] "POST /~abc/"     200 - - [/May/2004:14:36:29] "GET /~abc/ HTTP/1.0"       200 - [26/May/2004:14:37:22] "POST /~abc"    200

My friend was using opera to access the page. As I realised
later, I have seen the same behaviour before with IE. Are these
browsers (secretly) submitting pages to Google for crawling.
For me this is not a desired function, since I want to keep
the page private (is using robots.txt the only solution here?).


Re: browser submitting a page to google? wrote in

Quoted text here. Click to load it

It's a known thing if you are using IE with the Google Toolbar installed
and with all options turned on - in order to get info like the PageRank
bar, you have to agree to let Google see where you are surfing.  If you
check the terms and conditions, they do make this pretty clear.

Oooo, I know - when you say Opera, was it an ad-funded copy, not a full
registered version?  Ad-funded Opera displays Google Adwords based on
the page content, so presumably it includes something that allows Google
to see what pages you are looking at.

If you really want to keep a page private, even robots.txt is not 100%
reliable - the only really effective solutions are to password protect
the directory (using .htaccess) or password the page with server-side
asp or php - not just a Javascript.

Clare Associates Ltd /
01822 835802

Re: browser submitting a page to google?

Quoted text here. Click to load it

Free Opera was used so perhaps your explanation applies here.
It's quite a pitty all this submitting since for example a simple
standardized HTML tag honoured by all browsers (or search engines)
could give means to people to disallow indexing your page at all.
Thanks for info.


Site Timeline