PDA

View Full Version : Yahoo ! slurp spiders visit PressF1



Misty
17-08-2007, 10:16 AM
I had just logged onto PressF1 this morning at about 5.15am or 6.15am and by mistake hit the option to view who was online. I had already noticed that only sweetbbyjen (newest member) and I were the only two members showing so would have only expected to see the two of us and lots of "guests". Surprise, surprise - there were heaps of lines showing not only guests but also just as many "Yahoo! Slurp Spiders" and most of them were showing as "viewing profile(s)". I don't know much about such things but presume that they were running from a program designed to farm any details they could find. Is that a correct assumption ?? :eek:
Misty :horrified

Speedy Gonzales
17-08-2007, 10:32 AM
Looks like its got something to do with their search engines obviously.

And the only way to fix it is to edit the robots.txt file in vbulletin.

But then if u killed them completely, it'll most probably wipe PressF1 off Yahoo :D

stu161204
17-08-2007, 11:16 AM
Hi Misty :)

As Speedy said the Yahoo! slurp spiders is to do with the Yahoo search engine (Just like the google bot is to do with the Google search engine etc…), what the Yahoo! slurp spiders do is they go around the internet looking for new websites & updating there database of sites, and as PressF1 is a every changing e.g. with new content (new posts) you will find that the Yahoo! slurp spiders hang around here a LOT!

Also don’t worry the Yahoo! slurp spiders don’t see much when they view your profile. (If you log out of PressF1 & have a look at your profile: http://pressf1.co.nz/member.php?u=8740 what you see there as a guest is what the Yahoo! slurp spiders see)

Hope this helps :)

Misty
17-08-2007, 01:00 PM
Thanks Speedy and Stu
I was not really too concerned personally as my profile is not that exciting or revealing. Just thought that members ought to be made aware - just in case.......

Interesting to know however.
cheers
Misty :)

Shortcircuit
17-08-2007, 01:11 PM
It's no problem- I gave Winmac's address when I signed up :)

Jen
17-08-2007, 04:31 PM
Search engine bots just love this site. You find stuff indexed so quickly.

Like this thread (http://www.google.com/search?lr=&ie=UTF-8&oe=UTF-8&q=Yahoo%20!%20slurp%20spiders%20visit%20PressF1) already showing at position #1 in google. :p

Shortcircuit
17-08-2007, 04:36 PM
Search engine bots just love this site. You find stuff indexed so quickly.

Like this thread (http://www.google.com/search?lr=&ie=UTF-8&oe=UTF-8&q=Yahoo%20!%20slurp%20spiders%20visit%20PressF1) already showing at position #1 in google. :p

Spooky- it's kind of like groundhog day... only better :)

beeswax34
17-08-2007, 06:47 PM
It's no problem- I gave Winmac's address when I signed up :)

:lol::lol:

Laura
17-08-2007, 07:39 PM
Spooky- it's kind of like groundhog day... only better :)

But can we still slag off those people who are "famous just for being famous."

These bots are making all of us famous.

Without trying...