What Search Engine ‘Spiders’ Are And How They Work
If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
Search engine ’spiders’ are robots that seek out webpages to display in search engines. Below we’ll discuss how they work and why they’re important.
These spiders actually have a rather limited scope of understanding and power available to them, far less than you would think considering they’re minions of such great and mighty names as Google and Yahoo. There’s a lot of things out of the scope of their understanding, such as frames, visuals such as movies or pictures, and scripting via java. Nor can they peek into parts of sites protected by passwords, or click buttons. Well, that’s what they can’t do. What CAN they do?