What Search Engine are Really Seeing When They Spider Your Site


Many things need to be taken into consideration when analyzing a site to make it fully optimized to rank well. One thing that is often looked past is what the site looks like to search engine spiders. Are they seeing the same information as you are seeing on the screen? If not, they may be confused about what you are offering and thus could have an impact on your rankings.

Flash and Ajax are two prime examples of programs that spiders can’t read. Sure, Flash has the ability to make a site look great but all that content won’t do you any good when you are trying to rank for the keywords located on that page. All the spiders will see is a blank page.
Luckily, there are some programs available to test pages on your site to make sure all the data is shown properly. Both googlebot spoofer and SEO Browser are great tools.

Let’s look an example:

Here’s a screen shot of the well-known chain restaurant, Chili’s. Their homepage is filled with information about specials going on, changes to their restaurants, delicious menus – pretty much all the information that a restaurant would want to market on their website.


However, here is the information that the spiders are gathering from their homepage:

spider view

Not much, right?

Possible Solutions:

When a redesign is not an option and your site is entirely in Flash then it’s possible to create a couple of HTML pages that have content based on the keywords you want to target for your site. These pages could be linked from the footer of your site or located in your HTML sitemap. At least this would provide the spiders with important information about what your site is about and also create a good landing page for visitors who are searching for your services.

But above all, don’t neglect search engine spiders! They determine where your site will rank in the result pages so make sure they are also seeing the quality content your site provides.

Lindsay Atkinson is an SEO analyst for a Chicago internet marketing firm called Digital Third Coast. Besides devoting her days to optimizing websites, Lindsay also spends time volunteering in her community and trying to teach her dog new tricks.


Leave a reply