Recognizing Search Engine Penalties

by Craig Mazur - Copyright 2004 - All Rights Reserved

June 13, 2004

Has your Web site ever received search engine penalties? Would you know if your site has been penalized? Probably not. The vast majority of non-optimized Web sites are so far back in the rankings that they would never know. Furthermore, a search engine never notifies a site owner of a penalty situation, an they only very rarely respond to requests for information regarding possible penalties.

Almost everything we learn about search engine optimization is the result of experimentation. Search engine algorithms are proprietary in nature, and search engines rarely divulge their inner workings. Several major search engines do provide optimization guidelines to follow, but they tend to be more of a list of things that one should avoid.

Any technique intended to trick or deceive a search engine is called spam. There are a large number of spam techniques commonly in use that do produce short-term positive results, but only until they are recognized by a search engine and penalties are applied. Penalties can range from a reduction in ranking to a complete removal from a search engine's database.

Webmasters can unknowingly create spam situations, and even though there isn't any intended deception, the techniques can be labeled as spam and penalties applied by a search engine. The most common "innocent" spam situations are:

Improper use of redirections. There are a number of ways to create redirections from one Web page to another, or from one Web site to another. Certain techniques, such as the delivery of customized error pages or 301 permanent redirections when the domain name or page names change are recognized by most search engines as legitimate. However, a redirection using other techniques may be viewed as cloaking, which is the ultimate spam offense. Before you apply redirection techniques, be sure you know how to do it in a search engine friendly manner.
Mirror sites. Mirror sites are sometimes created when multiple domain names are directed to a single Web site. The alternate domain names need to resolve into the primary domain name using a 301 permanent redirect. If they don't, you may be creating a mirror site situation. If you find multiple instances of your Web site under different domain names, have created a mirror site. Mirror sites became penalty liabilities when search engines discovered that unethical Webmasters could grab the code from a top ranked Web site and set it up as their own under their domain name.
Repeated submissions. Once a site is picked up by a search engine, you never have to submit that site again. Although there are a lot of companies that sell submission services, they are for the most part useless. Why would you want to submit a site monthly to 10,000 search engines when the top 15 or 20 will drive 99.9% of your traffic? Do you believe that there really are 10,000 search engines? I don't. Search engines are so efficient today that they will find your site without any submissions. If you do feel compelled to submit a new site to the search engines, the only ones that you might want to submit the site to are Google, MSN and Yahoo. Search engines get so deluged with repeated submissions that several threaten to apply a spam penalty to repeat offenders.

A spam penalty can more easily be recognized if a site disappears from search results or is greatly reduced in rank. But not all reductions in rank are the result of spam penalties. In November of 2003, Google made some radical changes to their ranking algorithm that created huge ranking losses for thousands of Web sites.

But if you take algorithm changes out of the picture, any across-the-board loss in positions for a site could be the sign of a penalty.

The situation created with mirror sites is fairly common. Webmasters, designers and developers routinely direct multiple domain names to a single site. This is not a problem unless one or more alternate domain names find their way into the search engine databases. This will only happen if someone actually uses the domain name as a link to the site, which is then picked up by a search engine spider. This can easily happen if an alternate domain name is used for any purpose, including e-mail. When someone sees a domain name, they asume it is the primary domain, these situations therefore need to be avoided.

Here's one situation that we recently encountered. The company has a large, mature site that has multiple domain names pointing to a single site. The problem was initially evident by the lack of presence for their primary domain name, Domain A, in several search engines. Each of the secondary domains was used for e-mail or some other purpose at one time or another, though Domain D was never actively used for anything.

	Domain A	Domain B	Domain C	Domain D
Google	355	2	10	2
MSN	1	1	1	21
Yahoo! Web	1	1	1	251

The numbers represent the total number of Web pages found for each domain name in three major search engines. The chances are very good that this company is experiencing search engine penalties, but the penalties appear to have been applied to different domain names by different search engines. Other than the situation with mirror sites, the site is very clean and the company has never engaged in spam techniques.

This example shows what can happen if you are not careful with the use of multiple domain names. We tracked the use of Domain D to a couple of forums where the company's products were recommended. It appears that at some point someone entered the domain name in their browser to see if anyone was using it, found the company, thought that Domain D was their real domain name because the alternate domain was not set up to resolve into the primary domain, recommended the company's products in a forum or two using Domain D, which was then picked up by MSN and Yahoo who mistakenly thought it was the primary domain. Google appears to be the only search engine out of the three that got it right.

The almost total lack of presence in MSN and Yahoo for the primary domain (Domain A, in use for several years) plus Domain B and Domain C, appears to indicate the application of a penalty. While the site does show up under these domains, it has been limited to a single listing.

Other indications of a worst case spam penalty include the complete removal of a domain name from a search engine's database. This type of extreme penalty is usually reserved for egregious violations, such as cloaking. Other, innocent problems, such as the server being down during multiple attempts to spider a site, have been known to result in temporary eliminations from a database.

While it can sometimes be difficult to determine with any certainty that a penalty was actually applied to a site, or even the type of penalty that may applied to a site, there usually are indications when something is amiss.

Top Rank Solutions is located near Phoenix in Mesa, Arizona, and offers services for customers throughout the United States.