The essence of SEO is an understanding of how search engines operate…

Fundamentally the search engine is trying to deliver the most relevant match to the end user’s search phrase. As I wrote recently, this led to the original SEO tactic of keyword packing both on the page itself and in meta tags. This use of meta tags was specifically designed to allow the web author to talk directly to the search engine to highlight the particular content focus of each page.

Sensible use of keywords on the page is still essential but the use of the keywords meta tag in particular has been totally discredited. Basically it was too easy and too tempting to try and game the system. To remove the machine-generated keyword-packed spam that was destroying the quality of their results, the search engines had to ignore or even penalize the use of the keywords meta tag. Nowadays all search engines (apart from Inktomi) concentrate solely on the content that the end user sees. This was a necessary step, but it removed what should have been a legitimate and useful way for author and search engine to work together to deliver the most relevant content.

Today though, keywords are back with a vengeance in the form of Web 2.0 tagging.

Use any content management system (cms) such as Wordpress, Joomla and Drupal and all content contributors will be encouraged to tag each item that they add with relevant keywords (the same is true with other Web 2.0 systems such as Flickr and Technorati). These tags are then used to produce a major layer of navigation for the site. They are also used to generate summary pages of related teasers and can be used for other purposes such as automatically generating your site’s URL path structure and breadcrumbs.

As I see it, there are two essential differences between these Web 2.0 tags compared to the old keyword meta tags. First, the tag is no longer hidden but there openly on the page. Crucially this means that it will definitely be added to the search engine index. It also means that it’s not so tempting or easy to game the system with irrelevant keywords so the tags have more credibility.

Secondly each tag appears as a link. Again this is likely to boost the importance/credibility of your keyword as anchor text rates more highly than vanilla text. More to the point, it’s important to realise that the web isn’t built on text but rather hypertext. Google was the first search engine to fully appreciate this and to intelligently index not just content but links so building up a contextual picture of your site as a whole. Web 2.0 tagging is perfect in this regard as it allows Google to understand the content focus of your site as well as your page.

Put these benefits together with those of keyword-based url and breadcrumb structure and well-connected keyword-based summary pages and I think that you largely have the answer to the common question: “how come this guy’s personal blog is outscoring my professional site?”. It also helps explain why adding a blog to a site is generally recognized as a sure-fire way to boost traffic though this should only be seen as a halfway house to a full-blown, Web 2.0, tag-based cms site.

Web 1.0 / SEO 1.0 used to be a secretive oppositional battle of wits between search engine and optimizer. Web 2.0 / SEO 2.0 sees a fundamental shift towards a more open and co-operative user-oriented model. Web 2.0 tagging is the perfect example of this new co-operation in action with author/optimizer and search engine working seamlessly together to deliver the most relevant content to the end user both on site and via search.