ddhr.org
WordPress duplicate content (2) Thursday, Mar 1, 2007 4:24 pm

I've been concerned for a little while now with WordPress's default functionality of creating duplicate content.  Each post is located at its permalink (/2007/03/01/post-title/), its date (/2007/, /2007/03/, /2007/03/01/), its category (/topic/wordpress/), and its page with respect to the homepage (/page/3/).  This might be a good thing, but to me it seems like it makes things more difficult.  If somebody finds my site through a search engine, they might be directed to an old location that no longer has what they were looking for.  For example, if something like /topic/wordpress/page/2/ came up in Google's search results, it might contain old data seeing that the second page of the WordPress category will inevitably change over time (since everything is sorted in reverse chronological order).  The post you're currently reading will eventually move from the first page of the WordPress category, and it'll eventually continue to move on from there as I put new posts in the category. 

The solution to this is with a simple plugin called WordPress Duplicate Content Cure.  If the current item that's requested is a single post, a page, or the home page and it's not paged (i.e. /page/3/), the plugin tells search engine robots to index it.  If any of those conditions aren't met, it tells the robots to not index it.  Simple as that. 

We'll see how this works out in a few weeks or months.  If it has any negative affect on my Page Rank, I'll go back to the old way. 

xt commerce blog - Freier Holger Sunday, Mar 25, 2007 7:55 am

here my robots.txt to avoid dc

User-agent: *
Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$

Dave Sunday, Apr 1, 2007 5:20 pm

Yeah that works too.  It's essentially the same thing as all the if statements in the plugin.

About (more)
Hi, my name is Dave Hosier, and this website is where I write my unfounded opinions on trivial matters. Feel free to look around, but please refrain from reading anything.

contact | code | pictures | feed

Discussion (feed)
Rus on Internet echo chamber: The latter also subjects the...
Dave on Internet echo chamber: I read your comments, so...
Rus on Internet echo chamber: If he wasn't feeling so...
Rus on Where grass grows: Isn't there one house up...
Wendy on Where grass grows: Ehem...I suggested a garden in...
Rus on Favre watch: That last line is brilliant.
Dave on Where grass grows: Good point.  It wouldn't take...
Rus on Where grass grows: I feel your pain, but...
Viv on WinXP .mov thumbnails: Thank you, this worked for...
Dave on Belief and behavior: I would say he believed...