The Under Appreciated Robots Wordpress Plugin

I have been very busy as of late revamping some sites that have long been needed to be revamped. The ones that I use 100% Wordpress on I just throw in the Yoast Robots Wordpress plugin. It is great for using with any Wordpress site, but has greater use for sites that are already live that need revamping.

I usually take my time with these site rebuilds and will have partial pages going throughout the site. In those cases I simply put a nofollow & noindex on pages that are incomplete or orphaned as the site comes back together. It really is an underrated Wordpress plugin and has multiple uses outside of what I mentioned. Watch the video below for all that you can accomplish with it. Make sure you have it in your arsenal.

How to use the Robots Meta Plugin from Joost de Valk on Vimeo.

Matt Cutts On Google Deindexed Sites

Deindexed sites are common in the affiliate marketing world. This is not just for BANS sites as the common factors are usually site thinness or other major no nos that Google dislikes. While there is really no SURE FIRE way of getting it back in the index other than correcting the obvious mistakes and resubmitting to Google, it may be in your best interest to take your lumps and move on. You could 301 redirect the deindexed domain to a new one, but as Matt Cutts is quoted here:

Can you get hurt by 301′ing a penalized Web site to a new Web site?

Typically not because it could be a case where someone’s trying to Googlebowl you. However, if it’s a site that’s penalized and that Google doesn’t trust, then you may not be getting that much juice transferring over anyway. Sometimes its better to just start truly fresh.

Cutts is basically telling you to move on. Is this what everyone wants to hear? No! Are there situations where you can get a de-indexed site, re-indexed? Sure, I have done it myself, but it is hit and miss and no guarantees as far as a time factor.

So what do you do with all of that content you added after de-indexing? Personally, I would try and get all of the content to evaporate from the major search engines you are listed in for duplicate content sake. You can try to block all bots in the robots.txt and see how long it takes. Slowly add in your content to the new site and build backlinks slowly as well. Most importantly, learn from your past mistakes and do not let it happen again.

Block Internal Pages That Do Not Need To Be Indexed

Privacy policies, contact pages and other pages that are deemed necessary, but not needed to be indexed can be blocked in the robots.txt file. A robots.txt file tells most of the search engines, what you would like and not like them to have access to or index. Most major SERPs follow it's commands and the ones that don't are likely spam crawlers anyways.

More reading on robots.txt

Why would you care if they are blocked or not? You want you link juice to flow to pages that need it and use it for ranking. The less you split the juice the better. While not overly crucial, every bit helps. There is no sense to pass PR or link juice to those type of pages as they are their mostly to keep advertisers and affiliate programs happy and for on site surfers to read if they choose to.

You can use this robots.txt generator, to generate a robots.txt file for pages you want blocked from the SERPS and save that link juice for pages that need it and need more of it.

Robots.txt Wordpress Plugin

Forgot to mention a great robots.txt Wordpress plugin I came across recently. This is great for tweaking your robots file for your Wordpress BANS templates and should be used when Wordpress is installed on the root. Also here is a link to some frequently asked questions about robots text files.

eBay EPN Bot Clicks

I wanted to let everyone know that eBay EPN has confirmed that they still are having problems with bot clicks on affiliate links. This is not really a surprise and really not that big of a deal in my opinion. You should be watching your server stats for the most part anyway. Do not get so caught up in the clicks as much as monitoring where and how many uniques are getting into your niche store. If you have an optimized page and are bringing in relevant traffic or organic traffic, your store WILL convert. I have many, many stores across several niches and they all convert. Some way more than others. Let me also say that it is not in your best interest to read the EPN forum everyday. There are many conspiracy theories posted everyday and if you start reading them and other threads, you will start to believe them. Plain and simple ... If you think EPN or BANS does not work right, I highly suggest you check your link structure and make sure links are formatted correctly. If you are convinced you are structured correctly and still think EPN and BANS does not work correct, I highly suggest you move on to something else. I hate to be so harsh, but many of you need it. Drive traffic and build backlinks. Don't get caught up in everything else.

Watching your Niche Store Stats

I logged into one of my Build A Niche Store stats area on Thursday to find that one IP was eating 12 GB of bandwidth. After tracking down the IP and offending site, I merely blocked that IP (for now anyway). You should constantly be checking your stats on a regular basis to find out where you traffic is coming from, what search phrases were used to find your niche store and also to be on a lookout for bandwidth theifs. You really can learn about your site performance from the inside out. I prefer viewing using the awstats version in the cPanel with Hostgator.

Robots.txt and your niche store

Robots.txt files can play a big role in what is allowed to be searched and documented by the search engines. I see many BANS Stores that have no robot.txt file or improper file construction. Instead of giving a regurgitated lesson. I am going to link you to one of the better tutorials by GNC Web creations! This should clear up any confusion you may have on what to deny and what to allow.