New Wordpress Plugin - PingCrawl
I’ve been starting to use a new plugin I helped develop with the coding expertise of Josh Team from Dallas Nightlife Entertainment. It’s called PingCrawl. Its a plugin that helps get your Wordpress blogs deep links automatically on every post.
Plugin Summary
Every time you make a post on your blog it grabs similar posts from other blogs that allow pingbacks using the post tags. It then links to them at the bottom of the post as similar posts. It then executes the pingback on all the posts. You can specify how many posts to do per tag and that many will be done for each tag you use in your posts. Typically it has about an 80% successrate with each pingback and they are legit so the ones that fall into moderation tend to get approved. This creates quite a few deep links for each blog post you make and through time really helps with your link building. Especially for new blogs.
Theory Of Operation
* The plugin will listen to anytime a post is saved, published, updated, etc.
* The plugin on execution time will find all the tags on the post and perform the following per tag:
o Use Google API to check for ( 35 ) results with the tag name.
o With the ( 35 ) results it loops through them and performs the following
+ Does the result have a pingback meta tag?
+ Does the result have trackback somewhere in the source
+ (if yes to both) it stores the pingback xmlrpc location in memory.
+ (if no to either) we skip that record and move to the next.
+ Once their are 5 legit pingable servers we then append their links to the post we currently added.
+ We then retrieve the xmlrpc urls from memory, and execute a pingback.ping against the xmlrpc as defined in the pingback 1.0 spec. (due to the nature of pingbacks and php it is not a 100% guarantee. A lot of dependencies on state, server responses, headers, etc.)
Their are built in features such as caching google’s recordsets per tag, so you don’t have to make request out to google for the same use. And logic to know if you’ve already “PingCrawled” a post then on edit to ignore it, etc w/ a built in polling system.
Installation:
1. Download Plugin
2. Put file in the wp-content/plugins directory of your wordpress installation
3. Login to your blog dashboard
4. Click on Plugins
5. Click on Active to the Right of PingCrawl in the list
6. Make a Post
*Note ( because of the nature of the script any one tag can make as many as 41 HTTP request and storing source code into memory to run regular expressions against. Because of this I would try to limit my tags to no more than 3 (123 HTTP Request). Use more at your own risk.
Warning: This plugin can really slow down the time it takes to make your posts so I would recommend not using more than 3 tags per post. Also we coded in a small link injection which will put a link of mine into the mix about once every 10 posts. They will all be very white hat and clean links so no worries and if you left the code intact I’d consider that a substantial thank you for the plugin.
Download PingCrawl
Screenshot
*The size of the links are entirely customizable. I’d recommend making them very small at the bottom of the post but in this screenshot I made them big so you can see the format better.
Fuck yea, I really needed this for my next project.
Awesome concept Eli. I am going to install this now on a few of my blogs…and see what happens. I love you man. (Figure of speech.)
btw this works really really well on autoblogs
just be sure to slow down your post count.
This is hot shit!
I love you!
Nice one Eli. Good to have some new posts from you. I will try this out this week!
Great plug-in. I’m going to add this to some of my datapresser blogs to try it out. Thanks Eli.
ooo brilliant idea. I never thought of that.
I may crank out a few myself.
This is legit. I’m definitely going to give this a go and don’t mind the link injection, it’s a fair trade.
This is awesome! Never even considered this idea before … brilliant!
Very nice.. going to try it out now.
Great plugin ..thanks for the info
Thats a great post and thanks for the plugin
This is brilliant and generous. Much appreciated.
Wow, thank you for the tool. Looks like the more you post, the more linkbacks you get. So each post is basically 5 linkbacks?
Basically it works like this,
we go out to google and find 35 blogs. We then loop through the blogs and try to determine (the best we can) if the blog is really a blog, and if the blog can perform an XMLRPC pingback. If it can we do the pingback, with a max of 5 per tag.
both the 35 and 5 are easily changeable vars on line 140 and 141
$numberOfBlogsToGetFromGoogle = 35;
$numberOfMaxLinksToPing = 5;
With this you will never have MORE than 5 but could in theory have 0. Hope this makes sense.
Nice little link launder’er you have there
Good stuff… I’ll try’er out…
Sounds like an amazing plug in. Thanks so much for this information.
And glad to see regular posts form you again.:)
guys into gaming the system, living from fear and lack. they think it is the opposite.
Whoa. This is a nice little plugin, Eli. Thanks.
FYI, If anyone needs to remove the link injection antyant wrote instructions for it on his blog
http://www.anty.info/2008/08/06/pingcrawl-injects-links
It’s very simple even for noncoders.
If you need to remove the link injection to get what you need out of the plugin feel free I won’t take it personally.
Hey guys, hope it works well for everyone. Eli had an older version up, so please grab the newer version! haha. Anyways, it was written and assumes PHP5, so if it breaks for anyone let me know! I will port it if the demand is high enough for php4.
Let me know any feedback and suggestions. Also to Antyant and anyone else who wants to remove or alter the injection, by no means do I want to enforce people to automagically inject me and Eli’s links. Hence why I abstracted it out to a method call “injection” you can easily plug & play your own empire’s links their and have great results (in theory). But, if you don’t want to hassle, I wouldn’t mind the extra few links! haha.
Anyways, let me know if their is anything else I can add!
Hey Josh,
Thanks for the new version. I tried that one, but I am still getting this fatal error
Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or ‘}’ in /wp-content/plugins/PingCrawl.php on line 13
any ideas? struggling with it all night.
What version of php?
Hey Raydahz, what version of php are you using? Sorry my script has caused such issues :/
The only reason I would consider removing the link injection is to remove any possible footprints. Other than that I have no problem giving you backlinks for this great tool.
Maybe I will leave the injection on a few whitehat sites of mine, and remove it from the splogs.
Thanks, it will really help my new auto blogging system:)
I’m also getting the fatal error.
Hey Russ,
Mind telling me what version of php you are running?
Am I allowed 2 comments? I just tried this on a blog and it is working great. Donation for adding our links to the plugin
Thanks I have installed it on my new blog which is getting just 20 visitors a day, looking forward to my first spike!
you are probably not planning to release this for drupal, would you?
seriously thinking about going back to wordpress, and skipping 90% of the new features.
almost finished to move my site to drupal, and now this
hmmm its doable
Any takers on this?
Hmm I can take a look at the drupal API for plugin development, this was my first wordpress plugin. If there is a significant user group for Drupal interested I will port it over.. let me know.
if i wasn’t a sucking coder, i would do it by myself
afaik drupal and wordpress use a very similar hook system for plugins.
Was looking @ the drupal 6 API, are you referring to having the same thing happen when you insert a node into the drupal DB?
i’m not sure what you mean. are you describing the hook? nowadays 80% of drupal installs are drupal 5, cause many important modules are not yet ready for 6.
ahh, so you use Drupal 5? Well as soon as I get this damned thing ported to php4 I’ll port a version to php5 for you. But, will have to look into how to tokenize or index the node to find keywords to search for related blogs on.
awesome. i promiss to get the word out for you.
Useful information for making this happen: Drupal 5 does not support pingbacks by default, you need the trackback or pingback module.
http://drupal.org/project/trackback
http://drupal.org/project/pingback
PingCrawl would have to hook into one of those. Shouldn’t be too difficult though.
Again, would love to see this for Drupal 5 - Wordpress is just soooo last year!
My last comment didn’t show up apparently, but I just wanted to point out that a drupal 5 version of this plugin would need to hook into either the Trackback or pingback module. Shouldn’t be too hard, though. Hope it happens!
Drupal user here who would LOVE to see this as a module. Shouldn’t be too difficult to port over, but what do I know?
I am using Drupal 5, btw. I think if it was made for Drupal 5, interested users could take it over and port it to Drupal 6 when it’s a bit more mature.
Please, please, please make this happen!
Would love to see this on Drupal, I have around 70 sites based on drupal and only 3 on wordpress so would definitely like to see a port.
Looking forward to it.
Just a quick question because I’m not used to pingbacks. Would they basically be like reciprocal linking with around 5 sites on each post?
Thanks.
Would love to see a PHP4 version. Any chance?
Yeah Dave, I don’t mind porting it over; but would have to make sure enough people need a php4 version. (not trying to sound harsh) Going to watch on here and my own blog and if the demand is there I will start to work on one. Will need some php4 testers though!
Sent you on a mail.
Cheers.
Dave I made a few minor fixes, email me at joshteam at gmail [dot] com for a php4 version to test out!
Hah, I like the sneaky little backlink
I’m going to make some modifications to generate a tag if a post is made with no tags - do you want to receive any revisions made? I’ll happily make the source available.
David interested in how you are going about this, I was thinking of a way.. but only solution I could come up with is some “psuedo” lucene indexer to try and determine a keyword off of content. To make it look like it wasn’t spam, or a really cheesy of hack of exploding the post on spaces, and making all the words lowercase then seeing the top 1 or 2 words?
Well, it wasn’t going to be insanely clever - currently just based on the title itself, ignoring stopwords.
I have a bespoke keyword stripper that I might try hooking it into, and pop off the top two or three keywords - but my main issue is speed. As noted, it does slow posting down, which doesn’t help for an autoblogger…
I’ll do some tests tonight and see what I can come up with.
Yahoo actually has an API function that’ll return tags related to post text and keyword.
In my WordPress content plugin, I use the following function to pull back a bunch of related tags:
function find_tags($keyword, $content) {
$content=preg_replace('|]*>|',' ',"$keyword\n$content");
$content=preg_replace('|\s{2,}|',' ',$content);
$url = "http://search.yahooapis.com/ContentAnalysisService/V1/termExtraction";
$post = array('appid'=>'YahooDemo', 'context'=>$content, 'query'=>$keyword, 'output'=>'php');
$ch = curl_init();
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$response = curl_exec ($ch);
curl_close ($ch);
$results = unserialize($response);
$tags = @array_merge(explode(',',$tags), $results['ResultSet']['Result']);
@array_walk($tags, create_function('&$value','$value = ucwords(trim($value));'));
$tags = @array_unique($tags);
if(@in_array('',$tags)){
unset($tags[array_search('',$tags)]);
}
return $tags;
}
Unfortunately, it often pulls back too many tags so to efficiently use it with pingcrawl, you’ll need to trim the list somehow.
Oh yeah…for an example and documentation, see: http://developer.yahoo.com/search/content/V1/termExtraction.html
what a plugin - im now going to install in on all my autoblogs and pass it all out, will leave the autolink in in some of the whitehat sites
Sweet. this is a cool piece of kit, off to download and play about with it.
Excellent plugin eli..
thanks..
i get a
Fatal error: Call to undefined function: simplexml_load_string() in pingcrawl.php on line 67
I originally made it for php5+ tried porting it over, but simple xml object (which I use for the rss parsing) is a php5 only object. I will spend some time over the next couple of days working on this. Check here or my blog for updates!
Wonderful idea for a plugin Eli. Got a lot of projects in the air right now, but will give this go soon
This may be a stupid question, but do you see any problem with using this on a “legit” blog?
Vic
The links are in plain sight, on related content, etc. I use it on my white hat blogs and point it to my white hat sites. But maybe Eli would be a better person to consult?
Hi Josh,
I am nearly finished with a plugin that works similar to yours but with a sllightly different purpose and function.
I recommend building a db to rename the plugin each time it is installed.
This will prevent Google from identifying it.
GaryTheScubaGuy
Very cool Gary. On the renamer, I guess I am going to show my true “green” colors here, but how does google see the name of the plugin? That’s only accessed on Word Press admin. Do you mean the “Related Blogs on %keyword%” ?
awesome idea and the plugin works well for me, but… how could i do that “related blogs” block will not be the part of the post? similarly than the “related post” plugin. so, my blog main page page full of “related blogs” stuffs and it not so nice
You could alter pingcrawl.php to add in Wordpress’s ‘more’ tag just before the Related Posts UL.
This way, the related posts snippet should only show when viewing that article, and not on the front page. Similarly, I believe this would also suppress the related links from your RSS feed.
Explain in more detail what you are trying to accomplish. I’ll point you to the place(s) in code to look at.
Can I also have the info how to do this please, the extra content on the article page is fine but it is spioling the flow on the home page a bit
Thanks
Two more questions
Can you run this in conjunction with the DoFollow plugin?
Can it be run as achron to batch update?
Many thanks for a great plugin too!
One other thing - Aren’t trackback links nofollowed? Doesn’t that invalidate the point of this plugin?
Not being snarky, it’s a serious question.
Thanks,
Vic
Not 100% sure on that. I did a few test and didn’t find that, but even if it was true imagine this plugin on a cycle site that used an autoblogger.
Could generate a nice amount of traffic.
ScreenRant.com, it totally depends on the blog. If the blog has a plugin built in to make trackbacks and comments dofollow it will count. The chances are good that you should get at least one good link from the pingbacks, if you’re lucky…you’ll get all dofollow links
I agree, a link is still a link, especially when you didn’t have to pay for it! LOL
Whether it carries juice or not, it still shows up in Yahoo site explorer as a link and that makes the blog more valuable.
AL
Thanks, Josh.
I followed the instructions and tried both this version and Josh team’s and the posts too a while to come up, but there is no new text links at the end of each post. I am a nube so I dont know if i use PHP4 or 5 or what. it is hosted on hostgator if that helps. i wonder what i am doing wrong
sorry guys, its working now! just gotta work out how to change the text size now. is that in the php file? I will take a peek
I answered you here:
http://joshteam.wordpress.com/2008/08/05/pingcrawl-wordpress-plugin/#comment-56
I need it a lot. now that you are content with it, I can use plugin. Thanks for it.
Sounds pretty sweet. I tent to put in a lot of tags. Any way we can have check boxes to select what tags we use it for when w make a post?
This looks really cool. Gonna add it to my WordPress blogs today.
Interesting
shall thank you think
works like a charm. http://www.nowsayit.com
3 of 5. Not bad not bad at all. The other 2 are more than likely pending approval.
very cool plugin. Major lag time a bit when publishing and got a few time outs. Cut it to just one tag and it works beautifully now.
Thanks
I also got the fatal error using php5.
Eric, what’s the error?
It’s the same error as Tonos Moviles below.
I got some erros when i try to publish a post - but this is a great plugin and i;m sure that these bugs will be fixed
I get the following error on Dreamhost with PHP5 when i try to publish a post
Fatal error: Call to a member function asXML() on a non-object in /home/.dulce/iellas/tonosmovil.com.es/wp-content/plugins/pingcrawl.php on line 68
The issue appears to be the use of the file_get_contents function by the pingcrawl class.
Wow! I was just trying to make one myself!
I’m looking forward to a PHP4 version. The idea is great! Thank you!
Nice one, checking it out now.
I got rid of that message by upgrading to php 5
Nice! Can’t wait to share these.
What A nice link injection script… And I was wondering why they were only 80% of links what sticks , lol
Anyways … It is a still good script so I do not care as soon as it does not work like a footprint
This is exactly what I have been looking for thanks!