10 awesome things to do with cURL

by Jean-Baptiste Jung. 93 Comments -

cURL, and its PHP extension libcURL, are tools which can be used to simulate a web browser. In fact, it can for example, submit forms. In this article, I’m going to show you 10 incredible things that you can do using PHP and cURL.

Acknowledgments

New to cURL? If yes, check out the following articles to learn the purposes and basics of cURL/libcurl.

Please note that some of the techniques shown here can be used for “blackhat” methods. The goal of this article is only educationnal, please do not use any of the snippets below for illegal stuff.

1 – Update your Facebook status

Wanna update your facebook status, but don’t want to go to facebook.com, login, and finally being able to update your status? Simply save the following code on your server, define the variables, and voilà!

<?PHP
/*******************************
*	Facebook Status Updater
*	Christian Flickinger
*	http://nexdot.net/blog
*	April 20, 2007
*******************************/

$status = 'YOUR_STATUS';
$first_name = 'YOUR_FIRST_NAME';
$login_email = 'YOUR_LOGIN_EMAIL';
$login_pass = 'YOUR_PASSWORD';

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://login.facebook.com/login.php?m&amp;next=http%3A%2F%2Fm.facebook.com%2Fhome.php');
curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($login_email).'&pass='.urlencode($login_pass).'&login=Login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_exec($ch);

curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_URL, 'http://m.facebook.com/home.php');
$page = curl_exec($ch);

curl_setopt($ch, CURLOPT_POST, 1);
preg_match('/name="post_form_id" value="(.*)" \/>'.ucfirst($first_name).'/', $page, $form_id);
curl_setopt($ch, CURLOPT_POSTFIELDS,'post_form_id='.$form_id[1].'&status='.urlencode($status).'&update=Update');
curl_setopt($ch, CURLOPT_URL, 'http://m.facebook.com/home.php');
curl_exec($ch);
?>

Source: http://codesnippets.joyent.com/posts/show/1204

2 – Get download speed of your webserver

Do you ever wanted to know the exact download speed of your webserver (or any other?) If yes, you’ll love that code. You just have to initialize the $url variable with any resources from the webserver (images, pdf, etc), place the file on your server and point your browser to it. The output will be a full report of download speed.

<?php error_reporting(E_ALL | E_STRICT);
 
// Initialize cURL with given url
$url = 'http://download.bethere.co.uk/images/61859740_3c0c5dbc30_o.jpg';
$ch = curl_init($url);
 
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, 'Sitepoint Examples (thread 581410; http://www.sitepoint.com/forums/showthread.php?t=581410)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
 
set_time_limit(65);
 
$execute = curl_exec($ch);
$info = curl_getinfo($ch);
 
// Time spent downloading, I think
$time = $info['total_time'] 
      - $info['namelookup_time'] 
      - $info['connect_time'] 
      - $info['pretransfer_time'] 
      - $info['starttransfer_time'] 
      - $info['redirect_time'];
 
 
// Echo friendly messages
header('Content-Type: text/plain');
printf("Downloaded %d bytes in %0.4f seconds.\n", $info['size_download'], $time);
printf("Which is %0.4f mbps\n", $info['size_download'] * 8 / $time / 1024 / 1024);
printf("CURL said %0.4f mbps\n", $info['speed_download'] * 8 / 1024 / 1024);
 
echo "\n\ncurl_getinfo() said:\n", str_repeat('-', 31 + strlen($url)), "\n";
foreach ($info as $label => $value)
{
	printf("%-30s %s\n", $label, $value);
}
?>

Source: http://cowburn.info/2008/11/29/download-speed-php-curl

3 – Myspace login using cURL

<?php

function login( $data, $useragent = 'Mozilla 4.01', $proxy = false ) {
    $ch = curl_init();
    $hash = crc32( $data['email'].$data['pass'] );
    $hash = sprintf( "%u", $hash );
    $randnum = $hash.rand( 0, 9999999 );
    if( $proxy ) curl_setopt( $ch, CURLOPT_PROXY, $proxy );
    curl_setopt( $ch, CURLOPT_COOKIEJAR, '/tmp/cookiejar-'.$randnum );
    curl_setopt( $ch, CURLOPT_COOKIEFILE, '/tmp/cookiejar-'.$randnum );
    curl_setopt( $ch, CURLOPT_USERAGENT, $useragent );
    curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, 1 );
    curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
    curl_setopt( $ch, CURLOPT_POST, 0);
    curl_setopt( $ch, CURLOPT_URL, 'http://www.myspace.com' );
    $page = curl_exec( $ch );
    preg_match( '/MyToken=(.+?)"/i', $page, $token );
    if( $token[1] ) {
        curl_setopt( $ch, CURLOPT_URL, 'http://login.myspace.com/index.cfm?fuseaction=login.process&MyToken='.$token[1] );
        curl_setopt( $ch, CURLOPT_REFERER, 'http://www.myspace.com' );
        curl_setopt( $ch, CURLOPT_HTTPHEADER, Array( 'Content-Type: application/x-www-form-urlencoded' ) );
        curl_setopt( $ch, CURLOPT_POST, 1 );
        $postfields = 'NextPage=&email='.urlencode( $data['mail'] ).'&password='.urlencode( $data['pass'] ).'&loginbutton.x=&loginbutton.y=';
        curl_setopt( $ch, CURLOPT_POSTFIELDS, $postfields );
        $page = curl_exec( $ch );
        if( strpos( $page, 'SignOut' ) !== false ) {
                return $randnum;
        }
        else {
            preg_match( '/MyToken=(.+?)"/i', $page, $token );
            preg_match( '/replace\("([^\"]+)"/', $page, $redirpage );
            if( $token[1] ) {
                curl_setopt( $ch, CURLOPT_POST, 0 );
                curl_setopt( $ch, CURLOPT_URL, 'http://home.myspace.com/index.cfm?&fuseaction=user&Mytoken='.$token[1] );
                $page = curl_exec( $ch );
                curl_close( $ch );
                if( strpos( $page, 'SignOut' ) !== false ) {
                    return $randnum;
                }
            }
            elseif( $redirpage[1] ) {
                curl_setopt( $ch, CURLOPT_REFERER, 'http://login.myspace.com/index.cfm?fuseaction=login.process&MyToken='.$token[1] );
                curl_setopt( $ch, CURLOPT_URL, $redirpage[1] );
                curl_setopt( $ch, CURLOPT_POST, 0 );
                $page = curl_exec( $ch );
                curl_close( $ch );
                if( strpos( $page, 'SignOut' ) !== false ) {
                    return $randnum;
                }
            }
        }
    }
    return false;
}
?>

Source: http://www.seo-blackhat.com/article/myspace-login-function-php-curl.html

4 – Publish a post on your WordPress blog, using cURL

I know that most of you enjoy WordPress, so here is a nice “hack” as the ones I regulary publish on my other blog WpRecipes.
This function can post on your WordPress blog. You don’t need to login to your WP dashboard etc.
Though, you must activate the XMLRPC posting option in your WordPress blog. If this option isn’t activated, the code will not be able to insert anything into WordPress database. Another thing, make sure the XMLRPC functions are activated on your php.ini file.

function wpPostXMLRPC($title,$body,$rpcurl,$username,$password,$category,$keywords='',$encoding='UTF-8')
{
    $title = htmlentities($title,ENT_NOQUOTES,$encoding);
    $keywords = htmlentities($keywords,ENT_NOQUOTES,$encoding);
     
    $content = array(
        'title'=>$title,
        'description'=>$body,
        'mt_allow_comments'=>0,  // 1 to allow comments
        'mt_allow_pings'=>0,  // 1 to allow trackbacks
        'post_type'=>'post',
        'mt_keywords'=>$keywords,
        'categories'=>array($category)
    );
    $params = array(0,$username,$password,$content,true);
    $request = xmlrpc_encode_request('metaWeblog.newPost',$params);
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_POSTFIELDS, $request);
    curl_setopt($ch, CURLOPT_URL, $rpcurl);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_TIMEOUT, 1);
    $results = curl_exec($ch);
    curl_close($ch);
    return $results; 
?>

Source: http://porn-sex-viagra-casino-spam.com/coding/poster-automatiquement-sur-wordpress-avec-php/

5 – Test the existence of a given url

I know, it sounds basic. In fact, it is basic, but it is also very useful, especially when you have to work with external resources. 

<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.jellyandcustard.com/");
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
curl_close($ch)
echo $data;
?>

Source: http://www.phpsnippets.info/test-existence-of-a-given-url-with-curl

6 – Post comments on WordPress blogs

In a previous article, I have discussed how spammers spams your WordPress blog. To do so, they simply have to fill the $postfields array with the info they want to display and load the page.
Of course, this code is only for educationnal purposes.

<?php
$postfields = array();
$postfields["action"] = "submit";
$postfields["author"] = "Spammer";
$postfields["email"] = "spammer@spam.com";
$postfields["url"] = "http://www.iamaspammer.com/";
$postfields["comment"] = "I am a stupid spammer.";
$postfields["comment_post_ID"] = "123";
$postfields["_wp_unfiltered_html_comment"] = "0d870b294b";
//Url of the form submission
$url = "http://www.ablogthatdoesntexist.com/blog/suggerer_site.php?action=meta_pass&id_cat=0";
$useragent = "Mozilla/5.0";
$referer = $url; 

//Initialize CURL session
$ch = curl_init($url);
//CURL options
curl_setopt($ch, CURLOPT_POST, 1);
//We post $postfields data
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
//We define an useragent (Mozilla/5.0)
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
//We define a refferer ($url)
curl_setopt($ch, CURLOPT_REFERER, $referer);
//We get the result page in a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//We exits CURL
$result = curl_exec($ch);
curl_close($ch);

//Finally, we display the result
echo $result;
?>

Source: http://www.catswhocode.com/blog/how-spammers-spams-your-blog-comments

7 – Follow your Adsense earnings with an RSS reader

Most bloggers uses Adsense on their blog and (try to) make money with Google. This excellent snippet allows you to follow your Adsense earnings…with a RSS reader! Definitely awesome.
(Script too big to be displayed on the blog, click here to preview)
Source: http://planetozh.com/blog/my-projects/track-adsense-earnings-in-rss-feed/

8 – Get feed subscribers count in full text

If you’re a blogger, you’re probably using the popular FeedBurner service, which allo you to know how many people grabbed your rss feed. Feedburner have a chicklet to proudly display your subscriber count on your blog. I personally like the chicklet’s look, but I heard lots of bloggers complaining about it. happilly, cURL can simply grab the count value and return it to you as a variable so you can display it as you want on your blog.

//get cool feedburner count
$whaturl="https://feedburner.google.com/api/awareness/1.0/GetFeedData?uri=feedburner-id";

//Initialize the Curl session
$ch = curl_init();

//Set curl to return the data instead of printing it to the browser.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

//Set the URL
curl_setopt($ch, CURLOPT_URL, $whaturl);

//Execute the fetch
$data = curl_exec($ch);

//Close the connection
curl_close($ch);
$xml = new SimpleXMLElement($data);
$fb = $xml->feed->entry['circulation'];
//end get cool feedburner count

Source: http://www.hongkiat.com/blog/display-google-feed-subscriber-count-in-text/

9 – Get the content of a webpage into a PHP variable

This is a very basic thing to do with cURL, but with endless possibilities. Once you have a webpage in a PHP variable, you can for example, retrieve a particular information on the page to use on your own website.

<?php
    ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, "example.com");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $output = curl_exec($ch);
    curl_close($ch);     
?>

 

10 – Post to Twitter using PHP and cURL

Twitter is very popular since some time now, and you probably already have an account there. (We have one too) So, what about using cURL to tweet from your server without connectiong to Twitter?

<?php
// Set username and password
$username = 'username';
$password = 'password';
// The message you want to send
$message = 'is twittering from php using curl';
// The twitter API address
$url = 'http://twitter.com/statuses/update.xml';
// Alternative JSON version
// $url = 'http://twitter.com/statuses/update.json';
// Set up and execute the curl process
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "$url");
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_POST, 1);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, "status=$message");
curl_setopt($curl_handle, CURLOPT_USERPWD, "$username:$password");
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
// check for success or failure
if (empty($buffer)) {
    echo 'message';
} else {
    echo 'success';
}
?>

Source: http://morethanseven.net/2007/01/20/posting-to-twitter-using-php/

Comments (93) - Leave yours

  6. Tolga Besiktasli said:

    Thanks for compiling this list, nice overview. When I take the example for testing the existance of a URL, this could be done in a few lines without cURL too using fsockopen. I know this article is about cURL, however I wonder if cURL should be favoured because of performance or other things.

    Cheers,
    Tolga Besiktasli

  7. Jean-Baptiste Jung said:

    @Tolga: You’re right, I think the fsockopen method is more performant. But as you said, this article is all about cURL so I featured the cURL method :)

    @all: This post currently have 88 Diggs. Please, support Cats Who Code by digging this post. Thanks in advance!

  15. tomWarren05 said:

    I really need to learn more about this cURL method.It is a good tool for simulating a user’s actions at a web browser.Thanks for the Excellent guide.

  18. Amit Banerjee said:

    Great List and Wonderful Resources

    About number 10 : Can we tweak it in such a way such that it shows that the tweet came from my website ? I want to show my Website name and URL in The Byline

    Forexample : Change the byline “from Web” and “from tweetdeck” To “from My site”

    Is that Possible Dean ? Would be a lifesaver !

  20. 10 awesome things to do with cURL said:

    [...] 10 awesome things to do with cURL. Related Posts:10 Awesome Free Grunge Fonts | My Ink Blog5 Dinge die ein (Webdesign) Kunde unbedingt wissen sollte!26 neue und beeindruckende Web-AppsVektor Skate DesignInspirierendes Flyerdesign [...]

  22. Catrin W said:

    Wow, those are really interesting things to do with cURL. I really had no idea that those were even possible. Thanks for sharing this very useful infos and tips.

  26. Ed said:

    Hi Jean,

    Regarding #9: Get the content of a webpage into a PHP variable

    How would your approach be to get for example pricing information from a webpage?
    Since you can retrieve particular information from a web page?

    If you can provide more information/sources on that using cURL that would be awesome!

  27. Amy Rich said:

    Thats awesome! I really like the check the speed of your webserver one, post comments on wordpress blog, check the existing of the cUrl. Publish your post on the wordpress blog using cURL. It works really good, i have just tried it. These codes works wonderfully. Its really an useful information.Thanx for sharing! You are making aware more people about this cURl.
    keep up the good work.

  32. links for 2009-07-02 « Mandarine said:

    [...] 10 awesome things to do with cURL cURL, and its PHP extension libcURL, are tools which can be used to simulate a web browser. In fact, it can for example, submit forms. In this article, I’m going to show you 10 incredible things that you can do using PHP and cURL. (tags: webdev tutorial php) [...]

  45. derek nadeau said:

    Thanks for compiling this list, nice overview. When I take the example for testing the existance of a URL, this could be done in a few lines without cURL too using fsockopen.

  47. Alste said:

    I’m afraid the Facebook code won’t work (the urls used with curl are incorrect i believe)

    in fact when i try it out it gives me an error for this line:

    curl_setopt($ch, CURLOPT_POSTFIELDS,’post_form_id=’.$form_id[1].’&status=’.urlencode($status).’&update=Update’);

    there is a problem with post_form_id… facebook now uses a post form ajax/updatestatus.php (relative to home.php) to update status so i think this code needs revision

  48. Alste1 said:

    Ok I got it. I modified the code to get it working.

    I’ve tested it with PHP5 on WAMP server, Facebook status update is successful. Here’s the link to the source code:

    http://curl.hostoi.com/fstatus_curl.php

    enjoy!

  52. Milch said:

    Yes CURL could be very powerful tool. I like and use it a lot. The only bad thing about curl is not so good documentation and examples so this list and ideas are cool.

  55. vijay said:

    thank you all.

    I can change status successfully. Now I want to post perticular thing to profile using php curl. how can it possible?
    please help me.

    thanks in advance.

  60. Robert said:

    Hey there

    For some reason i cant get the facebook one to work, when i run it i check my facebook status and nothing seems to have changed…. any ideas?

    -Robert

  61. andrew said:

    Hi Jean,

    AweSome post! I was wondering how to save the results generated via curl instead of sending it to the browser to show. Finally found the answer using your examples! thanks! this is a great post.

    Regards,
    Andrew

  63. Sirviejo said:

    Really good examples.. i’ve loved them.

    @TUTORIAL CITY You have a simple tutorial here: http://www.arzion.com/internet-business/posts/Using-Curl-to-Post-Data

  64. OSX BOY said:

    6 – Post comments on WordPress blogs

    Would be useful to know how to prevent this in WordPress? I’m guessing looking at the HTTP_REFERER or REMOTE_ADDR wouldn’t work because that can be spoofed?

  71. Abeon said:

    Great tutorial with some nice examples.

    I have been trying to match an image that generates a random key.
    The img tag doesn’t use id or class styling, but the containing table and div does.

    How could I match that?

  73. Abeon said:

    Last comment was messed up….
    Number 9 should be:

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, “example.com”);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $output = curl_exec($ch);
    curl_close($ch);

  75. Ahsan said:

    Hey
    thanks a bunch.
    I was looking for curl examples, you made my day!!!
    I have always used fsockopen to deal with this stuff, i am starting to look into curl now.
    :-)

  76. Rachel said:

    Great post! I have a doubt… In “6 – Post comments on WordPress blogs”, say for example that I want to use an array of URLs instead of sending just one URL. I want the script to iterate the URLs which I will be having in a .txt file. How can I do that? Please help. I am not a lame, stupid spammer. I am asking this for a reasonable purpose!

    Thanks for your time!

  77. 15 регулярных выражений PHP в помощь веб-разработчику | Очередной блог фрилансера said:

    [...] Если вам когда-нибудь требовалось получить все картинки с веб-страницы, этот код должен быть Вы легко сможете создать загрузчик изображений с помощью возможностей cURL [...]

  79. omar said:

    The facebook one doesn’t seem to work anymore. I’m trying to use the code to update my status from a webpage according to content I provide, and I want to do this without messing with the API, as my technical level is probably not up to the task. Anyone has a working version of this?

  83. tonybaldwin said:

    I found this searching for cURL options to post to WP or LJ.
    But, what I was really looking for was how to do these things with cURL from the bash command line, rather than with php scripts.
    In truth, I’ve already got posting to twitter or identi.ca down with cURL, since that’s pretty simple (I use the simple method Pramod comments here), but was looking for how to post to livejournal and wordpress (both using xmlrpc).

    I’ve done all that stuff with tcl/tk scripts and http, and with python and tcllib, but was looking for a way to make some non-gui command line bash scripts with plain old CURL. I don’t really know squat about php or how to run scripts on a remote server. I suppose I could just use tclsh scripts without the tk gui…or try to figure out what php is doing here (there must be some xml library built into php that’s building the body of the xml to send?), but how are the variables set? I don’t see an html form or anything here. While I have html and css skills, I’m no web developer. I can send information from a form with cgi scripts, but mostly with copy/pasted code, and using an html page with a form in it. I don’t see where the form is in your scripts.

  84. You are now listed on FAQPAL said:

    10 awesome things to do with cURL…

    cURL, and its PHP extension libcURL, are tools which can be used to simulate a web browser. In fact, it can for example, submit forms. In this article, I’m going to show you 10 incredible things that you can do using PHP and cURL….

  86. Tera Merrick said:

    Wow that was strange. I just wrote an extremely long comment but after I clicked submit my comment didn’t show up. Grrrr… well I’m not writing all that over again. Anyhow, just wanted to say excellent blog!

  87. James said:

    Great Tips…. Could you explain why this PHP code below will not work. It will not enter the login info. and proceed to editing page. Thank in advance

  88. Paul said:

    I believe this code no longer works. I think everything has to be done through Facebook API. The reason I say this is because the code does not require an Access Key, which Facebook uses for security. But this is a great way to learn curl.

  90. Jeremy said:

    Great collection of what you can do with cURL! However having said that, a few of them are an obvious thing to do!

Leave a Reply

Your email address will not be published. Required fields are marked *

Please respect the following rules: No advertising, no spam, no keyword in name field. Thank you!