http://www.perlmonks.org?node_id=656877

sadarax has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to create a script that will automatically upload a file to www.divshare.com using my account name. I plan to have files upload one after another.

I think I can do this using WWW::Mechanize

My problem is that I do not know how to make it use my account name and password. Any ideas? I am sure some cookies may be involved, but I'm not sure.

Replies are listed 'Best First'.
Re: Automatic Uploader Script
by moritz (Cardinal) on Dec 13, 2007 at 18:40 UTC
    divshare has an API that allows file upload. Use that.

    It's documented, and you don't have to mess with web scraping and http cookies.

      Thanks. I did not know they had an API. Unfortunately it looks like there is no API for perl, hence my problem.

      I only know Perl and C/C++. I am not planning on learning PHP or something else just so I could write the script.

        There may be no API for perl yet, but they give detailed information about developing a new API. This should be enough for you to build the subset of the API you need (login, upload request, and an upload form). Alternatively, why not build the whole API and put it on CPAN for everyone to enjoy? The python example looked like about 200 lines of code.
        The API is language independent. It uses XML (afaict), which can be generated by any programming languages.

        It has no Perl implementation yet, but that should be easier to write than to start from WWW::Mechanize.

        So go ahead and create a good module, and upload it to CPAN ;-)

Re: Automatic Uploader Script
by moklevat (Priest) on Dec 13, 2007 at 22:48 UTC
    In the "put up or shut up" department, I needed some procrastination time, was looking for an excuse to do something with LWP::UserAgent and XML::Simple, and thought that I might test my assertion that the published API would make it possible to accomplish your task.

    This works for logging in and requesting an upload ticket, and you will need to enter your particulars (email address and password, api key, and secret key). You will also need to develop the upload form and deal with the site's redirection scheme for a complete solution.

    I welcome any and all suggestions for improvements!

    #!/usr/bin/perl use warnings; use strict; use LWP::UserAgent; use XML::Simple qw(:strict); use Digest::MD5 qw (md5_hex); my $api_key = ''; my $secret = ''; my $email = ''; my $password = ''; my $api_url = 'http://www.divshare.com/api/'; my $user_agent = LWP::UserAgent->new; $user_agent ->timeout(10); ## Login to Divshare and get an API Session key my $api_session_key = login($user_agent, $api_key, $email, $password); ## Using the API Session Key compute the API Signature using the "secr +et" key my $api_sig = gen_api_sig($secret, $api_session_key); ## Request an upload ticket my $upload_ticket = get_upload_ticket($user_agent, $api_key, $api_ses +sion_key, $api_sig); print "Your upload ticket is $upload_ticket\n"; ## Logout my $logout = logout($user_agent, $api_key, $api_session_key, $api_sig) +; sub login { my ($user_agent,$api_key,$email,$password)=@_; my $login_response = $user_agent->post($api_url, { 'method' => "login", 'api_key' => "$api_key", 'user_email' => "$email", 'user_password'=> "$password" }); my $api_session_xml= $login_response -> content; my $api_session_key= XMLin($api_session_xml, forcearray=>1, keyatt +r=>[] )->{api_session_key}; return $api_session_key->[0]; } sub gen_api_sig { my ($secret,$api_session_key) = @_; my $build_string = $secret.$api_session_key; my $api_sig = md5_hex($build_string); return $api_sig; } sub get_upload_ticket { my ($user_agent, $api_key, $api_session_key, $api_sig) = @_; my $upload_ticket_response = $user_agent->post($api_url, { 'method' => "get_upload_ticke +t", 'api_key' => "$api_key", 'api_session_key' => "$api_sess +ion_key", 'api_sig' => "$api_sig" }); my $upload_ticket_xml = $upload_ticket_response -> content; my $upload_ticket_result = XMLin($upload_ticket_xml, forcearray=>1 +, keyattr=>[] ) ->{upload_ticket}; return $upload_ticket_result->[0]; } sub logout { my ($user_agent, $api_key, $api_session_key, $api_sig) = @_; my $logout_response = $user_agent->post($api_url, { 'method' => "logout", 'api_key' => "$api_key", 'api_session_key' => "$api_session_key", 'api_sig' => "$api_sig" }); my $logout_xml = $logout_response -> content; my $logout_result = XMLin($logout_xml, forcearray=>1, keyattr=>[] +) ->{logged_out}; return $logout_result->[0]; }
      Wow Moklevat, that is a really cool. You definitely 'put up' the code.

      I will certainly be using this, and working on getting that upload function working. If all goes well, I may try to spin this into a module like everyone suggested and throw it up on CPAN. That is of course, if you don't mind me using your code.

      I'm still learning Modules, so it may take me a while though.

        I, moklevat, release the code for use without restriction and without warranty of merchantability for any purpose (or whatever).
Re: Automatic Uploader Script
by olus (Curate) on Dec 13, 2007 at 19:58 UTC
    Hi sadarax

    I downloaded the API for PHP and look into the source.
    It seems that all you need to do is generate an API key and secret key in your DivShare welcome page.
    With those in hand you can use LWP (or WWW::Mechanize, but I don't know this module) to post a request to:
    http://www.divshare.com/api/?method=login&api_key=API-KEY&user_email=Y +OUR_EMAIL&user_password=YOUR_PASSORD
    The server will respond with an XML that will have your SESSION_KEY.
    Then, from the API documentation, you must request for an UPLOAD_TICKET. Another post:
    http://www.divshare.com/api/?method=get_upload_ticket&api_key=API-KEY& +api_session_key=SESSION_KEY&api_sig=
    Parse the XML for the upload ticket and build the form as shown in http://www.divshare.com/integrate/api#uploading