Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Wiki size monitor.

by davido (Cardinal)
on Nov 20, 2004 at 09:16 UTC ( [id://409240]=sourcecode: print w/replies, xml ) Need Help??
Category: PerlMonks Related Scripts
Author/Contact Info davido
Description:

Really only useful to the gods, though possibly of interest to all Cabal: This is a quick way to check to see if any of the wikis are needing rotation.

My first time logging into the Monastery with WWW::Mechanize as myself (instead of just scraping anonymously), and my first time fetching Monastery pages via the xml displaytype. More straightforward than I expected. XML++.

I hope it comes in handy.

Update: Implemented demerphq's suggestion.


use strict;
use warnings;

use WWW::Mechanize;
use XML::Simple;

my $user   = '*****';
my $passwd = '*******';

my %wiki_ids = (    'Cabal'     => 322009,
                    'PMDev'     => 106868,
                    'PMDevToDo' => 222493,
                    'Port'      => 110265,
                    'Editors'   => 74283 ,
                    'SDC'       => 174180,
                    'SDC Plan'  => 240586,
                    'SCD ToDo'  => 237008,
                    'QAEditors' => 133122    );

my $max_wiki_size = 50_000;

my $agent = WWW::Mechanize->new();

$agent->env_proxy();

print "\n--- Wikis ---\n";

# Check each wiki's size.

my $login = "op=login;user=$user;passwd=$passwd;expires=+10y;";

foreach my $wiki ( sort keys %wiki_ids ) {
    $agent->get(  
          'http://www.perlmonks.org/index.pl?'
        . $login
        . "node_id=$wiki_ids{$wiki};"
        . 'displaytype=xml'
        . ';xmlstyle=clean'
    );
    $login = '';
    printf "%-10s (id=%7d): Size= ", $wiki, $wiki_ids{$wiki};
    if ( $agent->success() ) {
        my $hr_xml = XMLin( $agent->content() );
        my $size = length $hr_xml->{data}{field}{content};
        print $size,
              ( $size >= $max_wiki_size ) ? ". Rotate.\n" : ".\n";
    } else {
        print "Unable to fetch.\n"
    }
    sleep 2;
}
Replies are listed 'Best First'.
Re: Wiki size monitor.
by demerphq (Chancellor) on Nov 20, 2004 at 09:56 UTC

    IMO you should add 'xmlstyle=clean' or possibly 'xmlstyle=flat' to the argument list. The former will prevent the data in the node having newlines appended to the end, and the latter will do that and also use a different "flat" key layout that IMO makes things a little more logical.

    ---
    demerphq

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: sourcecode [id://409240]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (3)
As of 2025-06-17 02:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?
    erzuuliAnonymous Monks are no longer allowed to use Super Search, due to an excessive use of this resource by robots.