Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much

Yahoo Content Analyzer

by Your Mother (Chancellor)
on Jul 20, 2014 at 20:34 UTC ( #1094394=CUFP: print w/replies, xml ) Need Help??

Inspired by How to transmit text to Yahoo Content Analysis. Not sure how complete or correct it is, just threw it together for fun. Seems to work and Iíll make amendments as necessary or sanely suggested.

Requires: strictures, LWP::UserAgent, Getopt::Long, Pod::Usage, Path::Tiny.

#!/usr/bin/env perl use 5.010; use strictures; no warnings "uninitialized"; use LWP::UserAgent; use Getopt::Long; use Pod::Usage; use open qw( :encoding(UTF-8) :std ); use Path::Tiny; # use XML::LibXML; # For expansion... or XML::Rabbit my $service = ""; my %opt = ( text => undef, url => undef, max => 100 ); # These are, luckily, false by default for Yahoo, so we only care abou +t true. my %boolean = map {; $_ => 1 } qw/ related_entities show_metadata enable_categorizer /; # What we compose to query, e.g. not "verbose" or "file." my %sql = ( %opt, %boolean ); my $ok = GetOptions( \%opt, "text=s", "file=s", "url=s", "max=i", "verbose", "help", keys %boolean ); pod2usage( -verbose => 0, -exitval => 1, -message => "Options were not recognized." ) unless $ok; pod2usage( -verbose => 2 ) if $opt{help}; pod2usage( -verbose => 0, -exitval => 1, -message => "One of these, at most, allowed: text, url, fil +e." ) if 1 < grep defined, @opt{qw/ text url file /}; # Only one, text|file, is allowed by Getopt::Long. $opt{text} ||= path($opt{file})->slurp if $opt{file}; unless ( $opt{url} || $opt{text} ) # Accept from STDIN. { say "Type away. ^D to execute (on *nix anyway)."; chomp( my @input = <> ); $opt{text} = join " ", @input; die "Give some input!\n" unless $opt{text} =~ /\w/; } my @where; for my $key ( keys %opt ) { next unless defined $opt{$key} and exists $sql{$key}; $opt{$key} = "true" if $boolean{$key}; $opt{$key} =~ s/([\\"'\0])/\\$1/g; push @where, sprintf "%s = '%s'", $key, $opt{$key}; } my $q = sprintf "SELECT * FROM contentanalysis.analyze WHERE %s", join " AND ", @where; say "SQL >> $q\n" if $opt{verbose}; my $ua = LWP::UserAgent->new; my $response = $ua->post( $service, [ q => $q ] ); say $response->request->as_string if $opt{verbose}; say $opt{verbose} ? $response->as_string : $response->decoded_content(); exit ! $response->is_success; __END__ =pod =encoding utf8 =head1 Name yahoo-content-analyzer - command-line to query it. =head1 Synopsis yahoo-content-analyzer -text "Perl is a programming language." -text "{command line string}" -file (slurp and submit as text) -url -max [100 is default] -related_entities -show_metadata -enable_categorizer -verbose -help =head1 Description L<> =head1 Code Repository L<> =head1 See Also L<>. =head1 Author and License Your Mother, L<>. You may redistribute and modify this code under the same terms as Perl itself. =head1 Disclaimer of Warranty No warranty. No means no. =cut


  • Removed URI, only first draft used it.