http://www.perlmonks.org?node_id=947954


in reply to Re^2: Web::Magic 0.005
in thread Web::Magic 0.005

The dependencies kill this module... According cpantesters, this modules has a 42% chance of all tests passing. As for Mojo::UserAgent, it supports DOM, CSS selectors, and JSON. http://mojocasts.com/e5.

Replies are listed 'Best First'.
Re^4: Web::Magic 0.005
by tobyink (Canon) on Jan 15, 2012 at 09:32 UTC

    Mojo is quite impressive for what it manages to do without dependencies. But really, say I bundled all the other distributions Web::Magic replies upon, and packaged them with Web::Magic. Then I could say it had no dependencies too. Many of Web::Magic's dependencies are indeed modules that I wrote, so I could have easily chosen to package them all into one distribution.

    But it seems more logical to provide more, smaller, independently testable and independently usable releases. And rely on a tool to keep track of dependencies between them. Perhaps a tool with a good track record of managing dependencies for Perl modules? Perhaps a tool that has been bundled with Perl for well over a decade?

    That 42% (actually 43% according to deps.cpantesters.org today) statistic is very misleading. The site that calculates the statistic acknowledges this.

    In practice the only dependencies of Web::Magic which have worrying fail rates are B::Utils which fails a lot on BSD, but seems reliable on other operating systems, and XML::Feed which has some date formatting test cases that seem to sometimes fail. (An Atom test case which expects a datetime in the floating timezone, receives one in UTC.) A force install on XML::Feed will almost certainly still give you a working copy.

    That said, Mojo's claimed HTML5 credentials are a bit suspect. The following code illustrates that Mojo::DOM's HTML5 conformance is not quite there yet:

    use Modern::Perl; my $html5 = do { local $/ = <DATA> }; { say "HTML::HTML5::Parser:"; use HTML::HTML5::Parser; use XML::LibXML::QuerySelector; my $dom = HTML::HTML5::Parser->new->parse_string($html5); say $dom->querySelector('tbody')->textContent; } say "----"; { say "Mojo::DOM:"; use Mojo::DOM; my $dom = Mojo::DOM->new($html5); $dom->find('tbody')->each(sub{say $_->all_text}); } __DATA__ <!doctype html> <title>Greetings</title> <table><tr><td>Hello World</table> <!-- This is a valid HTML5 document. See http://validator.w3.org/check + -->

    For comparison, try the following in a modern, HTML5-capable browser (tested Firefox 3 and Opera 11):

    <!doctype html> <title>Greetings</title> <table><tr><td>Hello World</table> <script type="application/ecmascript"> window.alert(document.querySelector('tbody').textContent); </script>
      I have no experience with Web::Magic or Mojolicious. But, Let me explain my concerns with the dependencies. I work for a company that requires every module, that will be installed, to be review by the law and security departments. It's a painful process taking months. Any module that has more than a few dependencies I simply won't consider and I prefer to install modules with no dependencies.

        I'm in about the same boat. Thus far, I've been waiting ~2 years to get legal approval to ship a dozen or so modules, though getting approval to use modules internally have come along much faster than that (usually less than a month). I'm undeterred, though, and finally my management stream has gotten annoyed enough that they told me to go ahead without legal approval, told legal that we're going ahead, and they better get the approvals done in time to ship this product.

        If they're trying to make it painful to do the right thing, I just let it slide - I don't let it bother me. I leave that pain to my manager to deal with.

        Once this finally goes through, I'm going to turn around and immediately start to seek approval to ship a private copy of perl itself. Partly as a way to show that it's not phasing me, though mostly because it would be highly beneficial in many ways.

        Don't give up the fight if you have technical reasons to use those modules. If Web::Magic solves problems for you better than alternatives, go for it. I'm not sure it does/would, but if it does, then go get those approvals. At least here, I've seen no evidence suggesting it's the number of modules that is making this take so long, so whether I ask for 12 or 20, it probably doesn't really matter. We're talking about hours of extra work, not weeks.

Re^4: Web::Magic 0.005
by Tux (Abbot) on Jan 17, 2012 at 08:17 UTC

    Here it has 0% chance on success, as it uses modules that depend on modules (that depend on modules ...) that are blocked (forbidden) by company rules/standards.

    So yes, the dependency tree is too heavy.

    $ cpan Web::Magic : : Result: FAIL Failed 5/5 test programs. 2/3 subtests failed. make: *** [test_dynamic] Error 2 TOBYINK/Web-Magic-0.006.tar.gz 9 dependencies missing (HTML::HTML5::Writer,XML::LibXML::QuerySelector +,JSON::JOM,HTML::HTML5::Parser,JSON::JOM::Plugins::Dumper,RDF::RDFa:: +Parser,AnyEvent::HTTP,JSON::JOM::Plugins::JsonPath,common::sense); ad +ditionally test harness failed /usr/bin/make test -- NOT OK

    Enjoy, Have FUN! H.Merijn

      Here it has 0% chance on success, as it uses modules that depend on modules (that depend on modules ...) that are blocked (forbidden) by company rules/standards.

      Well, there's your problem!

      So yes, the dependency tree is too heavy.

      Your genius is unparallelled!

      Here it has 0% chance on success, as it uses modules that depend on modules (that depend on modules ...) that are blocked (forbidden) by company rules/standards.

      Here is has 0% of success, because I do not have Perl, and in fact I live in a subterranean fortress on the third moon orbiting Guargon 5 in the Crab Nebula, which has patchy Internet connectivity (at best).

      </sarcasm>
Re^4: Web::Magic 0.005
by Anonymous Monk on Jan 15, 2012 at 06:42 UTC

    The dependencies kill this module... According cpantesters, this modules has a 42% chance of all tests passing...

    Nice math

    Web-Magic 0.006 ( 75 ALL, 1 FAIL, 1 NA, 73 PASSes )

    invented number %42 chance of all tests passing

    You're a great advocate for mojo