Realtime get with LWP or Mechanize

[lorn]

Hi Monks!

I want to do a "realtime" get, LWP or Mechanize support this?

I dont want to wait ->get to return the all data, every byte they get, i want to see/parser something like what 'lynx -dump' do

With a filehandle interface perhaps? :) /me just dreaming...

while(my $line =~ $lwp->get("www.foo.com") ){
   #parser $line
}

Does anyone know anything like that? or how i can do that using another module?

[weismat]

There is LWP::Coro which allows non-blocking LWP.
Could you elaborate why you want to do this as I would expect that usually the time spend on the application server for serving the http request will be longer than the time needed for network communication and the parsing will get more complex when you process incomplete messages - especially fif the parser needs to be really robust.
If you want to speed up things, look into threads with multiple LWP instances running requests in paralle.

[lorn]

I work with a proprietary Java CMS ( Fatwire ) that have a 'online' log manager by log4j, for access this log i need to access a admin interface, and see the log in the browser, when the log is bigger the browser crash :/

So i'm thinking to create a http://github.com/lorn/remote-tail that, will do this for me, and show me the log in a shell console, becasue that i need the 'realtime' get

Thanks for your help :)

[Anonymous Monk]

Wouldn't it be easier to patch Fatwire to take a param(numofbytes) and use tell to only show the last numofbytes from the log?

[Anonymous Monk]

See LWP::UserAgent manual entry for get method, esp :content_cb     => \&callback

content_cb site:perlmonks.org