Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^4: Mechanize Firefox text Method

by halweitz (Novice)
on May 05, 2013 at 16:28 UTC ( #1032130=note: print w/ replies, xml ) Need Help??


in reply to Re^3: Mechanize Firefox text Method
in thread Mechanize Firefox text Method

Thanks again for your reply. Let me clarify a bit. Since I can read the documents in the browser I know they contain only text so OCR is not an issue. All the documents follow a similar set of templates but the content changes for each. I have viewed hundreds of these and any document that does not conform will be skipped.

Your comments on downloading and then using a pdftotext tool on the local file is inline with my current thinking as long as it can be scripted and run without intervention. Are there any other suggestions I should examine?


Comment on Re^4: Mechanize Firefox text Method
Re^5: Mechanize Firefox text Method
by afoken (Parson) on May 05, 2013 at 18:27 UTC
    Let me clarify a bit. Since I can read the documents in the browser I know they contain only text so OCR is not an issue.

    I think we have a little communication problem: Sure you can read text displayed in Firefox, because it was rendered from something like <html><body><h1>Hello</h1>. But you can also read text displayed in Firefox that was rendered from something like <html><body><img src="http://www.example.com/pics/hello.gif" alt="">. Your computer can't, at least not as easy as you. To extract the text from the latter, you need OCR.

    All the documents follow a similar set of templates but the content changes for each.

    Any chance to get access to the data before the template engine creates the PDF? Perhaps as XML, JSON, CSV or even HTML?

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Hi Alexander. Thanks for the reply. Instead of using the content method of Mechanize::Firefox I used CAM::PDF to extract the text from each PDF page and then search the text for my filter phrases. The only wrinkle is that I have to save the URL in a file and then load the file into CAM::PDF. When the script ends I get multiple clean up errors about removing a call back on an undefined value. This does not affect the script in any way.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1032130]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (6)
As of 2014-08-30 01:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (290 votes), past polls