Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Image Character Recognition

by booter2
on Dec 04, 2003 at 20:41 UTC ( #312311=perlquestion: print w/ replies, xml ) Need Help??
booter2 has asked for the wisdom of the Perl Monks concerning the following question:

Hi yawl,

I need some help with this problem and hope that someone out there has found a solution or knows of a site that has cracked this. The problem involves trying to automate a login script (in perl) for a secure website that requires the user to enter his/her ID and password, PLUS the text string that is shown in a graphical box. I'm certain that just about everone out there has seen the graphical box I am referring to. It is used by yahoo, amazon, and several other high end sites.

The graphical box consists of a background image and text, both of which change each time the page is refreshed. I am able to obtain the image file name (*.jpg) and therefore download the .jpg file to a pc for processing. What I now need is a program (OCR - optical character recognition) or routine that can automate and process the image file and report back the actual word hidden inside the image, so that I can then feed this to the form. I hope I have explained this clearly enough.

Any ideas? This is a really tough nut to crack!

Comment on Image Character Recognition
Re: Image Character Recognition
by holo (Monk) on Dec 04, 2003 at 20:51 UTC
    This is a really tough nut to crack!

    That's the whole point! It is a security feature used by sites to avoid abuse.

Re: Image Character Recognition
by hardburn (Abbot) on Dec 04, 2003 at 21:06 UTC

    You'll notice that if the developers on those sites have done it right, there are various forms of noise in those images (lines in a grid, pseudo-random placement of pixels, etc.). The noise is there specifically to foul OCR software in order to stop people from doing preciely what you're trying to do.

    The problem with these methods is that they pretty much stop any blind user from using these web sites, but that's another issue.

    ----
    I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
    -- Schemer

    : () { :|:& };:

    Note: All code is untested, unless otherwise stated

      The problem with these methods is that they pretty much stop any blind user from using these web sites, but that's another issue.

      A good way to handle that issue is to also offer a sound byte that reads out the word. This is also hard for automated scripts to bypass, and if a blind person is using a computer you can pretty much guarantee that they have sound.

      I don't know of any websites that actually implement both methods though. I guess the issue with implementation is that the graphic image is easy to randomly generate using software, but the sound byte is much more challenging to generate!

      Anyway, this is getting off topic, so I will stop here...

        I guess the issue with implementation is that the graphic image is easy to randomly generate using software, but the sound byte is much more challenging to generate!

        Not really. Just create a series of files each with it's own audio for a single letter, then string them all together and send the concatenated audio. We have an application that uses a similar technique to generate real-time navigation information for ships.

Re: Image Character Recognition
by elusion (Curate) on Dec 04, 2003 at 21:55 UTC
Re: Image Character Recognition
by Roger (Parson) on Dec 05, 2003 at 05:04 UTC
    Funny I was actually thinking about doing something like this the other day. I will write down what I think is a strategy to do 'intelligent' character recognition. I am quite confident that this technique is able to recognise printed characters in the noisy-ish images generated by sites like PayPal.

    Core Component
    Multi-layer, back propagating neural network. I will probably use the AI::NeuralNet::BackProp module to do this. The neural-net is then pre-trained with the fonts to be recognised.

    Image Processing
    You will definitely need to clean up the image somehow before feeding into the character recognition engine. The pre-processing would involve:
  • color image -> black/white convertion (to simplify the recognition)
  • noise reduction, including lines that run across the image
  • a pixel density count, statistics collection, determine text/character boundary

    Character Recognition Process
  • Input is an array of character bitmaps captured by the pre-processing steps
  • Feed the bitmap into the neural network, and get the best estimate of the character it contains
  • Output the characters recognised

Re: Image Character Recognition
by Coruscate (Sexton) on Dec 05, 2003 at 06:58 UTC

    I would just like to make sure of one point. Have you read the terms of use of the site you are trying to do this to? They put these things out there for a reason: they don't want you to automate tasks on their websites (usually just for registrations or to unlock an "abused account" (too many successive failed logins -> yahoo does this)). Just make sure you're being a "good customer". Don't be going and looking for trouble :)

Re: Image Character Recognition
by EvdB (Deacon) on Dec 05, 2003 at 08:41 UTC
    This should be fairly easy to achieve:
    1. By looking at several images extract a sample of each character and save these image files locally.
    2. Take the images that you want to 'read' and apply a fourier transform on it.
    3. For each character you extracted in 1 do a convolution of its fourier transform with the fourier transform from 2.
    4. If a maxima occurs note its position.
    5. After every charater has been tried list them in the order they appear in the image.
    Voila - you have the letters!

    PS you should check the maths here but I believe that it is roughtly right - long time since I did this at Uni.

    --tidiness is the memory loss of environmental mnemonics

Re: Image Character Recognition
by dragonchild (Archbishop) on Dec 05, 2003 at 13:29 UTC
    Try A little fun with merlyn. jcwren plays around with something similar.

    ------
    We are the carpenters and bricklayers of the Information Age.

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Re: Image Character Recognition
by zentara (Archbishop) on Dec 05, 2003 at 16:31 UTC
    Try ocrad

    It takes a pbm, but it seems to work pretty good, at least for it's examples. :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://312311]
Approved by hardburn
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (17)
As of 2014-10-01 15:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    What is your favourite meta-syntactic variable name?














    Results (29 votes), past polls