Image Analysis

[fluffyvoidwarrior]

Hi Monks, I have a thorny industrial control problem. I need to analyse an endless sequence of jpeg files produced by CCTV. They are of a moving belt of stuff that may contain foreign bodies. I need to cause an alarm when such foreign material is detected. Optical detection is the only option. The foreign material is man made and has distinct edges but is otherwise indistinguishable from the "good" material. I have enhanced these edges with Perl Magick and Edge Detect. In theory this will work because the resulting pics clearly show the guilty objects distinct from the background noise. The problem is that I don't know how to analyse the pics programatically in order to say "This pic contains man made objects". Basically, how do I programatically analyse a jpeg to determine if it contains lines or just random visual noise? Thanks for your ideas.

[raybies]

I don't envy this task. Image recognition isn't a trivial task. When i was in college I did some work with Neural Networks. The trick was to represent the data in some numeric/data means and then train the neural net by showing it a whole lot of solutions to the problem. The work you've done to create/enhance edges is definitely a step in th right direction. Then using a Neural Net, you'd have hundreds of valid images and images that were "rejects" and "Valid" and eventually you could train the neural net to pick the right images. There's a whole science behind how to setup such a network, and what is right for which situations.

I doubt there's an API or module that simply does what you want, that's anything other than experimental, but it's been ages since I've looked at the technology. There are sw tools for image recognition based upon Neural networks, and doing a quick google search for Perl Neural Networks yields a few links (looks like there's even a perl module). That might be one direction to investigate, but you've probably got a lot of experimentation yet to do. Good luck.

[Limbic~Region]

fluffyvoidwarrior,
I had contemplated writing software once to analyze security footage to determine if there was anything worth a human looking at. The idea was that a stationary camera recording a room with no activity should really just be a series of nearly identical still images. If any image looked different enough from the others it meant something came into view of the camera.

I don't know if a similar technique would work here without seeing samples both of the "ok" and the "foreign" shots.

[mjscott2702]

If you could translate the JPEG images to something that is more X,Y,Value based (where the Value is some numerical number, grayscale values are ideal for this), it should be possible to identify "edges" which correspond to step-changes between adjacent "pixels". This is analaguous to "digital differentiation" - large deltas between 2 pixels, in the X or Y direction correspond to edges.

Without more info on the nature of the actual images, this might be a way of identifying images with a larger than normal edge count, without becoming embroiled in the neural network approach, and all that involves - training the network etc.

Pretty sure there must be some modules out there to convert from JPEG to "XYZ" formats, hopefully more knowledgeable monks out there could provide some direction there?

[AnomalousMonk]

Don't know if this will be of use, but I was casually associated several years ago with Dr. Richard E. Haskell, at Oakland U. in Michigan, who was developing feature extraction techniques. See his publications. (Google: 'richard haskell feature extraction')

AFAIR, his approach depended on analysing only two scan lines at a time for feature 'training' and recognition, so the need for computational resources was minimal, and even a simple black-and-white line-scan camera over a moving conveyor belt would be sufficient; recognition could operate in real time. The feature extractors produced were translationally and aspect ratio invariant, but not rotationally invariant; however, rotational invariance could be simulated by 'learning' an object in a whole bunch of different rotational presentations.

I guess the idea for your app would be to learn to recognize 'hard edges' and a 'proper' object, then anything with hard edges that is not coincident with a proper object is 'foreign'. Or something...

[Xilman]

The age-old approach for finding straight lines (and other easily parameterizable features) in an image is to use the Hough Transform. There is a good article on the HT in Wikipedia.

I've seen the HT used on extremely noisy images with amazing effect. For instance, laser ranging of corner-cube bearing satellites can find the distance to the satellite to a few millimetres. The signal to noise ratio at the detector is rather less than one in a million (the one being a reflected photon; the million being the sky brightness) yet because all the signal falls on a straight line in a plot of photon arrival time against distance along the satellite track, the great majority of the noise is rejected.

Yes, in a previous life I worked in a machine vision lab. With Mike Brady at Oxford, to be precise.

Paul

[scorpio17]

Maybe you could simply force the images to black and white, then count the number of black pixels in each image - images with more black pixels have foreign matter?

For a more complex case, that may be too simple, but generating a histogram based on pixel color might be one way to map each image into some form of numerical data for further comparison.

[fluffyvoidwarrior]

Yes I'd considered that approach. The trouble is that the level of background noise can vary quite considerabley (but gradually). So I considered keeping a running average of quiescent noise and comparing each sample frame to this variable baseline. Instant changes are a good indicator. Currently I'm thinking along the lines of a spam filter with several tests returning a score and when they add up to a given threshold level it is considered a "hit". The occasional false positive isn't a problem but operators will simply ignore a frequent false warning alarm

[zentara]

In an odd synchonicity, I was just looking at the docs for Motion homepage. An interesting example is at motion example

The real tool that comes to mind with motion, regarding your problem, is a setup function called "tuning motion" which has graphics at tuning motion . Look at the noise level setting. It is possible to filter out increasing levels of noise, leaving only pure motion behind.

It sounds like you have an exciting project going there. It ought to be easy to separate video noise from actual motion with good cameras and a bit of Perl. By tracking motion, random noise events can be removed, because the noise won't move....they are sporadic. Only objects will be trackable.

[fluffyvoidwarrior]

That seems very interesting. However the objects are moving within a moving stream of sludge. Everything moves. Also, when I referred to noise I didn't mean video noise, I meant random background artefacts caused by the sludge flow. Everything is coated in sludge so I can't use colour, everything is the same temp so I can't use IR, nothing is metallic so I can't use electromagnetic detectors. The only option that is plainly visible is hard edges within a rough surfaced undulating flow of gunge. Unfortunately I can't show you all what I'm working with - my client would go ape and consider it a total infringement of confidentiality. So I can only describe the problem.

[zentara]

The only option that is plainly visible is hard edges within a rough surfaced undulating flow of gunge.

It seems then that you should first concentrate on edge-detection, see the difference between two colors, and how to describe a color. You might use some sort of Fourier analysis to filter out everything except for the highfreq ( sharp edges ). See ImageMagick Fourier Edge Detection and Gimp -sharpening and Gimp-sketch-effect

Work out a method to put those edges into a set, which determines one entity, probably by some intersection of curves/lines. Then make a quick center-of-mass computation for each entity, then watch the center-of-mass motion.

Another idea, is to put a small IR heater on the flow, then watch the stuff on an IR camera, and rely on the heat absorption coefficients of the various objects to reveal themselves.

Of course, all this depends on how fast the conveyor is moving, and whether Perl's computational power can keep up. If the ImageMagick Fourier analysis seems to give good results, you could switch to using PDL, which will give you the speed of Fortran.... very fast....even though it's controlled from Perl. PDL uses some image formats that are specifically designed for image analysis of outer space photos. If I were you, I would ask this question on the PDL maillist. They may have already a solution for your problem within PDL, and they are very helpful.

[BrowserUk]

• By how many pixels do things in successive images move?
• Is the frame-rate consistent with the conveyor rate?
• Is the movement linear in an ordinal direction? (Left to right or top to bottom.)
• How good is the registration of successive images? (ie. is the camera rigidly fixed or wobbly?)
• How long between frames and how much processing power is available to process the images?

[fluffyvoidwarrior]

1. By how many pixels do things in successive images move?
The objects I want to detect - about 5%. The background slop - about 60%

2. Is the frame-rate consistent with the conveyor rate?
No

3. Is the movement linear in an ordinal direction? (Left to right or top to bottom.)
Mostly. The sludge is random and linear in the direction of flow. The objects have some limited random motion and travel fairly reliably in the direction of flow.

4. How good is the registration of successive images? (ie. is the camera rigidly fixed or wobbly?)
In theory good. In practice likely to wobble.

5. How long between frames and how much processing power is available to process the images?
Frames can be delivered as discrete jpegs at 5 per sec or as a digital cctv video stream. Processing power of an ordinary desktop PC.

[BrowserUk]

about 5%.

5% of what? (Ie. what size are the images? Also how many bits per pixel.

5% +/- how much?

2. Is the frame-rate consistent with the conveyor rate? No.

I think this may be the show stopper for what I was thinking about.

Processing power of an ordinary desktop PC

With or without a GPU and libraries?