Using ImageMagick to process fingerprint photos

[pipitas]

The background for this question are the following (and some related) news stories, which made headlines world wide in recent days:

• http://www.geek.com/apple/bypassing-tou ... o-1612511/
• http://world.news-round.com/secretary-o ... he-leyens/
• http://world.news-round.com/hacker-copy ... r-leyen-2/
• http://www.unlockpwd.com/31c3-they-repr ... f-defence/
• http://www.saudigazette.com.sa/index.cf ... 1231229025
• http://panteres.com/2014/12/28/31c3-hac ... int-fuses/

The "off-the-shelf"-software mentioned in the articles which was used for this hack is a commercial offering called VeriFinger costing almost 400.- EUR (see also http://download.neurotechnology.com/Ver ... -04-17.pdf).

The same thing should be possible with ImageMagick, I'm pretty sure. Can we proof it? Can we help to make IT more safe by showing that the idea biometric authentication schemes are br0ken in many ways?

Look at the fingerprint photo below:

The source of this photo is this website: http://www.pbs.org/wgbh/nova/next/tech/ ... be-hacked/

My question is: how can we, with ImageMagick, convert this photo to a black+white only representation of a fingerprint? I guess a simple, single command line will not work, because:

• different areas of the image need different image setting parameters
• there is a special challenge in that the embossments are very "shiny" because of skin fat in some areas, and rather dark in the shadows
• the photo is taken from an angle and the resulting image should probably account somewhat for some additional rotation ins space.

The resulting image should look somewhat like this (which obviously was not derived from the above one):


I think a script may be successful if it knows to apply a clever combination of the new '-canny' edge detection with some variants of '-*contrast*' and '-threshold' improvements.

[snibgo]

"-lat" does a good first cut, I'd say.

Code: Select all

convert fingerprint-1024x575.jpg -colorspace gray -lat 15x15+0 fl.png

Removing the background is trivial. Deskewing would be harder: find the finger's edge, find the least-curved part of that, and this defines the angle (in this case, roughly south-west to north-east).

"Flattening" the finger would be harder. Pressing the finger to a surface converts the round shape to a plane, changing the geometry slightly. I suspect that most fingers need the same parameters for this.

[pipitas]

"-lat" does a good first cut, I'd say.

Code: Select all

convert fingerprint-1024x575.jpg -colorspace gray -lat 15x15+0 fl.png

Whoahhh! That's already a pretty good approximation! (I had thought that getting the lines in black and white would be the most difficult part...)

I haven't used or played with local adaptive thresholding ('-lat') before. But now I will!

For now I used

Code: Select all

convert fingerprint-1024x575.jpg -fuzz 10% -fill black -floodfill +10+10 black -colorspace gray -lat 15x15-1%  fp.png

This produced:

Thanks for the tipps!

[fmw42]

try this

Code: Select all

convert fingerprint-1024x575.jpg -colorspace gray -auto-level -gamma 0.3 -enhance -equalize -negate -lat 20x20+10% -negate fingerprint_result.gif

[snibgo]

Deskewing would be harder: find the finger's edge, find the least-curved part of that, and this defines the angle (in this case, roughly south-west to north-east).

Not very hard (Windows BAT script):

Code: Select all

%IM%convert fingerprint-1024x575.jpg ^
  -threshold 20%% ^
  -canny 0x1+10%%+30%% ^
  -background none -fill red -stroke red -strokewidth 2 ^
  -hough-lines 9x9+150 ^
  ft.png

I manually tweaked the "+150" parameter until I got one line. This could be done in a script, of course.

[pipitas]

Deskewing would be harder: find the finger's edge, find the least-curved part of that, and this defines the angle (in this case, roughly south-west to north-east).

Not very hard (Windows BAT script):

Code: Select all

%IM%convert fingerprint-1024x575.jpg ^
  -threshold 20%% ^
  -canny 0x1+10%%+30%% ^
  -background none -fill red -stroke red -strokewidth 2 ^
  -hough-lines 9x9+150 ^
  ft.png

I manually tweaked the "+150" parameter until I got one line. This could be done in a script, of course.

Hah!, I had done something very similar to this in order to determine the real rotation angles needed for the two images I've now received. I'm free to share them, so my next post will do so and show the results I currently achieve with them.

I didn't create the real visual red line output at first, but went for a textual 'mvg:-' output. There I determined the line with the highest number of pixels (indicated as the number at the end of each line following the '#' character. Then I looped through my variations of '+150' parameters in order to get that line as the only line. Semi-scripted, but not yet completely...

Thanks for the input, snibgo!

[pipitas]

Here are two real-world fingerprint photos (showing the same finger from different angles and distances) which I'm now trying to tame:



Currently I've arrived at these two commands:

Code: Select all

convert                                 \
     http://i.stack.imgur.com/XFKU9.jpg \
    -alpha on                           \
    -fuzz 40%                           \
    -fill none                          \
    -floodfill +0+0 blue                \
    -colorspace gray                    \
    -lat 15x15+0.3%                     \
    -negate                             \
    -rotate  -3                         \
    -trim                               \
    +repage                             \
    -extent x573                        \
    fp-number-1.png

convert                                 \
     http://i.stack.imgur.com/vlGab.jpg \
    -alpha on                           \
    -fuzz 40%                           \
    -fill none                          \
    -floodfill +0+0 blue                \
    -colorspace gray                    \
    -lat 15x15+0.3%                     \
    -negate                             \
    -rotate -47                         \
    -trim                               \
    +repage                             \
     fp-number-2.png

The first command has the addition of '-extent' so that both resulting images have the same height. This is meant as a step towards a better way to later merge them somehow into a "unified" fingerprint.

These are my results for now:





My next challenges are:

• Try to "smoothen" the black lines as much as possible.
• Make the black and white "lines" equal in width.
• Distort one fingerprint in a way so that certain anchor points move to the equivalent anchor points of the other fingerprint (or distort both to move to the anchor points of a third print).

The smoothened end result should look similar to this (sans rotation, sans white areas):

[pipitas]

try this

Code: Select all

convert fingerprint-1024x575.jpg -colorspace gray -auto-level -gamma 0.3 -enhance -equalize -negate -lat 20x20+10% -negate fingerprint_result.gif

Thank you, Fred.

I tried to add '-auto-level', '-enhance', '-equalize' and various variations of '-gamma' to the commands for my new pictures.

I wasn't able to find a combination that would improve my results equally well as your command does for my old example image.

[fmw42]

try prefiltering with either -kuwahara or -mean-shift and vary the sizes, e.g.

Code: Select all

convert fingerprint-1024x575.jpg -kuwahara 3 -fuzz 10% -fill black -floodfill +10+10 black -colorspace gray \
-lat 15x15-1% test1.gif

Code: Select all

convert fingerprint-1024x575.jpg -colorspace YIQ -mean-shift 9x9+10% -set colorspace YIQ -colorspace sRGB \
-fuzz 10% -fill black -floodfill +10+10 black -colorspace gray -lat 15x15-1% test2.gif

[pipitas]

try prefiltering with either -kuwahara or -mean-shift and vary the sizes

I tried filtering with '-kuwahara' already before -- however not pre-filtering, but filtering what my current result is.

I do not see any advantage when pre-filtering with kuwahara, having now tried it as per your suggestion on my last two example, 'real' images. (Your example commands use the previous sample, where it seems to work.)

Doesn't kuwahara, the way you applied it on 'fingerprint-1024x575.jpg', smoothen out the original photo, making it more difficult to apply the '-lat' operator afterwards to produce the b+w graphic?

Shouldn't we first produce the b+w graphic, and then apply some filtering (either -kuwahara, or -mean-shift, or something else) on the graphic?

[fmw42]

It is an edge preserving noise filter. So it should not smooth anything. I do not know whether it will work better on the color or grayscale image. Probably be about the same, but it is easy enough for you to try both ways and see the difference. It is not intended to be a noise filter for binary images. Likewise the mean-shift is not intended for use on a binary image, but works best on color data in the YIQ colorspace.