Deleting blanks pages

[ylafont]

Newbie here,

After days of searching, i have been able to configure my scanner with SANE and ScanBD. In configuring, I seem to now need to delete blank scanned pages when using the ADF, although this is not my only problem i figured i start there.

I have found a few examples. one here
http://superuser.com/questions/343385/d ... mage-files

which suggest to use

Code: Select all

identify -format "%#" source.png,

which is incorrect. after some googleing, i found that the the corerct syntac is

Code: Select all

 identify -format "%k" source.png

. however that always returns a "2" indicating more than one color.

so i continued wht search. and founds this posting.

viewtopic.php?f=1&t=20339

that uses,

This command to check for mostly white space:
convert image_page5.tif -format "%[fx:mean>0.99?1:0]" info:

Which i am assume to take that if the value return is not "0" then remove the file. which i think should work. however when i run the command it take about 15 to 20 seconds to complete per page. Is this normal behavior? This can take forever on large documents.

IS there a standard method or a best practice methods for deleting blank pages? any assistance would be greatly appreciated. thank you.

FYI - Wanted to mention i am using Raspberry pi for this.

[snibgo]

It will depend on how many pixels, and the speed of the Pi.

I suppose by "blank" you mean "close to white".

Maybe "min" would be slightly faster than "mean".

A custom process module could be written, and this might be faster for non-blank pages because the search can end when the first dark pixel is found.

[fmw42]

you could use compare with one black pixel and the arguments -subimage-search, dissimilarity-threshold and -similarity-threshold. This will stop when it finds one pixel that is not close enough to white. For example:

# create pure white image

Code: Select all

convert -size 1000x1000 xc:white white.png

# create white image with 1 pixel that is 98% white at location 10,10

Code: Select all

convert -size 1000x1000 xc:white -size 1x1 xc:"gray(98%)" -geometry +10+10 -compose over -composite white1.png

# test of white1 (will stop if finds any pixel within 98% of black, i.e. gray(98%) or darker

Code: Select all

time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white1.png \( -size 1x1 xc:black \) null:

64224 (0.979995) @ 10,10
real 0m0.940s
user 0m0.834s
sys 0m0.088s


# test of pure white image

Code: Select all

time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) null:

65535 (1) @ 0,0
real 1m7.789s
user 1m5.578s
sys 0m2.110s


See
http://www.imagemagick.org/script/compare.php
http://www.imagemagick.org/Usage/compare/
http://www.imagemagick.org/Usage/compare/#statistics
http://www.imagemagick.org/script/comma ... age-search
http://www.imagemagick.org/script/comma ... -threshold
http://www.imagemagick.org/script/comma ... -threshold

[ylafont]

Thank you for the suggestions, Let me research on how to use. I am sure i will have questions since this is my first time IM.

[ylafont]

fmw42 two things.

there is no -similarity-threshold option shoudl that be remove or replace with a different option?

time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) null:

second, how am to determine if a page is to be delete from its output?

Code: Select all

pi@PiScanner /tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) null:
compare.im6: unrecognized option `-similarity-threshold' @ error/compare.c/CompareImageCommand/860.

real    0m0.105s
user    0m0.090s
sys     0m0.000s

[Bonzo]

There is a -similarity-threshold see http://www.imagemagick.org/script/comma ... -threshold

Your Imagemagick version could be to old; can you find out what the installed version is? convert -version should tell you.

[fmw42]

What version of IM are you using? It may be too old for the current commands that I specified. You need at least IM 6.8.3-10. Please always provide your IM version and platform when posting questions.

The file is to be deleted if it produces a perfect match at 0,0 with score (1). Since that means it is totally white within 2% of pure white.

65535 (1) @ 0,0

I have assumed that you are allowing for near white throughout the image. That is within 2% of white. You can adjust that by changing the -similarity-threshold 0.98.

[ylafont]

Debian linux (RaspberryPi Wheezy)

Just went from 6.7.7 to 6.8.9-10, I made the mistake of assuming the repository had the latest version.

Code: Select all

pi@PiScanner /tmp/TestScans $ convert -version
Version: ImageMagick 6.7.7-10 2014-04-09 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP


convert -version
Version: ImageMagick 6.8.9-10 Q16 armv6l 2014-11-04 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC OpenMP
Delegates: freetype jbig jpeg tiff zlib

However now i get the following when testing.

Code: Select all

$ sudo convert -size 1000x1000 xc:white white.png
convert: no encode delegate for this image format `XC' @ error/constitute.c/WriteImage/1167.

$ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) null:
compare: no decode delegate for this image format `PNG' @ error/constitute.c/ReadImage/501.

real    0m0.138s
user    0m0.100s
sys     0m0.040s

Performing a make check, now. no errors reported yet, So i am assuming i am missing something.

Notice I do not get perfect match or the score you specified.

"65535 (1) @ 0,0"

..... and FYI

I have already build and install the PNG delegatges. located here
http://www.imagemagick.org/download/delegates/

as soon as the make check will reboot to see it that helps.

[fmw42]

Code: Select all

$ sudo convert -size 1000x1000 xc:white white.png
convert: no encode delegate for this image format `XC' @ error/constitute.c/WriteImage/1167.

$ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) null:
compare: no decode delegate for this image format `PNG' @ error/constitute.c/ReadImage/501.

The first problem seems to be that you are missing some delegate library. But xc: is an internal IM specification of a constant image. So I do not understand why this is showing. But I can only assume you are missing some delegate. Perhaps it is the same issue as the second, which says you did not install the PNG (libpng) delegate library. Also I do not think you need sudo to run IM convert. But you may have to preface convert with the path to that version, to be sure you are not running your older version.

freetype jbig jpeg tiff zlib

This is too few delegates. You need a few more:

Mine shows:

bzlib cairo fftw fontconfig freetype gslib jbig jng jp2 jpeg lcms lqr ltdl lzma openexr png ps rsvg tiff webp x xml zlib

The purple ones you might need in the future. The red ones you likely need now.

[ylafont]

I have already build and install the PNG delegatges. located here
http://www.imagemagick.org/download/delegates/

as soon as the make check completes will reboot to see it that helps.

[ylafont]

Ok, i was able to finally fix the installation problems. Not sure why the delegates where to installed during the initial build.

Code: Select all

 convert -version
Version: ImageMagick 6.8.9-10 Q16 armv6l 2014-11-05 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC OpenMP
Delegates: bzlib djvu fontconfig freetype jbig jng jpeg lcms lqr openexr png tiff x xml zlib

however, still having issues with this on .tif images. here are my results.

Your files.

Code: Select all

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white.png \( -size 1x1 xc:black \) n                     ull:
65535 (1) @ 0,0
real    19m59.656s
user    18m39.850s
sys     0m38.730s
pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 white1.png \( -size 1x1 xc:black \) null:
64224 (0.979995) @ 10,10
real    0m15.794s
user    0m14.630s
sys     0m0.610s

My tif files.

Code: Select all

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 out2.tif \( -size 1x1 xc:black \) null:
compare: out2.tif: Seek error accessing TIFF directory. `TIFFFetchDirectory' @ error/tiff.c/TIFFErrors/582.
compare: out2.tif: Failed to read directory at offset 0. `TIFFReadDirectory' @ error/tiff.c/TIFFErrors/582.

real    0m0.253s
user    0m0.190s
sys     0m0.030s

Can i be missing something else?

[fmw42]

Looks like some corruption of your tif file or it has been customized such that IM does not think it is valid. The problem with Tiff is that everyone can change it to suit.

Can you post one of your tiff files to dropbox.com and put the URL here so others can test with it? Does you file have multiple layers? That could be the issue?

One thing to try. First see if IM can convert it to PNG and try that with compare. If you get the same error when converting it to PNG, then I think it may be corrupt or malformed.

[ylafont]

I could not identify or convert, the images were fine since i could view them. It appears it may be something with the TIFF delegates. I am upgrading now to see if it helps.

[ylafont]

I was finally able to straiten out all the delegate issues, command are now being accepted.

it still takes to almost 30 seconds per page, also, there is no score of (1) on even pages (out4.tif and out6.tif), which are white pages. any way of improving?

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 out1.tif \( -size 1x1 xc:black \) null:
0 (0) @ 7,0
real 0m27.467s
user 0m25.560s
sys 0m1.340s

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 out3.tif \( -size 1x1 xc:black \) null:
0 (0) @ 1,0
real 0m27.506s
user 0m25.750s
sys 0m1.110s

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 out4.tif \( -size 1x1 xc:black \) null:
0 (0) @ 11,0
real 0m27.255s
user 0m25.420s
sys 0m1.370s

pi@PiScanner ~/tmp/TestScans $ time compare -metric rmse -subimage-search -dissimilarity-threshold 1 -similarity-threshold 0.98 out6.tif \( -size 1x1 xc:black \) null:
0 (0) @ 10,0
real 0m29.699s
user 0m27.800s
sys 0m1.380s

[fmw42]

what do you get when you test with the commands I mentioned earlier with the white image and a white image with one gray(98%) pixel

Do you get my same results?

Can you upload your images (one you consider white and one that is not) to dropbox.com and post the URLs here?

Nothing to improve speed except more threads if you have multicore system.

You might find that snibgo's method of finding the mean value may be faster on average over all your images. You should test that.


What version of IM are you now using and what platform?