what's going on behind the scenes when applying the crop?

[jack-mann]

Here is my sample uses case:

1) extract PNG from a PDF (vector to raster). This yields a 20992x8960 PNG file (~17MB) (I am using ghostscript for this)

2) convert large.png -crop 256x256 -set "filename:t" "%[fx:page.x/256]_%[fx:page.y/256]" +repage +adjoin PNG8:'.$workDir.'"/img_%[filename:t].png"

Step 2 generates 2870 cropped PNGs and takes about 2 minutes.

1) My first question is why does imagemagick need 2 mins to do the cropping? I would assume that cropping is a very simple operation so couldn't the performance be better?
(please note that I assume that there are very very good reasons that the cropping takes so long. I just would like to know why?)

2) On looking at the use case could anybody suggest a faster way to generate the cropped images? For example maybe extract from PDF to a different image format?

Version: ImageMagick 6.8.7-7 Q16 x86_64
Mac OSX

[glennrp]

Here is my sample uses case:

1) extract PNG from a PDF (vector to raster). This yields a 20992x8960 PNG file (~17MB)

The filesize of the PNG is not relevant. What's important is the memory required to hold the image after
it has been decoded, namely 21x9 Megapixels x 8 bytes per 16-bit RGBA pixel, for a total of
around 1.5 Gbytes.

You can use a Q8 build to cut that memory requirement in half, and you could use the "-limit"
option to help speed things up if you have enough available memory.

[jack-mann]

on convert -list

Code: Select all

I  File       Area     Memory        Map       Disk   Thread  Throttle       Time
--------------------------------------------------------------------------------
   192    4.295GB       2GiB       4GiB  unlimited        1         0  unlimited

I tried a limit of 4GiB
but got no improvement (even 5 secs worse)
I read the link viewtopic.php?f=3&t=19642&p=77546#p77546
and tried to use

Code: Select all

  convert input_image.png \
               \( +clone -crop 128x128+0+0 +repage -write tile_001.png +delete \) \
               \( +clone -crop 128x128+128+0 +repage -write tile_001.png +delete \) \

But I then get arguments to long exception since I produce 2870 cropped PNGs

[jack-mann]

I am achieving quite good results now if I preprocess the crop and cut the large image before in stripes.
Because the stripes have for example only a height of 256 px the crop can use much less memory.

Thanks glennrp for pointing out how important memory is for cropping.

I tried to chain the "cropping into stripes" (y-axis) and then "cropping into tiles" (x-axis) all in one convert.

Code: Select all

convert img.png -crop 20900x +repage +adjoin  -crop 256x256 -set "filename:t" "%[fx:page.x/256]" +repage +adjoin ......

But then again the process was slow again. It seems that all the stripes are held in memory so the strip cutting becomes useless.

[glennrp]

I tried a single-color image with the dimensions you have, and it took 70 seconds.
the "png8" reduction can be expensive if you have more than 256 colors in the source image.
I tried cropping just .png instead of .png8, and it took about 55 seconds, but then I followed
that with mogrify -format png8 *.png and that took 25 seconds. So if you know your
image already has fewer than 256 colors you can omit the "PNG8:" and save a few
seconds.

[glennrp]

Switching to using a Q8 build of ImageMagick did indeed save a lot of CPU time, reduced the crop operation to 34 seconds on my machine, which has 4GB of memory.

[snibgo]

For me, using a gradient image, IM v6.8.8-7 Q16 Windows 8.1 with 12 GB memory:

Writing to jpg is faster than png, eg 10 seconds for 2870 jpegs versus 45 seconds for 2870 pngs. Not that I would recommend jpeg for anything much, as it is lossy. Output to jp2 took 34 seconds.

Taking the input from mpc instead of png saved 3 seconds.

EDIT: My more usual " +adjoin -compress zip x.tiff" took 21 seconds.

[glennrp]

For me, using a gradient image, IM v6.8.8-7 Q16 Windows 8.1 with 12 GB memory:

Writing to jpg is faster than png, eg 10 seconds for 2870 jpegs versus 45 seconds for 2870 pngs. Not that I would recommend jpeg for anything much, as it is lossy. Output to jp2 took 34 seconds.

Try

Code: Select all

convert -quality 1 input.png output.png

if you are more interested in speed than filesize. For a typical photo it will run twice as fast and be about 5 percent larger.

[jack-mann]

the option -quality 1 got me a 20% performance push! I am now with the cutting into stripes and using -quality 1 down to 38 secs...