quick benchmark comparison of gcc compilation flags

[NicolasRobidoux]

Here is a quick and dirty comparison of various compile ImageMagick
options, with some very shaky conclusions (only one task was used for
the comparison).

-------
Summary
-------

On a 6 month old mid-line consumer laptop with bleeding edge
ImageMagick takes clear under 2 seconds to produce a 40x40 thumbnail
from a 4000x3000 jpeg (using -distort resize, which is not as fast as
plain -resize, but which generally gives better looking results).

------
Caveat
------

I have not tested older versions of ImageMagick. The tests were not
done on a perfectly quiet system. My test machine only has 2 cores, so
I cannot really check "multi-core parallel scaling".

--------
Hardware
--------

ThinkPad T520i
Intel Core i3-2310M CPU @ 2.10GHz
4GiB SODIMM DDR3 Synchronous 1333 MHz (0.8 ns)
ATA Disk HITACHI HTS72323 buffered disk reads: 91.46 MB/sec
L1 cache: 64KiB
L2 cache: 256KiB
L3 cache: 3MiB

--------
Software
--------

Linux Mint 12 (= Ubuntu 11.11 = Debian Unstable ???) with all updates.
Jupiter power management set to maximum performance.
All relevant software straight from the package manager except ImageMagick.
gcc version: 4.6.1

-----------
ImageMagick
-----------

Version 7.0.0, the most current svn (bleeding edge), compiled from source.

----
Task
----

Produce a 40x40 thumbnail directly from big.jpg, a 4000x3000 sRGB jpeg with
file size 1.3MB.

Command:
time ( convert big.jpg -filter Robidoux -distort resize 40x40^ -gravity center -extent 40x40 -quality 75 -sampling-factor 2x2 thumb.jpg )

(Some of the flags are not needed because they are the default. I like explicit.)

Real time is reported below.

Comment: Using -crop instead of -extent adds something like 1% to the
run time. It may make more of a difference with larger images. -extent
is recommended over -crop here:
http://www.imagemagick.org/Usage/resize/#fill

-------------------------------------
Results with a 1.3MB JPEG (4000x3000)
-------------------------------------

1.764 seconds with plain vanilla config/compilation: OpenMP, Q16, no arch:
./configure

1.754 seconds with -march=native:
./configure --with-gcc-arch=native
or
CFLAGS="-march=native" ./configure

1.761 seconds with -march=native, debugging (-g) off, adding -fomit-frame-pointer:
./configure CFLAGS="-fopenmp -fomit-frame-pointer -O2 -Wall -march=native -pthread" CXXFLAGS="-O2 -pthread"

(P.S. With more careful tests, I found out that removing -g and adding -fomit-frame-pointer speeds things up by about 2%.)

2.931 seconds with OpenMP turned off:
./configure --disable-openmp

2.907 seconds with -march=native and OpenMP turned off:
./configure --with-gcc-arch=native --disable-openmp

(-fomit-frame-pointer is suggested here: http://www.gentoo.org/doc/en/gcc-optimization.xml)

(Warning: make clean before make if you want your new configure parameters to stick: viewtopic.php?f=3&t=20496.)

-----------------
Results with HDRI
-----------------

2.026 seconds with -march=native, debugging (-g) off, adding -fomit-frame-pointer:
./configure CFLAGS="-fopenmp -fomit-frame-pointer -O2 -Wall -march=native -pthread" CXXFLAGS="-O2 -pthread"

-----------------------------------
Results with a 2MB JPEG (4000x3000)
-----------------------------------

1.776 seconds with -march=native, debugging (-g) off, adding -fomit-frame-pointer:
./configure CFLAGS="-fopenmp -fomit-frame-pointer -O2 -Wall -march=native -pthread" CXXFLAGS="-O2 -pthread"

-------------
"Conclusions"
-------------

(Don't quote me on this.)

The input file size does not matter very much, it's mostly the size in pixels.

Fancy compile flags do not make much of a difference.

HDRI does not cost much more, even with large images.

At least on a fairly quiet 2-core machine, OpenMP wins.

[NicolasRobidoux]

Now, what I'm trying to do is figure out whether/why ImageMagick appears to be slow on a state-of-the-art monster machine running CentOS 5.6 (machine which I don't have access to).

My first pass, documented above, had to do with testing on one of my laptops running Linux Mint 12.

P.S. The following results don't mean too much except that one has to make sure that all key libraries are installed in compatible versions otherwise IM may run, but very very slowly.

Next, I installed CentOS 5.7 (as well as all its updates) on an almost identical but better laptop, only different in that it has a better chip (i5 instead of i3; the i5 has considerably more cache) and twice has much (and better) RAM (8GiB instead of 4GiB).

ImageMagick 6.7.5-10 (latest stable) installed from source with plain vanilla ./configure takes a lot longer to perform the same task than 7.0.0 within Linux Mint 12. Barring a discovery that there was a big jump in IM's performance going to 7.0.0 from 6.7.5-10, this suggests that there is a pretty steep performance cost to running "conservative" distros (at least if one does not manually upgrade quite a few things; for example, make emits quite a few "skipping incompatible .../.../lib.../" so I clearly was somewhat careless in my install).

The numbers
Making a 40x40 thumbnail from a 1.3M 4000x3000 jpeg takes 9.215 seconds on the faster machine running CentOS 5.7 VS 1.764 seconds on the slower machine running Linux Mint 12.

More numbers
Running with export MAGICK_THREAD_LIMIT=1: 24.195s
Running with export MAGICK_THREAD_LIMIT=2: 14.128s
Running with export MAGICK_THREAD_LIMIT=3: 11.049s
Running with export MAGICK_THREAD_LIMIT=4: 9.233s (4 threads is the recommended number for the i5 chip)
Running with export MAGICK_THREAD_LIMIT=5: 9.181s (but much more erratic run times; on average, roughly as slow as with limit=3)
Running with export MAGICK_THREAD_LIMIT=6: 9.040s (erratic run times)
Compiled with ./configure --disable-openmp: 24.115s

Tomorrow, I'll benchmark on CentOS 5.8.

(Minor annoyance: IM compiles much more slowly on CentOS 5.7 than on Linux Mint 12.)

[NicolasRobidoux]

Before moving to CentOS 5.8, I should have seen if I could install gcc44 and then use it with

Code: Select all

# yum install gcc44 gcc44-c++
# export CC=gcc44
# export CXX=g++44

This being said, RedHat recommends moving to 6.x and "provides a backport of gcc44 to help the transition", which means that not all benefits can be reaped while sticking to an older version of 5.x. (Current libraries/kernel have got to matter too.)

On the other hand, there are of course good reasons not to upgrade (and stick with a veryy laggy distro).

P.S.: viewtopic.php?f=1&t=20504#p81963

[NicolasRobidoux]

At some point I'm going to have to RTFM, e.g. http://www.imagemagick.org/Usage/api/#speed

[NicolasRobidoux]

Definitely RTFM: viewtopic.php?f=2&t=20012&p=79367#p79188

[NicolasRobidoux]

...Tomorrow, I'll benchmark on CentOS 5.8.

Well, whatever feeds power to my spare laptop's dvd drive went kaput, so my plans of carefully checking CentOS 5.8, installing the rpm found at ftp://ftp.imagemagick.org/pub/ImageMagi ... OS/x86_64/ on CentOS 5.7, and seeing how much better things are on CentOS 6.x went up in smoke. Too bad, really, because CentOS 6.2 seems really appetizing: http://www.linuxfordevices.com/c/a/News ... d-RHEL-58/.