-distort Resize instead of -resize w/out HDRI esp. enlarging

[NicolasRobidoux]

http://entropymine.com/imageworsener/clamp-int/ suggests that IM resize, because it uses two-pass orthogonal resampling, which (allegedly) stores intermediate values in a clamped range when not in HDRI mode, leads to unexpected artifacts when enlarging a lot. This makes sense, and I would expect this to also happen when downsampling, but much much less, because haloing is not much of an issue when downsampling as a result of "averaging" over larger areas.

Does IM indeed store the result that sits between the two orthogonal passes in a format that forces clamping of the intermediate result?

When using a filter that does not over/undershoot, like Triangle (bilinear), Point (nearest), Cubic (cubic B-spline smoothing), Quadratic (quadratic B-spline smoothing), or Gaussian (Gaussian blur), it does not matter, because there are no over and undershoots to clamp. This is mostly an issue with the strongly sharpening filters, like Catrom or Lanczos, and to a lesser extent, with Mitchell.

I've not looked carefully enough at the code to see if this is really what's going on, or checked that I can see these artifacts, but one quick solution to this problem would be for a user to swith to -distort Resize (which of course brings other issues to the table), because it does not go through an intermediate storage. When running a Q8 version of IM, for example, this may be worth it. Q8 using -distort Resize may actually run faster than Q16 with -resize for a given result quality. It also suggests using less sharpening filters at low quantum.

This also could be resolved using an intermediate float or double image just to pass values from one orthogonal sweep to the other, not elsewhere.

A "deeper" but way way more complicated solution would be to use a rotating double precision array of image rows, never storing the entire intermediate image (at low or high quantum). Is this what IM does already? (If so, my hat is off to whoever programmed this, because doing this well has got to be painful given the various filter extents and the handling of up- and downsampling.)

[NicolasRobidoux]

I'm a believer: resize with -filter Lanczos gives "very different" results with -resize with Q16 and HDRI (through linear light, the only case I checked), but almost perfectly identical results with -distort Resize, with both input and output images 16-bit sRGB uncompressed TIFF. Of course the comparison is a bit unfair (EWA Lanczos is much smoother than orthogonal Lanczos) but the discrepancy in max error is so big that I'm convinced, without even looking at the code, that this is a side effect of clamped storage between the horizontal and vertical passes of tensor resizing.

Moral of the story: Consider using -distort Resize when quantum is "low", esp. when enlarging. You may not quite be getting what you ordered. Even more so if transparency is involved. Or else don't use filters with significant negative lobes.

[magick]

ResizeImage() is two pass subject to clamping unless HDRI is enabled. We utilize the pixel cache during the two passes to ensure huge images can be resized (e.g. 250000x250000) as will as convenient access to virtual pixels. Allocating huge images in memory is not practical. After we implemented ResizeImage(), we had a need for 2D storage that can handle huge requirements in the Deskew algorithm. It uses 2D cells and can be allocated in memory, memory-mapped, or disk depending on the storage requirements. We could port this storage algorithm, either generalize it, or use it for intermediate double values for the two pass resize algorithm to prevent clamping. However, even with then, virtual pixel handling would be challenging. The recommended solution. Enable HDRI in ImageMagick or use -distort Resize.

[anthony]

http://entropymine.com/imageworsener/clamp-int/ suggests that IM resize, because it uses two-pass orthogonal resampling, which (allegedly) stores intermediate values in a clamped range when not in HDRI mode, leads to unexpected artifacts when enlarging a lot
...
This makes sense, and I would expect this to also happen when downsampling, but much much less, because haloing is not much of an issue when downsampling as a result of "averaging" over larger areas.

It isn't alleged - it actually does. It is a 2 pass distortion with an intermediate image in-between. Distort Resize however is a single pass, so does not have intermediate clamping. NOTE this also happens when blurring images.
See the discussion on Blur vs Gaussian Blur, for exactly the same reason!
http://www.imagemagick.org/Usage/convol ... an_vs_blur

The bigger difference between using Distort scale, and resize, is as mentioned in another discussion, the use virtual pixels near edges, where resize ignores virtual pixels completely. This is the effect that 'Distort Resize' is attempting to fix.

WARNING: In recent 'play' I discovered that Distort Resize edge handling fails if you turn off EWA, so as to resize using interpolation! This was one reason why I backported -interpolative-resize to IMv6.

Note I generally try not to use it in too many examples using that operator, as the option name will change in IMv7, along with most other 'special' resizes, including -distort Resize - which is not really a distort method.

one quick solution to this problem would be for a user to swith to -distort Resize

Or use -interpolative-resize, which is also single pass, and really only suitable for upsampling as it is a unscaled interpolated 'point' sampling (no area resampling).

When running a Q8 version of IM, for example, this may be worth it.

Definatally! Q8 has a lot of problems due to quantum rounding effects. Resize will work, but it will have vertical/horizontal effects, as Jason Summers, demonstrates. But IM Q16 will not be as bad as many other image programs wit ha Q8 limitation, including the original 'zoom' program by Paul Heckbert, and from which all modern resize techniques evolved.

This also could be resolved using an intermediate float or double image just to pass values from one orthogonal sweep to the other, not elsewhere. Which most other image processing programs which uses intergers to save memory, can NOT do.

A "deeper" but way way more complicated solution would be to use a rotating double precision array of image rows, never storing the entire intermediate image (at low or high quantum). Is this what IM does already? (If so, my hat is off to whoever programmed this, because doing this well has got to be painful given the various filter extents and the handling of up- and downsampling.)

It could be possible to have a third resize method that is a one pass resize, but it would have to use a sort of generated 2-dimentional convolution kernel (with a floating point 'center' for sampling alignment) for each and every pixel in the destination. Distort EWA does something like this for each pixel in the support range (an ellipse) but the weights are still a linear 'distance from sample point' lookup of a 1-d cylindrical radius, and not a 2-dimentional matrix generated from a 2-pass tensor filters.

[anthony]

In summery. Is it really worth the effort! I doubt it.
As Nicolas says. Use distort resize if this is a problem, or switch to HDRI if you really want that level of accuracy.

(You'd want HDRI if you are doing Fourier Transform stuff anyway!

[NicolasRobidoux]

If and when I get around to program the PWA (Parallelogram Weighted Averaging) variant of EWA (Elliptical Weighted Averaging) this would fix the problem (with a performance cost: tensor resizing methods would run at -distort Resize speed, because they would be computed essentially the same way).

[anthony]

If and when I get around to program the PWA (Parallelogram Weighted Averaging) variant of EWA (Elliptical Weighted Averaging) this would fix the problem (with a performance cost: tensor resizing methods would run at -distort Resize speed, because they would be computed essentially the same way).

PWA, is currently a theoretical resampling method, developed during a conversation between Nicholas and myself. Essentially using the very well known tensor filtering, but using a rotated axis matching the axis's generated for EWA. Basically this gets us the same benefits for simple scaling and no-op situations (no tensor vector rotations), but the elliptical benefits when doing rotations or perspective type distortions.