-transparent results in faded-out images

[Misty]

Hi. I'm trying to use ImageMagick to produce PDFs with transparent backgrounds, so that they can be stamped on top of another PDF using pdftk. The files always use a pure white background, which makes them easily scriptable. Unfortunately, I've been running into a problem. When I attempt to make all white pixels transparent, the remaining image looks faded out when I view it in Acrobat or Acrobat Pro. The colours display correctly if I create the PDF without transparent portions. If I create a transparent background PDF using Photoshop the colours display correctly in Acrobat, but I'd rather use ImageMagick if I can.

Here's a sample command:

convert -transparent white -compress JPEG2000 0060.sep.tif 0060.sep.pdf

I've tried both JPEG and JPEG2000 compression, with the same results both times. I'm using ImageMagick 6.6.3-0 on Windows. Here are a few sample pages:

With transparency: http://www.mediafire.com/?vsj0i0kj3o80s25
Without transparency: http://www.mediafire.com/?fauevghh3pg2h71

Thanks!

[fmw42]

control of transparency in pdfs is problematic. your command should be rewritten in correct syntax as

convert 0060.sep.tif -transparent white -compress JPEG2000 0060.sep.pdf

but that may not make any difference.

[Misty]

Unfortunately, that didn't change the output. The images are still displaying as washed out.

Is there a way to fix this? If not in ImageMagick, is there another open-source program I could use? I'd prefer to use ImageMagick or other open-source software to doing the image PDF encoding in Photoshop.

Thanks for your help.

[fmw42]

try the same process, saving as png, for example, then use some other tool to convert that to PDF

[Misty]

I tried converting to a PNG with transparency, then converting that into a PDF with Acrobat. The result is that the image still appears washed-out. However, if I open the same PNG in Photoshop and convert it to a PDF, that PDF opens with the correct colours. If I create the PNG with transparent background in Photoshop, it opens in Acrobat using the correct colours.

I was wondering if it was a colour management problem, but when I added a colour profile step (-profile sRGB.icm before -transparent) and created a PDF, the result still looked the same as without the profile step.

Here's a zip file containing Photoshop and ImageMagick versions of the same PNG: http://www.mediafire.com/?15xbc5us5actyh2

[fmw42]

I looked at your two images in Mac Preview and they look identical. IM compare shows no differences.

compare -metric rmse 0060.sep.im.png 0060.sep.ps.png null:
0 (0)


I think we need to see your input image to see what is special about it, if anything and to do some testing.

What/where is it washed out in either of these images?

What system are you on? What version of IM (perhaps an upgrade for IM and Ghostscript is in order)? What viewer are you using to examine them and where you see the washed out issue?

PS

Your -compress jp2000 probably won't be valid for the PNG output and if you save as .jp2, then you won't get any transparency.

[Misty]

The two images look identical to me in the built-in Windows image viewer, but display differently from each other as PDFs and are slightly different in Photoshop. It's strange.

Now that I look at them, it appears the two images are opening slightly differently from each other in Photoshop, though they both appear at roughly the correct brightness. The version Photoshop created is an RGB file, while the one ImageMagick created is greyscale. This suggests to me that it is a colour management problem, possibly from conversion to/from greyscale with incorrect gamma. When I tried creating a PDF with transparent background from a page with a colour photo, the colours display correctly.

I'm not sure where to proceed from here, however. If I explicitly tell ImageMagick to load the image as sRGB with the sRGB profile, it still doesn't display correctly. If I try to embed a colour profile and save a PNG or JPG, Photoshop tells me that the profile is invalid.

Here's a zip containing the original TIFF: http://www.mediafire.com/?mrnn6d2ub51jhjl

Thanks again for the help.

Edit: But, come to think of it, I don't understand why colour management would affect things only when the transparency is added.

Here's what the correct colours look like:


And here's how Acrobat displays the version with the transparent background:

[fmw42]

Looking at your original, which is a tiff, which can be problematic, with identify -verbose


Image: 0060.sep.tif
Format: TIFF (Tagged Image File Format)
Class: DirectClass
Geometry: 813x1032+0+0
Resolution: 100x100
Print size: 8.13x10.32
Units: PixelsPerInch
Type: Grayscale
Base type: Grayscale
Endianess: MSB
Colorspace: RGB
Depth: 8-bit
Channel depth:
gray: 8-bit
Channel statistics:
Gray:
min: 0 (0)
max: 255 (1)
mean: 188.92 (0.740862)
standard deviation: 75.3128 (0.295344)
kurtosis: -0.449048
skewness: -0.928053
...

Rendering intent: Undefined
Interlace: None
Background color: white
Border color: rgb(223,223,223)
Matte color: grey74
Transparent color: black
Compose: Over
Page geometry: 813x1032+0+0
Dispose: Undefined
Iterations: 0
Compression: LZW
Orientation: TopLeft
Properties:
date:create: 2010-08-31T17:35:14-07:00
date:modify: 2010-08-30T11:02:48-07:00
signature: d2459898dc3e837f4526018dc0404e7b86d473ca9c5eaba3869e6cd030cc5677
tiff:endian: lsb
tiff:photometric: RGB
tiff:rows-per-strip: 5
Artifacts:
verbose: true
Tainted: False
Filesize: 889KB
Number pixels: 839K
Pixels per second: 1.353M
User time: 0.180u
Elapsed time: 0:01.619
Version: ImageMagick 6.6.3-10 2010-08-31 Q16 http://www.imagemagick.org


As the tiff is compressed, it may have to be decompressed. The other thing to keep in mind is that IM uses Ghostscript to make PDFs and that depends upon the device being used by Ghostscript. Some devices don't allow transparency and those that do may have other issues. I am not really an expert on this. So hopefully someone else may be able to explain.

One interesting thing is that IM reports MSB for the endian, but the tiff properties show LSB. I don't know what to make of that. Fortunately, there don't seem to be any profiles.


Exiftool reports

exiftool -s -ee -g1 -u -n -D 0060.sep.tif
---- ExifTool ----
- ExifToolVersion : 8.24
---- System ----
- FileName : 0060.sep.tif
- Directory : .
- FileSize : 889082
- FileModifyDate : 2010:08:30 11:02:48-07:00
- FilePermissions : 755
---- File ----
- FileType : TIFF
- MIMEType : image/tiff
- ExifByteOrder : II
---- IFD0 ----
254 SubfileType : 0
256 ImageWidth : 813
257 ImageHeight : 1032
258 BitsPerSample : 8 8 8
259 Compression : 5
262 PhotometricInterpretation : 2
273 StripOffsets : (Binary data 1395 bytes, use -b option to extract)
277 SamplesPerPixel : 3
278 RowsPerStrip : 5
279 StripByteCounts : (Binary data 1000 bytes, use -b option to extract)
282 XResolution : 100
283 YResolution : 100
284 PlanarConfiguration : 1
296 ResolutionUnit : 2
317 Predictor : 1
---- Composite ----
- ImageSize : 813x1032



But this does not tell me much more. Perhaps someone more experienced with exiftool can say if there is anything strange here.


It is also possible that PS does something extra to the data (gamma or brightness contrast, etc) before converting to PDF (or possibly on opening the file). Look for settings on your PS, especially with regard to PDF. I don't know if PS relies upon GS or not.


Good luck as I don't know what else to tell you.

[Misty]

I've been able to replicate the fading in Photoshop. It looks like what happens is that Photoshop opens every greyscale file without an embedded profile as Gamma 2.2; if I have Photoshop assign the original TIFF the Dot Gain 10% profile, it looks identical to the faded PDFs being produced by ImageMagick. Photoshop's default colour management behaviour is probably "fixing" the PNGs and other files being produced by ImageMagick, because it overrides them all to Gamma 2.2 which provides a correct gamma. Unfortunately, using +gamma 2.2 in ImageMagick to assign a gamma to the image without altering the pixels has no effect on the output PDF.

Why the PDFs being produced by ImageMagick are being interpreted as different colour profiles or gamma depending on whether or not they have transparency is still a bit mysterious to me. Maybe a Ghostscript issue? Either way, I wonder if there's a possible workaround for it by assigning the correct profile (Gamma 2.2 for greyscale, sRGB for colour) before converting to PDF. Unfortunately the script I'm using ImageMagick in has to process a large number of TIFFs without any awareness of their content. The files may be a mixture of both greyscale and colour. Is there a way to tell ImageMagick to assign the correct profile based on whether the input is RGB or greyscale?

Edit: I tried assigning a profile to the original TIFF using Photoshop, then converting to PDF with transparent background, but the resulting PDF still displays with an incorrect gamma. The only workaround I have found so far is to assign an sRGB profile in Acrobat after creating the PDF.

[Drarakel]

The other thing to keep in mind is that IM uses Ghostscript to make PDFs and that depends upon the device being used by Ghostscript. Some devices don't allow transparency and those that do may have other issues.

That's true for reading PDFs - but writing PDFs is a different thing. The PDF files are created by just ImageMagick itself, I would say.

If I try to embed a colour profile and save a PNG or JPG, Photoshop tells me that the profile is invalid.

You tried to assign a sRGB profile to a greyscale picture - right? In order to get a more compatible output with something like that, you will need an additional option: "-type TrueColorMatte" or "-type TrueColor" (depending if it's with or without alpha channel). Without that option, IM will write the file in a grayscale format - and then the colorspaces of the image and of the color profile are not identical.


The original problem with the different PDFs is a bit more difficult...
About the PDFs (from the first post) itself: '0060.sep-trans.pdf' and '0060.sep-notrans.pdf' are stored in a very similar way. The main images (the JPEG2000 compressed images inside the PDFs) are bit-for-bit identical. '0060.sep-trans.pdf' of course has the additional transparency (it's stored as an additional, RLE compressed mask with binary transparency). That's basically the only difference.
It's really a bit strange with the different looking PDFs. ImageMagick stored the main (JP2) image in a grayscale format - but that applies to both PDFs. So, I'm also not completely sure why the additional transparency triggers the different outcome of the complete image. But I think you were already on the right track.. Could be that a color profile is needed when an additional mask is used. And if the images are stored as grayscale, all that can change the picture (in a conversion from a default grayscale color profile to e.g. sRGB).

The 'solutions' would be:
- Store the main image in the PDF with the correct color profile (ICCBased).
That's not possible with ImageMagick as far as I know.
- Store the main image as full RGB - not as grayscale. (Some default grayscale color profile won't be used then anymore.)
That's also a bit trickier with PDFs, but it should be possible. The key is to use "-type TrueColor" before you add transparency. (If you do it the other way round, then the transparency will be gone. And "-type TrueColorMatte" doesn't help here, as the image can still be written in a grayscale format in PDFs with that.)

I would convert such TIFF files like that:

Code: Select all

convert 0060.sep.tif -strip -type TrueColor -transparent white -compress zip -quality 90 0060.sep-trans2.pdf

You should get a result that looks like your '0060.sep-notrans.pdf' (but with transparency).

[Misty]

Excellent, that did the job! Thanks a lot. The shades are displaying fine now.

Unfortunately it does seem to produce much much larger JPEG2000 files (max quality files are 1331KB with -strip -type TrueColor vs 448KB normally), so I'm going to have to see if I can find a way to get efficient compression when using those options. Regular JPEG is producing files closer to the normal size (498KB for max quality, lower with the quality options I'm going to be using in my script). I may just fall back on that, unless there's a way to get the JPEG2000 files closer to their normal sizes.

[Drarakel]

Unfortunately it does seem to produce much much larger JPEG2000 files

Sure - as with "-type TrueColor", the main image is written as full RGB (well, it's YUV or YCbCr, but it's still 3 channels) instead of plain grayscale (1 channel). But that's probably the only workaround for this in ImageMagick.
You could get your 'old' JP2 compression size with.. "-quality 87" (which is still a pretty good quality). Or - at least with this example - use the ZIP compression (707KB - and lossless like with the default JP2 compression).

[Misty]

It makes sense that it's using more channels, but why is the JPEG2000 output producing larger files than regular JPEGs? JPEG2000 is normally much more efficient compression, but it's producing files that are much bigger here. I can't produce equivalent filesizes to JPEG without using unacceptable compression, even when both are using -strip -type TrueColor.

Edit: It also seems a bit strange to me that lossy JPEG2000 is producing larger files than lossless ZIP or LZW compression.

I'm using ImageMagick as part of a script to produce small web-quality PDFs; the images are one of two layers forming a page. The reason I'm using JPEG2000 is to get smaller images at a medium compression level than what JPEG can give. It's unfortunate to have to drop back to JPEG for that reason, since it limits how small I can get the pages while retaining a reasonable quality.

[Drarakel]

I don't use the JP2 compression much myself - so I can't give you many hints here..

But regarding the filesizes:
The default for JP2 is quality 100. And that's lossless. A lossless compression is not available in (normal) JPG compressed files, so it's natural that a JP2 file with quality 100 will often become much bigger than a JPG file with quality 100. And it seems that the lossless compression is not that efficient in JP2, so it's again bigger than lossless ZIP or LZW (could vary with different images of course).
And even if you don't use quality 100, the quality value in ImageMagick for JP2 is not directly comparable to the same JPG quality value. With JPG, the quality value is an indicator for the visual quality as it sets certain quantizer values (and a quality value below 50 is not recommended) - but the filesize with that is sort of unpredictable. The quality value for JP2 however is a shortcut for the compression factor. Usually, you have to choose a lower JP2 quality value to reach the same filesize as with a JPG file (but the JP2 file could still produce a better visual quality).

See also:
http://www.imagemagick.org/script/comma ... hp#quality
http://www.imagemagick.org/script/comma ... php#define (jp2:rate)
http://www.imagemagick.org/script/jp2.php

[Misty]

Sorry to bring back the thread after so long. I've been working on my script again, and with some experimentation I've found some strange, inconsistent behaviour in the JPEG2000 compression which explains why I was having some of the problems I've had.

It appears that the problem with the filesize is due to the transparent background (obtained by performing -transparent white). The same file processed using the same quality commands will produce a visibly different level of quality depending on whether the image has a transparent background. The filesize also differs wildly. This happens only when compressing using JPEG2000, not when using regular JPEG. Here are a few examples demonstrating this, in JP2 and JPG format.
0128.jp2.pdf - Transparent background. Compressed using -quality 55. Filesize: 128KB. Image quality: Extremely low.
0128.jp2-notrans.pdf - Opaque background. Compressed using -quality 55. Filesize: 83KB. Image quality: Excellent.
0128.jpg.pdf - Transparent background. Compressed using -quality 85. Filesize: 123KB. Image quality: Excellent.
0128.jpg-notrans.pdf - Opaque background. Compressed using -quality 85. Filesize: 123KB. Image quality: Excellent.

Going back to the question of -type TrueColor earlier as well, I noticed that regular JPEG has an almost unnoticeable quality difference between files compressed with or without that option. In my testing, I've noticed a filesize difference of only about 2KB. The issue with the significantly increased filesize is also only occurring with JP2.

Edit: Since it's been awhile, I'll reiterate that I'm on Windows. I'm using IM 6.6.5-4 from the official binaries.