ImageMagick on rucore-dev

Project:RUcore Software Configuration
Version:8.1
Component:Upgrade software
Category:task
Priority:normal
Assigned:dhoover
Status:closed
Description

Please upgrade to the latest stable build. Version 6.4.7-10 and upwards is needed to support the high-resolution image specification.

https://rucore.libraries.rutgers.edu/collab/ref/spc_sawg_r8_1_high_resol...

Comments

#1

I started working with the ImageMagick installed on rucore-dev. I am running into a problem creating pyramidal TIFFs with JPEG compression on the server. Once the PTIF is created and accessed with IIPImage upon initial load the following error is in the iipsrv.log

TIFFReadEncodedTile failed for /mellon/jpeg2000/images/test-tiff-03-07-2016-linux-rutgers-lib-498.tif
Sending HTTP 404 Not Found

I looked around and found this thread

https://sourceforge.net/p/iipimage/discussion/299494/thread/58cadd87/

Based on that thread I tried two things.

1) Generated pyramidal TIFF on my workstation (Windows) with JPEG compression
2) Generated pyramidal TIFF on the rucore-dev server with JPEG compression

Both playback using IIPImage; but not the PTIF with JPEG compression created on the server. There is a mention in the above thread about the libtiff library needing to be compiled with JPEG support.

#2

Assigned to:dhoover» chadmills

ImageMagick's convert command is compiled with
dhoover@rep-devel2:~> ldd /usr/local/bin/convert |egrep "tiff|jpeg"
libtiff.so.3 => /usr/lib64/libtiff.so.3 (0x00007f2de94dd000)
libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x00007f2de8a75000)

and /usr/lib64/libtiff.so.3 is compiled with
dhoover@rep-devel2:~> ldd /usr/lib64/libtiff.so.3
libjpeg.so.62 => /usr/lib64/libjpeg.so.62 (0x00007fda2b8db000)

So it is compiled with jpeg support

About your comments:

1) Generated pyramidal TIFF on my workstation (Windows) with JPEG compression

- what version of libtiff and libjpeg are in use on the Windows server?

2) Generated pyramidal TIFF on the rucore-dev server with JPEG compression

Both playback using IIPImage; but not the PTIF with JPEG compression created on the server

- what is the difference between the two statements:
- Generated pyramidal TIFF on the rucore-dev server with JPEG compression
- but not the PTIF with JPEG compression created on the server

I thought pyramidal TIFF and PTIF mean the same thing?

Likewise I thought these meant the same server "created on rucore-dev" and "created on the server"

So why does one play and one not.

#3

1) On my Windows workstation this is listed.

c:\ImageMagick>convert -version
Version: ImageMagick 6.9.3-0 Q16 x64 2016-01-02 <a href="http://www.imagemagick.org" title="http://www.imagemagick.org">http://www.imagemagick.org</a>
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: <a href="http://www.imagemagick.org/script/license.php" title="http://www.imagemagick.org/script/license.php">http://www.imagemagick.org/script/license.php</a>
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib cairo freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

I don't expect the same libraries to used between the same systems.

2) I messed up my explanation; sorry about that. So I made three PTIF files.
A) On windows WITH JPEG compression
B) On linux WITH JPEG compression.
C) On linux WITHOUT JPEG compression

I was only to get IIPImage to render A and C. Item B returns the reported message in the IIPImage server log:

TIFFReadEncodedTile failed for /mellon/jpeg2000/images/test-tiff-03-07-2016-linux-rutgers-lib-498.tif
Sending HTTP 404 Not Found

PTIF and pyramidal TIFF are the same thing; just my inconsistent use of terms. Sorry for that as well. From this point forward I will try to use PTIF.

So the PTIFs for IIPImage can be created with or without JPEG compression. The PTIF with JPEG compression are considerably smaller. THe current issue I am having is with creating a PTIF with JPEG compression on rucore dev. I am going to try to just create a general TIFF with JPEG compression and remove the PTIF and tiling from the equation. I'll report back what I find.

#4

Not sure how much this helps but...

On rep-dev when I try to convert a TIF to a TIF with JPEG compression I get the following error.

cmmills@rep-devel2:/mellon/jpeg2000/images> convert cmm.tif -compress JPEG cmm-jpeg-compression.tif
convert: cmm-jpeg-compression.tif: Invalid tag "Predictor" (not supported by codec). `_TIFFVGetField' @ error/tiff.c/TIFFErrors/557.

On rep-test I do not get that error using its version of convert with the same sample TIFF.

#5

Assigned to:chadmills» dhoover

Dave,

Any luck with this on rep-dev? I have been thinking about this and it looks more and more to a libtiff issue. While not ideal, I was wondering if that has been, or can be, manually updated to test the results.

Thanks,
Chad

#6

I have numerous different configurations of ImageMagick and libtiff/libjpeg
on different servers (see attachement) and have been unable to get any
to work fully (though I may just be messing up the commands or URL
specification for IIPImage.

I would like to see if any of these configurations work when put through
the proper sequence without just "upgrading libtiff" with no version being
mentioned. I asked before if you know what version of libtiff is being used
in your Windows setup but I don't think you responded.

Can you supply me with
- a starting tiff file (or is it just anyone in /workarea ?)
- convert command to make it pyramidal
- Place to put it on rep-dev.libraries
- IIPIMage URL to request it.

Also if you want a new version of libtiff, what version should it be?

I have downloaded two different source code versions and also looked
into an OpenSuse rpm (which can;t be used because it require a new
lib.c which can't be upgraded since most other software depends on it.

#7

1) Starting TIFF can be found @rep-devel2:/mellon/jpeg2000/images/issue/source_rep_dev_499.tiff
This TIFF was taken from the DARCH of object rutgers-lib:499 from the rep-dev server

2) convert command is: convert {source file} -define tiff:tile-geometry=256x256 -compress jpeg {output file}
An example run on rep-dev is: convert source_rep_dev_499.tiff -define tiff:tile-geometry=256x256 -compress jpeg linux_rep_dev_499.ptif

The response when running the above command on the 499 TIFF is:

cmmills@rep-devel2:/mellon/jpeg2000/images/issue> convert source_rep_dev_499.tiff -define tiff:tile-geometry=256x256 -compress jpeg linux_rep_dev_499.ptif
convert: source_rep_dev_499.tiff: unknown field with tag 40961 (0xa001) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/891.

3) Place to put it on rep-dev. I assume you mean the output pyramidal TIFF, let me know if you meant something else. I created a directory for this issue @rep-devel2:/mellon/jpeg2000/images/issue/

4) IIPIMage URL to request it. The Base URI is to the FCGI is: http://rep-dev.libraries.rutgers.edu/fcgi-bin/iipsrv.fcgi
An example query would be: http://rep-dev.libraries.rutgers.edu/fcgi-bin/iipsrv.fcgi?FIF=issue/windows_rep_dev_499.ptif&WID=400&CVT=jpeg
Note the FIF parameter points to the windows generated sample. The final two parameters are WID(width) and CVT(format). CVT must always come last.

5) I asked before if you know what version of libtiff is being used in your Windows setup but I don't think you responded.
I can't determine if my Windows workstation is using libtiff DLLs or Visual C++ libraries.

6) Also if you want a new version of libtiff, what version should it be?
I would imagine the most recent stable version available. http://www.remotesensing.org/libtiff/

7) Bonus!
In #2 above I looked into the "warning/tiff.c/TIFFWarnings/891" some more just now. I found this thread that seems to suggest Photoshop might assign a color profile that ImageMagick doesn't like. I ran identify on the "source_rep_dev_499.tiff" and the color profile in question exists.

Profiles:
Profile-8bim: 8984 bytes
Profile-xmp: 5651 bytes

According to this post, https://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=29325, there might be acolorspace issue that is causing the problem. When trying to convert the colorspace new errors are thrown: Invalid tag "Predictor" (not supported by codec). `_TIFFVSetField' @ error/tiff.c/TIFFErrors/557.

Looking into this error it leads to more possible colorspace problems. Only good news is that the above post and other are very recent. Actually within the last couple of weeks. I'll keep looking into this and report any findings.

#8

One thought I am having is that maybe we can install ImageMagick without the libtiff dependency and just draw on ImageMagick's internal dependencies for TIFF support. I was able to make a pyramidal TIFF on a host site I pay for outside of work that doesn't use libtiff as a dependency. I will attach the information from that install to this comment.

@rep-dev/mellon/jpeg2000/images/issue/alt_linux_cmm_host_499.ptif

#9

I realize this isn't a apples to apples comparison and I hope this doesn't add any extra confusion to the situation. Here is the info from my working hosted site.

convert -version
==================================================
<a href="mailto:hpcwhorg@hpcwh.org">hpcwhorg@hpcwh.org</a> [~]# convert -version
Version: ImageMagick 6.7.9-10 2012-10-05 Q16 <a href="http://www.imagemagick.org" title="http://www.imagemagick.org">http://www.imagemagick.org</a>
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP

convert -list configure
==================================================
<a href="mailto:hpcwhorg@hpcwh.org">hpcwhorg@hpcwh.org</a> [~]# convert -list configure

Path: /usr/lib64/ImageMagick-6.7.9/config/configure.xml

Name Value
-------------------------------------------------------------------------------
CC gcc -std=gnu99 -std=gnu99
CFLAGS -fopenmp -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -pthread
CODER_PATH /usr/lib64/ImageMagick-6.7.9/modules-Q16/coders
CONFIGURE ./configure '--build=x86_64-unknown-linux-gnu' '--host=x86_64-unknown-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-shared' '--disable-static' '--with-modules' '--with-perl' '--with-x' '--with-threads' '--with-magick_plus_plus' '--with-gslib' '--with-gs-font-dir=/usr/share/fonts/default/ghostscript' '--with-wmf' '--with-lcms' '--with-rsvg' '--with-xml' '--with-perl-options=INSTALLDIRS=vendor CC='\''gcc -L/usr/src/rpm/BUILD/ImageMagick-6.7.9-10/magick/.libs'\'' LDDLFLAGS='\''-shared -L/usr/src/rpm/BUILD/ImageMagick-6.7.9-10/magick/.libs'\''' '--without-dps' '--without-included-ltdl' '--with-ltdl-include=/usr/include' '--with-ltdl-lib=/usr/lib64' 'build_alias=x86_64-unknown-linux-gnu' 'host_alias=x86_64-unknown-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic'
CONFIGURE_PATH /etc/ImageMagick/
COPYRIGHT Copyright (C) 1999-2012 ImageMagick Studio LLC
CPPFLAGS -I/usr/include/ImageMagick
CXX g++
CXXFLAGS -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -pthread
DEFS -DHAVE_CONFIG_H
DELEGATES bzlib djvu fontconfig freetype gs jpeg jng jp2 lcms pango png rsvg tiff x11 xml wmf zlib
DISTCHECK_CONFIG_FLAGS 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --disable-deprecated --with-quantum-depth=16 --with-umem=no --with-autotrace=no --with-dps=no --with-fontpath= --with-gs-font-dir=/usr/share/fonts/default/ghostscript --with-gvc=no
DOCUMENTATION_PATH /usr/share/doc/ImageMagick-6.7.9
EXEC-PREFIX /usr
EXECUTABLE_PATH /usr/bin
FEATURES OpenMP
FILTER_PATH /usr/lib64/ImageMagick-6.7.9/modules-Q16/filters
HOST x86_64-unknown-linux-gnu
INCLUDE_PATH /usr/include/ImageMagick
LDFLAGS -L/usr/lib64 -L/usr/lib
LIB_VERSION 0x679
LIB_VERSION_NUMBER 6,7,9,10
LIBRARY_PATH /usr/lib64/ImageMagick-6.7.9
LIBS -lMagickCore -llcms -lfreetype -lfontconfig -lXext -lSM -lICE -lX11 -lXt -lbz2 -lz -lm -lgomp -lpthread -lltdl
NAME ImageMagick
PCFLAGS -fopenmp
PREFIX /usr
QuantumDepth 16
RELEASE_DATE 2012-10-05
SHARE_PATH /usr/share/ImageMagick-6.7.9
SVN_REVISION 9513
TARGET_CPU x86_64
TARGET_OS linux-gnu
TARGET_VENDOR unknown
VERSION 6.7.9
WEBSITE <a href="http://www.imagemagick.org" title="http://www.imagemagick.org">http://www.imagemagick.org</a>

Path: [built-in]

Name Value
-------------------------------------------------------------------------------
NAME ImageMagick

ldd convert
============================
<a href="mailto:hpcwhorg@hpcwh.org">hpcwhorg@hpcwh.org</a> [~]# ldd /bin/convert
linux-vdso.so.1 => (0x00007ffc027ee000)
libMagickCore.so.5 => /usr/lib64/libMagickCore.so.5 (0x000000376e800000)
libMagickWand.so.5 => /usr/lib64/libMagickWand.so.5 (0x000000376d800000)
liblcms.so.1 => /usr/lib64/liblcms.so.1 (0x000000376dc00000)
libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x0000003005000000)
libfontconfig.so.1 => /usr/lib64/libfontconfig.so.1 (0x0000003df7400000)
libXext.so.6 => /usr/lib64/libXext.so.6 (0x0000003dfa000000)
libXt.so.6 => /usr/lib64/libXt.so.6 (0x000000376e000000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003003000000)
libz.so.1 => /lib64/libz.so.1 (0x000000300f800000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003000c00000)
libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x0000003009000000)
libSM.so.6 => /usr/lib64/libSM.so.6 (0x0000003dfe000000)
libICE.so.6 => /usr/lib64/libICE.so.6 (0x0000003dfd800000)
libX11.so.6 => /usr/lib64/libX11.so.6 (0x0000003011800000)
libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x000000376e400000)
libm.so.6 => /lib64/libm.so.6 (0x0000003001000000)
libc.so.6 => /lib64/libc.so.6 (0x0000003000400000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003005400000)
libexpat.so.1 => /lib64/libexpat.so.1 (0x0000003012c00000)
/lib64/ld-linux-x86-64.so.2 (0x0000003000000000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003000800000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x0000003dfdc00000)
libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x0000003015800000)
librt.so.1 => /lib64/librt.so.1 (0x0000003001400000)
libXau.so.6 => /usr/lib64/libXau.so.6 (0x0000003011c00000)

#10

Looking at this further last night, on rep-dev.libraries
I downloaded and compiled tiff-4.0.6. and ran make; make install
which gave me convert compiled against

libtiff.so.5 => /usr/local/lib/libtiff.so.5 (0x00007f0faa425000)
(I subsequently copied the libtiff.so.5.2.4 file and made the symlink
of libtiff.so.5 to point to it in /usr/lib64 as /usr/local/lib is not in the
default library search path).

I did a quick test and it looked ok. Today after reading your message
I went back through the steps you outlined using the source file mentioned
and the convert showed no errors (nor did identify -verbose or tiffinfo against
the ptif file). I then placed it in the /mellon/jpeg200/images/issues directory and
was able to use the IIPImage URL to open the file.

So now the directory contains:
-> 2692851 rep-dev-tiff-4.0.6_499.ptif
-> 2692935 sles-12-compiled-tiff-4.0.6_499.ptif
-> 2693763 sles-12-package-tiff-4.0.6_499.ptif

So give it a try on rep-dev.libraries and let me know what you find. It
is comforting to me that SLES 12 seems to already have the right version
of libtiff in place and that both the package version and the compiled version
produce a proper file.

#11

Status:active» fixed

Yes, very cool now. Working as expected on dev. Marking fixed. Thanks for your help and follow through with this item.

#12

Status:fixed» closed

Back to top