Gilles' sysadmin & dev blog for Earth Science

Enabling gdal java binding for Geoserver on macOS

This is a quick guide to enable “gdal” java-binding for Geoserver on macOS.

This will allow Geoserver to read a bunch of different image formats (like EnviHDR files) using GDAL, see here

There are several ways to enable GDAL support for Geoserver 2.x on macOS.

Here are three ways described:

  1. Install gdal 1.x with Homebrew package manager (the easy way)
  2. Compile gdal 2.x with Homebrew lib dependencies (preferred)
  3. Compile gdal 2.x with Kyng-Chaos Frameworks (outdated)

Tested on macOS 10.11.6 with gdal 2.2.2, Geoserver 2.12.0, JAVA SE RE 8.0.151, Tomcat 8.5.23.

Pre-requirements

1. Apple’s Xcode developer tools; install with command: #xcode-select --install

2. Latest “Java 8 SE Runtime Environment” from Oracle, download from:
http://www.oracle.com/technetwork/java/javase/downloads/index.html

3. Homebrew package manager for macOS; for installation see here:
https://brew.sh

4. Edit your ~/.bash_profile or ~/.bashrc and add the following lines:

# For Tomcat Server, increase memory
# Note: Set these variables in /Library/Tomcat/bin/setenv.sh
export CATALINA_HOME=/Library/Tomcat/
export JAVA_HOME=$(/usr/libexec/java_home)
export JRE_HOME=$(/usr/libexec/java_home)
# For geoserver gdal extension
export GDAL_DATA="$HOME/gdal-data/"

5. Download “gdal-data.zip” from:
http://demo.geo-solutions.it/share/github/imageio-ext/releases/1.1.X/1.1.17/native/gdal/

Install the extracted “gdal-data.zip” to your $HOME directory.

Follow the instructions from http://docs.geoserver.org/latest/en/user/data/raster/gdal.html

Install gdal 1.x with Homebrew

This is the easiest way to install gdal.
However, Homebrew installs gdal 1.1.x (as of October 2017). So if you want to install gdal2

1a. Update Homebrew

#brew update

1b. Install gdal 1.x with Homebrew but with swig-java option

#brew install --with-swig-java gdal

Homebrew will install the native gdal (dynamic) libraries in /usr/local/Cellar/gdal/1.11.5_3/lib/
Homebrew will also create a symlink from /usr/local/Cellar/gdal/1.11.5_3/lib/ to /usr/local/lib/

You should see something like:

#ls -logh /usr/local/Cellar/gdal/1.11.5_3/lib/
total 51896
-r--r--r-- 1 101K Mar 22 11:30 gdal.jar
-r--r--r-- 1 8.7M Mar 22 11:30 libgdal.1.dylib
-r--r--r-- 1 16M Mar 22 11:28 libgdal.a
lrwxr-xr-x 1 15B Mar 22 11:28 libgdal.dylib -> libgdal.1.dylib
-r--r--r-- 1 39K Mar 22 11:30 libgdalconstjni.1.dylib
-r--r--r-- 1 39K Mar 22 11:30 libgdalconstjni.dylib
-r--r--r-- 1 190K Mar 22 11:30 libgdaljni.1.dylib
-r--r--r-- 1 190K Mar 22 11:30 libgdaljni.dylib
-r--r--r-- 1 129K Mar 22 11:30 libogrjni.1.dylib
-r--r--r-- 1 129K Mar 22 11:30 libogrjni.dylib
-r--r--r-- 1 80K Mar 22 11:30 libosrjni.1.dylib
-r--r--r-- 1 80K Mar 22 11:30 libosrjni.dylib
drwxr-xr-x 3 102B Mar 22 11:28 pkgconfig
drwxr-xr-x 3 102B Mar 22 11:30 python2.7

1b. Follow the instructions from “Geoserver’s GDAL extension” page

as described here: http://docs.geoserver.org/latest/en/user/data/raster/gdal.html

New since Geoserver 2.12 is that the “imageio-ext-gdal-bindings-1.9.2.jar” java-binding file is back in again in the GDAL-extension directory.

The “imageio-ext-gdal-bindings-1.9.2.jar” file is the Java-Binding file which is used to make the binding with Geoserver and gdal commands.
However “imageio-ext-gdal-bindings-1.9.2.jar” will only work with gdal version 1.x but not with gdal version 2.x !

Install the GDAL-Extension “*.jar” files in $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/

IMPORTANT: Since Homebrew gdal-version has also the “gdal.jar” in /usr/local/lib/ you could copy it to $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/.

Using only one copy gdal java bindings (“gdal.jar” or “imageio-ext-gdal-bindings-1.9.2.jar”) will work.
Just be sure to have just one copy of the java-binding file installed in $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/

1e. Edit Geoserver’s startup script “setenv.sh”

Before starting Geoserver we need to add the JAVA_OPTS of the gdal native libraries, or else Java will fail to load the gdal binding.
Homebrew automatically makes various symlinks of the dynamic gdal libs from /usr/local/lib/ to /usr/local/Cellar/gdal/1.11.5_3/lib

Edit $CATALINA_HOME/bin/setenv.sh and add the following line to it:

# this is setenv.sh for Tomcat
# Setting path for gdal java binding - gdal installed with Homebrew
# Command used: brew install --with-swig-java gdal
export JAVA_OPTS="-Djava.library.path=/usr/local/lib/"

1f. Start Tomcat

Start Geoserver by starting Tomcat first.
To verify if you have the gdal extension properly installed, login to:
http://localhost:8080/geoserver/web/wicket/bookmarkable/org.geoserver.web.data.store.NewDataPage

Browse to “Data” > “Stores” > “Add new Stores” Geoserver should display “ENVIHdr” and other raster format.

2. Compile gdal 2.x with Homebrew dependencies (preferred)

This is the preferred version, since it compiles the latest gdal 2.x version along with its latest dependencies.

2a. Install Homebrew

Copy & paste the following command:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

more info at Homebrew website https://brew.sh

2b. Install gdal 2.x dependencies

#brew install swig
#brew install packagemanager
#brew install jasper
#brew install libtiff
#brew install libjpeg
#brew install libgif
#brew install proj
#brew install libgeotiff
#brew install geos
#brew install curl
#brew install netcdf
#brew install hdf5
#brew install expat

2c. Compile gdal2

Compile gdal2 with –prefix to /usr/local/gdal2. This will install everything in: /usr/local/gdal2/

#cd to ~/Downloads/gdal2-2.2.2/
./configure --prefix=/usr/local/gdal2/ \
--with-threads \
--disable-static \
--without-grass \
--with-jasper=/usr/local/lib \
--with-libtiff=/usr/local/lib \
--with-jpeg=/usr/local/lib\
--with-gif=/usr/local/lib \
--with-png=/usr/local/lib \
--with-geotiff=/usr/local/lib \
--with-pcraster=internal \
--with-geos=/usr/local/lib \
--with-static-proj4=/usr/local/lib \
--with-expat=/usr/local/lib \
--with-curl=/usr/local/lib \
--with-netcdf=/usr/local/lib \
--with-hdf5=/usr/local/lib \
--with-opencl \
--without-python \
--with-java \
--with-jvm-lib=/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home \
--with-libz=internal

#make
#sudo make install

2d. Compile the “gdal.jar” java-binding with its dynamic libraries *.dylib

The “gdal.jar” file is the Java-Binding file which will be used to make the binding with Geoserver 2.x and gdal2.x

IMPORTANT: You must compile and install the dynamic libraries *.dylib to /usr/local/gdal2/lib
Note: The compiled “gdal.jar” java-binding file will not be installed in /usr/local/gdal2/lib
We don’t need to copy “gdal.jar” to /usr/local/lib/ but in $CATALINA_HOME/geoserver/WEB-INF/lib instead

#cd swig/java
#make CFLAGS='-I$(JAVA_HOME)/include/darwin/'
#sudo make install

This will install the dynamic libs *.dylib in /usr/local/gdal2/lib/
#ls -l /usr/local/gdal2/lib/

Should print something like:

libgdalconstjni.20.dylib
libgdalconstjni.dylib
libgdaljni.20.dylib
libgdaljni.dylib
libogrjni.20.dylib
libogrjni.dylib
libosrjni.20.dylib
libosrjni.dylib

IMPORTANT:
Since we are using gdal 2.x the default installed gdal extension “imageio-ext-gdal-bindings-1.9.2.jar” will not work with Geoserver 2.x !! We need to remove “imageio-ext-gdal-bindings-1.9.2.jar” from $CATALINA_HOME/WEB-INF/lib/ and copy our manually compiled “gdal.jar” file to $CATALINA_HOME/WEB-INF/lib/

2e. Copy the “gdal.jar” java-binding to $CATALINA_HOME/geoserver/WEB-INF/lib

#cp gdal-2.2.2/swig/java/gdal.jar $CATALINA_HOME/geoserver/WEB-INF/lib/gdal-2.2.2-java-binding.jar

From Geoserver website:


Warning If you are using a version of GDAL more recent than 1.9.2, replace the “imageio-ext-gdal-bindings-1.9.2.jar” file with the equivalent java binding jar (typically named either gdal.jar or imageio-ext-gdal-bindings-*.jar) included with your GDAL version. If your GDAL version does not include a bindings jar, it was probably not compiled with the java bindings and will not work with GeoServer.

2f. Edit Geoserver’s startup script “setenv.sh”

Before starting Geoserver 2.x, set JAVA_OPTS to point to the location of gdal2 native libraries, or else Java will fail to load the gdal2.x binding.

Edit $CATALINA_HOME/bin/setenv.sh and add the following line to it:

# This is setenv.sh for Tomcat
# Setting path for gdal java binding - gdal2.x installed in /usr/local/gdal2/
# Command used: brew install --with-swig-java gdal
export JAVA_OPTS="-Djava.library.path=/usr/local/gdal2/lib/"

2g. Start Geoserver with Tomcat

Start Geoserver by starting Tomcat first.

To verify if you have the gdal2 extension properly installed, login to:
http://localhost:8080/geoserver/web/wicket/bookmarkable/org.geoserver.web.data.store.NewDataPage

Browse to “Data” > “Stores” > “Add new Stores” Geoserver should display “ENVIHdr” and other raster format.

If it doesn’t work check if you don’t have “imageio-ext-gdal-bindings-1.9.2.jar” in $CATALINA_HOME/WEB-INF/lib/ installed.
If you have just remove it and don’t forget your compiled “gdal.jar” to $CATALINA_HOME/WEB-INF/lib/

3. Compile gdal 2.x with Kyng-Chaos framework (outdated)

Another possibility is to use Kyng-Chaos precompiled UnixImageIO, PROJ and GEOS frameworks to compile gdal 2.x manually.
This chapter is somewhat outdated! Prefer the compilation of gdal2.x with Homebrew dependencies.

Pre-requirements:

  • Xcode developer tools; easiest way is to install with command: #xcode-select --install
  • Homebrew for swig and packagemanager

3a. Install Kyng Chaos precompiled frameworks

We’ll need the following precompiled frameworks to compile gdal 2.x manually:

  • UnixImageIO
  • GEOS framework
  • PROJ framework

Download these frameworks from Kyng Chaos website, here:
http://www.kyngchaos.com/software:frameworks

These frameworks will be installed in /Library/Frameworks/

Check your Java version and path

Verify your installed Java Path; we will need the path for the configure script:
# /usr/libexec/java_home

It should print something like:
/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home

Before compiling gdal from source, install “swig” and “packagemanager” with Homebrew
#brew install swig
#brew install packagemanager

3b. Compile the native gdal2 libraries from source

Download gdal 2.x from:
http://www.gdal.org

Extract the gdal archive, and invoke the ./configure command
Note: we use the --prefix=/usr/local/gdal2 here to install everything in: /usr/local/gdal2

Omitting --prefix will install everything in : /usr/local/gdal2

#cd gdal-2.2.2
#./configure --prefix=/usr/local/gdal2/ --with-threads --disable-static --without-grass --with-jasper=/Library/Frameworks/UnixImageIO.framework/unix --with-libtiff=/Library/Frameworks/UnixImageIO.framework/unix --with-jpeg=/Library/Frameworks/UnixImageIO.framework/unix --with-gif=/Library/Frameworks/UnixImageIO.framework/unix --with-png=/Library/Frameworks/UnixImageIO.framework/unix --with-geotiff=/Library/Frameworks/UnixImageIO.framework/unix --with-pcraster=internal --with-geos=/Library/Frameworks/GEOS.framework/unix/bin/geos-config --with-static-proj4=/Library/Frameworks/PROJ.framework/unix --with-expat --with-python -with-java -with-jvm-lib=/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home --with-libz=internal

Then rebuild again:

#make
#sudo make install

This will compile and install binaries, lib and includes etc. in the directory /usr/local/gdal2/

3c. Build the gdal 2.x java-binding file “gdal.jar”

#cd swig/java
To avoid the
#make CFLAGS='-I$(JAVA_HOME)/include/darwin/'
#sudo make install

This will install the files in /usr/local/gdal2/lib/
#ls -l /usr/local/gdal2/lib/
libgdalconstjni.20.dylib
libgdalconstjni.dylib
libgdaljni.20.dylib
libgdaljni.dylib
libogrjni.20.dylib
libogrjni.dylib
libosrjni.20.dylib
libosrjni.dylib

However the gdal java binding file “gdal.jar” will not be installed in /usr/local/gdal2/lib/

Note: the “gdal.jar” file is the Java-Binding file which can be used to make the binding with Geoserver and gdal.

You must copy “gdal.jar” to $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/ and should rename it to “gdal-2.2.2-java-binding.jar” with command:

#cp gdal.jar $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/gdal-2.2.2-java-binding.jar

IMPORTANT: Geoserver’s GDAL extension contains the file “imageio-ext-gdal-bindings-1.9.2.jar” which shall be removed from $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/.
Don’t install “imageio-ext-gdal-bindings-1.9.2.jar” in $CATALINA_HOME/webapps/geoserver/WEB-INF/lib/ since this version will only work with gdal 1.x versions and not gdal 2.2.x !!

# Setting path for gdal java binding - gdal installed with Homebrew
export JAVA_OPTS="-Djava.library.path=/usr/local/gdal2/lib/"

Start Geoserver via Tomcat, and you should see the ENVIHdr format in Geoserver > Data > “Stores” > “New Stores”

Errors, anyone ?

If you get the error: "jni_md.h is missing" then you can fix it by copying:

/Library/Java/JavaVirtualMachines/jdk.jdk/Contents/Home/include/darwin/jni_md.h
to:
/Library/Java/JavaVirtualMachines/jdk.jdk/Contents/Home/include/jni_md.h

with command:
cp -rp /Library/Java/JavaVirtualMachines/jdk.jdk/Contents/Home/include/darwin/jni_md.h /Library/Java/JavaVirtualMachines/jdk.jdk/Contents/Home/include/