JPEG is Dead, Long Live JPEG!

TL;DR: see this GitHub issue for summary (Dutch but the pictures there and Figure 1 tell the story).

Somewhere around 1995, building my first website, it was already quite a feat to embed images. Non-aware of image-formats I played with (animated!) GIF and JPEG. Naively image-editing I noticed that my JPEGs became worse and worse after each save…I learned quickly about lossless and lossy encodings back then. Later on came PNG. When I entered the geospatial domain there appeared to be a common convention that JPEG was to be used for arial/satellite images and PNG for rasterized vector renderings. So the years went by and all geo-folks, including me, followed that rule.

Recently I developed and launched Map5.nl : a cloud service for topographic maps mostly made with Dutch Open Geo-data. See my previous post Tales from the Topographic Lowlands how this service evolved. After quite some research I finally settled on an Open Source geo-stack with GDAL , MapServer and MapProxy on Ubuntu . For raster-map serving this appeared to be a golden combination. The full stack, including pre-processing of TIFF-files (and some PNGs) is depicted below. The arrows denote the flow of data.

In a later post I will dive more into the details of this architecture. For now I should explicitly mention the work of Jan-Willem van Aalst , who designed the OpenTopo-related maps provided on Map5.nl using QGIS .

But this post is about JPEG and how/why I found it needs revival in the context of raster-data preparation and serving. I explicitly mention ‘preparation’ as JPEG was applied at two steps within the stack shown in Figure 2.

JPEG Step 1 – Encoding in GeoTIFF

Within the NLExtract project I was already preprocessing historical maps from PNG (with world files) to GeoTIFF. At some point I found out that GeoTIFF, as a container-image format, supports multiple image-encodings. Using JPEG encoding it appeared that the resulting GeoTIFF files were much smaller (about 50%) without hardly any loss in image quality. For the OpenTopo layers, I devised this shell-script to create GeoTIFFs ready to be served by any WMS-server. Without going into details of the script, this line, using gdal_translate , does the actual JPEG-encoding:

1gdal_translate -b 1 -b 2 -b 3 -of GTiff  
2 -co TILED=YES  
3 -co PROFILE=Geotiff -co COMPRESS=JPEG  
4 -co JPEG_QUALITY=95  
5 -co PHOTOMETRIC=YCBCR -co BLOCKXSIZE=512 -co BLOCKYSIZE=512  
6 -a\_srs EPSG:28992 $src\_tif $dst_tif

Later on, I happily noticed, that others, like the great Paul Ramsey also advertised JPEG encoding in GeoTIFF . So JPEG lived up here.

JPEG Step 2 – Encoding in WMS and Tiles

Still within the rest of the geo-stack used for actual map serving with the MapServer/MapProxy combo I was still obeying the old rule to use PNG for serving non-arial/satellite images. I struggled and tested endless variations in configuration settings for PNG. My goal was to serve small tiles with just enough quality blazingly fast. Now PNG has many options, but broadly put one has to make a choice between PNG24 (24 bits) or PNG8 (8 bits, 256 colors). The latter uses a colormap encoding which for the rich color variations of the OpenTopo and coloured hillshading layers had quite noticeable degraded image quality. PNG24 on the other hand rendered great tiled images but with the penalty of significant tile-sizes. A Catch-22 situation … Enter JPEG. Configuring MapProxy to serve JPEG-tiles gave much better results but needed some tweaking:

• use at least faktor 90 JPEG-compression (also in MapServer)
• disable meta-tiling and buffering, i.e. request 256×256 JPEG maps from the MapServer source

So some excerpts from the MapProxy config:

 1opentopo_file_cache:
 2  grids: [geonovum_grid, opentopo_extent_grid]
 3  sources: [opentopo_wms]
 4  format: image/jpeg
 5  meta_buffer: 0
 6  meta_size: [1,1]
 7
 8 opentopo_wms:
 9  type: wms
10  req:
11    url: http://ms.HOST_URL/go?
12    layers: opentopo
13    format: image/jpeg
14    transparent: false
15  coverage:
16    bbox: [10000.000,299995.559,279997.956,625000.000]
17    srs: 'EPSG:28992'

This gave optimal results. JPEG tiles were around 4.5 times as small as PNG24. See the results in Figure 1 above. For the Hillshading layer the differences were striking especially when zoomed-in. See Figure 3 below.

So my choice was to settle for JPEG for the topographic and hillshading maps. You can browse all Map5.nl layers in the NLTopo App.

So yes, JPEG seems the most optimal for these type of map-layers, but am I missing something? Some proponed: “Yes, but JPEG has no transparency nor alpha-channel”. Hmm, true, but does this matter in most modern web-clients like OpenLayers or Leaflet ? From what I observed, JPEG-layers will happily obey opacity-settings in these web-clients. For example, Figure 4 below shows the national Dutch Topographic map overlayed with the Map5.nl hillshading layer.

So what to conclude? Basically the title of this post should say it. Further I would again like to acknowledge Jan-Willem van Aalst for his outstanding work on OpenTopo maps and Frank Steggink for making the basic hillshading map from the free Dutch Lidar- pointcloud-data (AHN2 ). And further the developers of MapServer and MapProxy, what an awesome combo. Even without pre-tiling maps are served blazingly fast! I am really fond of the Hillshading map. The Netherlands, known to be “flatland”, can now reveal also its past. See for example figure 5 below, a Roman Fort from about 2000 years ago!