JPEG is Dead, Long Live JPEG!

TL;DR: see this GitHub issue for summary (Dutch but the pictures there and Figure 1 tell the story).

opentopo-png-vs-jpeg

Figure 1. Image encoding-comparison for MapProxy-tiles

Somewhere around 1995, building my first website, it was already quite a feat to embed images. Non-aware of image-formats I played with (animated!) GIF and JPEG. Naively image-editing I noticed that my JPEGs became worse and worse after each save…I learned quickly about lossless and lossy encodings back then.  Later on came PNG. When I entered the geospatial domain there appeared to be a common convention that JPEG was to be used for arial/satellite images and PNG for rasterized vector renderings. So the years went by and all geo-folks, including me, followed that rule.

Recently I developed and launched Map5.nl: a cloud service for topographic maps mostly made with Dutch Open Geo-data. See my previous post Tales from the Topographic Lowlands how this service evolved.  After quite some research I finally settled on an Open Source geo-stack with GDALMapServer and MapProxy on Ubuntu. For raster-map serving this appeared to be a golden combination.  The full stack, including pre-processing of TIFF-files (and some PNGs) is depicted below. The arrows denote the flow of data.

geostack1

Figure 2. Map5.nl dataflow for Raster data serving

In a later post I will dive more into the details of this architecture. For now I should explicitly mention the work of Jan-Willem van Aalst, who designed the OpenTopo-related maps provided on Map5.nl using QGIS.

But this post is about JPEG and how/why I found it needs revival in the context of raster-data preparation and serving. I explicitly mention ‘preparation’ as JPEG was applied at two steps within the stack shown in Figure 2.

JPEG Step 1 – Encoding in GeoTIFF

Within the NLExtract project I was already preprocessing historical maps from PNG (with world files) to GeoTIFF. At some point I found out that GeoTIFF, as a container-image format, supports multiple  image-encodings. Using JPEG encoding it appeared that the resulting GeoTIFF files were much smaller (about 50%)  without hardly any loss in image quality. For the OpenTopo layers, I devised this shell-script  to create GeoTIFFs ready to be served by any WMS-server. Without going into details of the script, this line, using gdal_translate, does the actual JPEG-encoding:

gdal_translate -b 1 -b 2 -b 3 -of GTiff
-co TILED=YES
-co PROFILE=Geotiff -co COMPRESS=JPEG
-co JPEG_QUALITY=95
-co PHOTOMETRIC=YCBCR -co BLOCKXSIZE=512 -co BLOCKYSIZE=512
-a_srs EPSG:28992 $src_tif $dst_tif

Later on, I happily noticed, that others, like the great Paul Ramsey also advertised JPEG encoding in GeoTIFF. So JPEG lived up here.

JPEG Step 2 – Encoding in WMS and Tiles

Still within the rest of the geo-stack used for actual map serving with the MapServer/MapProxy combo I was still obeying the old rule to use PNG for serving non-arial/satellite images. I struggled and tested endless variations in configuration settings for PNG. My goal was to serve small tiles with just enough quality blazingly fast.  Now PNG has many options, but broadly put one has to make a choice between PNG24 (24 bits) or PNG8 (8 bits, 256 colors). The latter uses a colormap encoding which for the rich color variations of the OpenTopo and coloured hillshading layers had quite noticeable degraded image quality. PNG24 on the other hand rendered great tiled images but with the penalty of significant tile-sizes. A Catch-22 situation… Enter JPEG. Configuring MapProxy to serve JPEG-tiles gave much better results but needed some tweaking:

  • use at least faktor 90 JPEG-compression (also in MapServer)
  • disable meta-tiling and buffering, i.e. request 256×256 JPEG maps from the MapServer source

So some excerpts from the MapProxy config:

opentopo_file_cache:
  grids: [geonovum_grid, opentopo_extent_grid]
  sources: [opentopo_wms]
  format: image/jpeg
  meta_buffer: 0
  meta_size: [1,1]
opentopo_wms:
  type: wms
  req:
    url: http://ms.HOST_URL/go?
    layers: opentopo
    format: image/jpeg
    transparent: false
  coverage:
    bbox: [10000.000,299995.559,279997.956,625000.000]
    srs: 'EPSG:28992'

This gave optimal results. JPEG tiles were around 4.5 times as small as PNG24. See the results in Figure 1 above. For the Hillshading layer the differences were striking especially when zoomed-in. See Figure 3 below.

relief_struct-png-vs-jpeg

Figure 3. Tiles and filesizes for different image encodings. Click image for full picture.

So my choice was to settle for JPEG for the topographic and hillshading maps. You can browse all Map5.nl layers in the NLTopo App.

So yes, JPEG seems the most optimal for these type of map-layers, but am I missing something? Some proponed:  “Yes, but JPEG has no transparency nor alpha-channel”. Hmm, true, but does this matter in most modern web-clients like OpenLayers or Leaflet? From what I observed, JPEG-layers will happily obey opacity-settings in these web-clients. For example, Figure 4 below shows the national Dutch Topographic map overlayed with the Map5.nl hillshading layer.

kadaster-top25-relief-struct

Figure 4 – Dutch 1:25000 raster map transparently overlayed with Map5.nl JPEG hillshading layer

So what to conclude? Basically the title of this post should say it. Further I would again like to acknowledge Jan-Willem van Aalst for his outstanding work on OpenTopo maps and Frank Steggink for making the basic hillshading map from the free Dutch Lidar- pointcloud-data (AHN2). And further the developers of MapServer and MapProxy, what an awesome combo. Even without pre-tiling maps are served blazingly fast! I am really fond of the Hillshading map. The Netherlands, known to be “flatland”, can now reveal also its past. See for example figure 5 below, a Roman Fort from about 2000 years ago!

Figure 5 - Contours from a Roman Fort near Speuld

Figure 5 – Contours from a Roman Fort near Speuld

 

 

3 Comments:

  1. Hi.

    Same story here, albeit with a different stack (Geoserver + GeoWebCache).
    JPEG are much, much smaller than PNG24 (sometimes even 10 times smaller).
    So I decided to publish vector data in PNG (to preserve transparency) and raster data (ortophoto, satellite images, geological maps, ecc…) in JPEG.

    Now I would like to experiment with palette images (png8 with custom palette) for vector data, as soon as I have some spare time.
    See http://docs.geoserver.org/latest/en/user/tutorials/palettedimage/palettedimage.html.

    There should be something similar for Mapserver.

    Bye, Stefano.

    • Just van den Broecke

      Yes, I am aware of the possibilities of GeoServer w.r.t. PNG(8) encoding in other projects. Their recent encoder in Java, think from GeoSolutions, gave very good results. In the MapProxy/MapServer-stack I had MapServer returning PNG24 via WMS and MapProxy (via PIL(Low)) do the PNG8 encoding. I now realize that my OpenTopo and Hillshading maps are really a sort of photographic pictures mainly because of the rich hillshading details. But still I see a lot of PNG24 tiles being used for these type of maps… Best, Just

  2. Pingback: JPEG is Dead, Long Live JPEG! || #GIS #JPEG #JPG #PNG #GeoTiff | Geo-How-To News

Leave a Reply

Your email address will not be published.