Adiós 2021

Jan 4, 2022 · 13 min read · FOSS FOSS4G GeoFabriek GeoHealthCheck geoqos GeoSpatial geotoko.nl GIS GroteGeoShow Map5.nl NLExtract OSgeo OSGeo.nl python QGIS RTMP Streaming StreamYard  ·
Share on:

Overview of my professional life in 2021. Highlights of living and working in the Open Source Geospatial and OSGeo(.nl |.org )-world in 2021. Organized by "Theme".

Like 2020, 2021 was again a "special year" in pandemic terms. How to cope? Like Paul Ramsey writes: I feel like sharing some of my favourite things . Will do in a later post: Bird-watching , Wolf-tracking workshop , 'catching' big game on new wildcam, developed outdoor GPS-game (Veluwe Big Five), some progress on guitar-playing, discovering The Netherlands outback, becoming a Billy Strings fan. These are a few of my favourite things, more later....

TL;DR Main three 2021 highlights:

Below follow 2021-highlights organized by theme and/or project.

1. Fun Home Automation Projects

Some of my 2022 fun resolutions resolved in 2021:

Some mammals on the Wildcam

More on these later.

2. New Website

For years my website/blog justobjects.nl ran on Wordpress at the famous, and first (est. 1993), Dutch internet/hosting provider XS4ALL . XS4ALL was sold to KPN (1998), remained an independent brand, but was finally dismantled in 2021.

So in early 2021 a new hosting provider was needed. This was also a good moment to migrate my website to Hugo . With the great effort by @peter-govind (via freelance.com ), we converted the complete site (about 20 years of blogs) to Hugo+Github , using GitHub Workflows to "publish-on-commit". My first piece of GitOps, more to follow, in 2021. Though there are still some fixes to do, very happy with the result. Maintaining is now a breeze.

3. GitOps

Gradually GitOps became my main theme in 2021. Discovered that there was actually a name (GitOps) for CI/CD practices I (you) have been doing over the past years. For example, in 2019 I worked on the pygeoapi demo server . GitOps can always be taken further until reaching the phase: "look ma, no hands!". No more "Box Hugging", i.e. no more SSH-ing, manual actions on VMs etc.

So what is GitOps? Many definitions are available. Often tied to Kubernetes (K8s), but GitOps is broader than K8s. The most concise I found at RedHat :

"GitOps is a set of practices to manage infrastructure and application configurations using Git, an open source version control system. GitOps works by using Git as a single source of truth for declarative infrastructure and applications."

I often quote here: "The Truth is in Git!". Especially now in times of fake-news, I love speaking those words in presentations. The quest of humanity is over: We Found The Truth, It Is In Git :-). Pun aside, it takes effort to have 100% of your deployment sources and configs in Git. In particular all required credentials (passwords, keys etc). Think of storing credentials in a public GitHub repo, how? Did some amazing discoveries in that area like (combining) GitHub Secrets and Ansible Vault .

I was very lucky that my main three contracts in 2021 involved GitOps. One project sort of followed the other.

Geonovum

It all began in June at Geonovum who started an OGC API Testbed as part of their ongoing Dutch API Strategy theme.

Together with GeoCat Paul van Genuchten , developed a full-stack deployment for several OGC API Features implementations. The picture below depicts the stack elements:

GitOps Deployment - Operational Stack

The challenge was to provide a testbed environment where third parties could further extend and experiment. Eventually we developed a deployment fully based on GitOps within GitHub as a GitHub Template repo. You can read details in the documentation . The picture below depicts the GitOps workflow:

GitOps Workflow

These results were presented at several occasions in 2021. See the Presentations below.

Geocat

For GeoCat (again with Paul van Genuchten), exploring and setting out how to further migrate to full GitOps for the GeoCat Live environment.

European Commission - Joint Research Centre

Was already working on a contract with EC-JRC for pan-EU data sharing as a member of the "EU JRC ELISE Pool of Experts". An honour to be (still) working with all these great minds from the international geospatial world!

My focus within the team was/is on data services deployment. My pitch and first deliverable, was called "The Power of Orchestrated Containerisation for Data Services".

The full outcome of the project will be published (publicly) as a JRC Technical Report (JRC127730?) titled: "Emerging approaches for data-driven innovation in Europe". For this report, wrote the chapter "Deployment of an OGC API Features web service using GitOps"

Apart from reporting in text, also provided a full-stack OGC API Features service based on pygeoapi . Using the Geonovum OGC API GitHub Template on a fresh Ubuntu VM provided by JRC, this was a great Proof-of-Concept: took just 2 hours to get the complete stack, including stuff like SSL-certificates, Traefik proxy (love Traefik!), OS-hardening etc, up and running . For now the GitHub repo is at github.com/justb4/ogc-api-jrc/ .

pygeoapi for EC JRC

4. OSGeo.nl

FOSS4G-NL 2021 - IRL

Yes, it happened! In between COVID-Waves a LOC lead by the great Erik Meerburg achieved to organize an in-real-life (IRL) 4th OSGeo.nl FOSS4G-NL conference . Abiding with COVID-regulations at the time. At ITC Enschede on October 19-20 2021. Using 2G access. Zero COVID-cases after.

One day of Workshops followed by a one-day conference with great international keynotes by Amelie A Gagnon from UNESCO-IIEP and Kurt Menke from Septima DK , plus three parallel sessions. We are still preparing the website to publish all presentations.

Group Photo FOSS4G-NL 2021

Looks unreal this group photo, but this was in 2021! Thanks to all who made this event a success!

The Big Geo Show

This was already my 2020 highlight. You can watch back all shows via tv.osgeo.nl or directly on the OSGeo.nl YouTube Channel . Hell, you can even binge-watch all episodes !

Episode 15 - OpenStreetMap Special with Ilya Zverev

In 2021 we did six shows, each a Special:

Episode 16 - GIS with your Keyboard

With a real instant-music-composer-performer, Luciën Greefkes in our team, we together made a new DGGS leader video .

OSGeo.nl - Ten Year Anniversary

OSGeo.nl was established on Dec 2, 2011. So 10 years, still going strong! On dec 2, 2021 we were in a new wave, so a small online event. Below the epic picture (by Arnulf Christl) taken at the GIN Congres Utrecht 2011 where we got together to start OSGeo.nl.

OSGeo.nl Founding Mothers and Fathers - 2011

A special episode of the Big Geo Show presents: Ten Years of OSGeo.nl - Story in Pictures .

5. Geospatial Cloud Services

Moved into providing Geospatial Cloud Services last few years, both as a source of income and to support/strengthen underlying open source projects with which they are developed. Warning: shameless ads below. All below was ongoing in 2021.

  • map5.nl is a subscription service for Dutch topographic, historical- and embellished hill-shade and arial maps I started to host in 2015. In 2021 work started for a map5.nl user portal.
  • GeoQoS.com is a hosted GeoHealthCheck (GHC) service on a subscription basis. GHC is an uptime and QoS monitor for (OGC) web services. Customers get their own GHC instance. GeoQoS.com saves the burden of self-hosting GHC. In 2021 started work on a GHC API, using FastAPI , and a new GHC-UI.
  • geotoko.nl , in short GeoToko, is basically a webshop to download Dutch open geo-datasets. One may ask: Dutch geospatial data like Topography, Addresses and Buildings, is already open and publicly available, mainly via Kadaster-PDOK . So why bother reselling? Explanation follows.

Most Dutch Open geospatial datasets, available from governmental institutions like Kadaster, are provided in a neutral exchange-format. In practice: GML (Application Schema). Most users are not able to directly digest complex GML in their applications (or don't want to spend time on that). For over 10 years we run the Open Source project NLExtract to convert these datasets into formats like PostGIS and CSV that can be directly used in applications. NLExtract itself builds on Stetl , an Open Source geospatial ETL-library in Python. NLExtract/Stetl is used a lot, but still will require users to install/maintain an NLExtract/Stetl installation, download datasets, run the ETL, check results etc. That may also be a bridge too far, if e.g. all a customer needs is a one-time CSV of say, all 10 million addresses in The Netherlands. So GeoToko taps into this niche, providing ready-to-use, often enriched Dutch datasets. Pricing is reasonable, compared to other providers, plus organizations and individuals may get highly reduced pricing when they work on Open data applications like OpenStreetMap.

As for development: the GeoToko webshop was developed with Django, Flask/nginx (Download management), Stripe (backend payments) and CKAN . The latter to provide a product-catalogue, sample data and metadata in general, all via GeoCatalogus.nl .

GeoFabriek - Production Chain

Further development concerned automating the NLExtract/Stetl ETL-processes. For this a framework called GeoFabriek, "GeoFactory", was developed. This automates the entire chain from checking new dataset versions at Kadaster, through downloading, conversions, packing for download and updating the metadata (in GeoCatalogus.nl).

6. Open Source Contributions

Continuous work as a contributor. Apart from some GitLab Projects, you can find/follow me best on GitHub .

GitHub contributions

New in 2021:

Ongoing in 2021:

More Contributions - Handy Docker Images

To support many of the Cloud services and Open Source projects, I develop and maintain handy Docker Images , also available from my DockerHub . Ongoing in 2021. In progress of moving all to Github Workflows build/push, replacing DockerHub Autobuild (still awaiting DockerHub Open Source waiver...):

7. Conferences - Attended

FOSS4G Buenos Aires needs special mention. As an online conference, apart from amazing content, I was very impressed with the technical and organizational setup. Even the social events were fun to join. Folks are tired of Zoom, Teams, etc but the FOSS4G LOC had introduced tools like Work Adventure and StreamYard (also used by OSgeo.nl in Big Geo Show) to enable more "cozyness". Offcourse nothing can replace a real event, but this came close. The Gala Dinner was really fun!

Jody Garnett words it very well in his Tweet:

Thank You!

Never forget, always in our hearts: Malena Libman (1983-2021). FOSS4G 2021 was also a conference dedicated to, and in memory of, Malena Libman. She was a pillar of the FOSS4G community and a leading organizer of this conference.

Malena - Photo from mappery.org

Malena was the right and truthful winner of the Sol Katz Award 2021. As the OSGeo.org website states : "Malena Libman is the Sol Katz Award winner 2021. The prize was presented at the FOSS4G closing session and handed over to her friends & family who accepted the award on her behalf since we sadly lost Malena in April 2021 to COVID-19."

8. Talks & Workshops - Provided

Below talks and workshops I provided in 2021. A complete list of presentations is available.

  • "Ten Years of OSGeo.nl - A Story in Pictures" - De Grote Geo Show - Episode 19 (video) - December 2, 2021 - Online - [Video recording on YouTube] - [Episode Info] .

  • "OGC API Features" - Geonovum - Kennisplatform APIs - Dutch API Strategy - Geo-extension WG - November 24, 2021 - Online - [PDF Slides by OGC] .

  • "FOSS4G-NL 2021 - Closing Words" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede - [PDF Slides] .

  • "Zo gemakkelijk kun je geo-webdiensten uitrollen!" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede - On GitOps for OSGeo Software - [PDF Slides] .

  • "GeoHealthCheck - A Quality of Service Monitor for Geospatial Web Services" - FOSS4G 2021 - September 30, 2021 - [HTML Slides] - [Abstract] - [Video recording on YouTube] .

  • "Doing Geospatial in Python" - FOSS4G 2021 - September 28, 2021 - Workshop (4h): introduction to performing common GIS/geospatial tasks using Python geospatial tools such as OWSLib, Shapely, Fiona/Rasterio, and common geospatial libraries like GDAL, PROJ, pycsw, as well as other tools from the geopython toolchain. - [HTML Startpage] - [Abstract] .

  • "Geonovum OGC API Testbed" - Open Geodag 2021 - September 14, 2021 - [PDF Slides] - [Video Recording] .

  • "Geonovum OGC API Testbed" - DiS Online: OGC en toegankelijke APIs - September 7, 2021 - [Event en PDF Slides] .

  • "Read BAG (Dutch Buildings & Addresses dataset) with Docker in 5 minutes" - 2021-04-08 - De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Ep. 16 "GIS met je Toetsenbord". Aired on april 8, 2021. Video recording on YouTube .

  • "OSGeo.nl New Year's Party - Slides OSGeo.nl" - 2021-01-24 - Presentation I gave on behalf of OSGeo.nl on the online joint QGIS-NL, OpenStreetMap-NL and OSGeo.nl New Year's party on January 24, 2021.- [Slideshare]

Geopython Workshop at FOSS4G Buenos Aires

Not as much fun as in 2019 Bukarest, but providing the Geopython Workshop at FOSS4G Buenos Aires with these great minds is still a 2021-highlight.

9. Resolutions for 2022

  • Customer portal for map5.nl
  • Get back into Kubernetes
  • Advance further into full-lifecycle GitOps
  • Advance further into Vector Tiling
  • More effort into Wegue project
  • Revive old GeoTracing projects like georambling.com
  • More favourite things
  • Whatever comes around.