Just Objects on Clarity
https://justobjects.nl/
Recent content in Just Objects on Clarity
Hugo -- gohugo.io
en-us
Copyright © 2008–2018, Steve Francia and the Hugo Authors; all rights reserved.
Thu, 12 Oct 2023 12:49:12 +0000
-
Adiós 2021
https://justobjects.nl/adios-2021/
Tue, 04 Jan 2022 13:24:25 +0000
https://justobjects.nl/adios-2021/
<p>Overview of my professional life in 2021. Highlights of living and working in the Open Source Geospatial and OSGeo(<a href="https://osgeo.nl/" target="_blank" rel="noreferrer noopener">.nl</a>
|.<a href="https://www.osgeo.org/" target="_blank" rel="noreferrer noopener">org</a>
)-world in 2021.
Organized by "Theme".</p>
<p>Like 2020, 2021 was again a "special year" in pandemic terms. How to cope?
Like Paul Ramsey writes: <a href="http://blog.cleverelephant.ca/2022/01/things.html" target="_blank" rel="noreferrer noopener">I feel like sharing some of my favourite things</a>
.
Will do in a later post: <a href="https://waarneming.nl/users/146535/observations/" target="_blank" rel="noreferrer noopener">Bird-watching</a>
, <a href="https://www.weylintracking.nl/workshop-wolvensporen/" target="_blank" rel="noreferrer noopener">Wolf-tracking workshop</a>
, 'catching' big game on new wildcam, developed outdoor GPS-game (Veluwe Big Five), some progress on guitar-playing,
discovering The Netherlands outback, becoming
a <a href="https://www.billystrings.com/" target="_blank" rel="noreferrer noopener">Billy Strings</a>
fan. These are a few of my favourite things, more later....</p>
<p>TL;DR Main three 2021 highlights:</p>
<ul>
<li>Organizing and attending an in-real-life <a href="https://foss4g.nl/" target="_blank" rel="noreferrer noopener">FOSS4G-NL</a>
with OSGeo.nl</li>
<li>Attending, presenting, moderating, online <a href="https://2021.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G Buenos Aires</a>
</li>
<li>Deep-diving into <a href="https://www.redhat.com/en/topics/devops/what-is-gitop" target="_blank" rel="noreferrer noopener">GitOps</a>
.</li>
</ul>
<p>Below follow 2021-highlights organized by theme and/or project.</p>
<h2 id="1-fun-home-automation-projects">1. Fun Home Automation Projects</h2>
<p>Some of my 2022 fun resolutions resolved in 2021:</p>
<ul>
<li>Installed an <a href="https://www.froggit.de/product_info.php?language=en&info=p436_hp1000se-pro-wi-fi-internet-wireless-weather-station.html" target="_blank" rel="noreferrer noopener">HP1000SE PRO Wi-Fi Internet Wireless Weather Station</a>
. Using <a href="https://github.com/justb4/weatherstation" target="_blank" rel="noreferrer noopener">weewx on Raspberry Pi</a>
. Publishing a.o. to <a href="https://www.wunderground.com/dashboard/pws/IGELDERL153" target="_blank" rel="noreferrer noopener">Weather Underground</a>
.</li>
<li>Wildcam</li>
</ul>
<div class="">
<a href="">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/wildcam.jpg"
alt="Some mammals on the Wildcam" />
</a>
</div>
<p>More on these later.</p>
<h2 id="2-new-website">2. New Website</h2>
<p>For years my website/blog <a href="https://justobjects.nl" target="_blank" rel="noreferrer noopener">justobjects.nl</a>
ran on Wordpress at the famous, and first (est. 1993),
Dutch internet/hosting provider <a href="https://en.wikipedia.org/wiki/XS4ALL" target="_blank" rel="noreferrer noopener">XS4ALL</a>
. XS4ALL was sold to KPN (1998), remained an independent brand,
but was finally dismantled in 2021.</p>
<p>So in early 2021 a new hosting provider was needed.
This was also a good moment to migrate my website to <a href="https://gohugo.io/" target="_blank" rel="noreferrer noopener">Hugo</a>
.
With the great effort by <a href="https://github.com/peter-govind" target="_blank" rel="noreferrer noopener">@peter-govind</a>
(via <a href="https://freelance.com" target="_blank" rel="noreferrer noopener">freelance.com</a>
), we converted the complete site (about 20 years of blogs)
to <a href="https://github.com/justb4/justobjects.nl" target="_blank" rel="noreferrer noopener">Hugo+Github</a>
, using GitHub Workflows to "publish-on-commit". My first piece of GitOps, more to follow, in 2021.
Though there are still some fixes to do, very happy with the result. Maintaining is now a breeze.</p>
<h2 id="3-gitops">3. GitOps</h2>
<p>Gradually <a href="https://www.weave.works/technologies/gitops/" target="_blank" rel="noreferrer noopener">GitOps</a>
became my main theme in 2021.
Discovered that there was actually a name (GitOps) for CI/CD practices I (you) have been doing over the past years.
For example, in 2019 I worked on the <a href="https://github.com/geopython/demo.pygeoapi.io" target="_blank" rel="noreferrer noopener">pygeoapi demo server</a>
. GitOps can always be taken further
until reaching the phase: "look ma, no hands!". No more "Box Hugging", i.e. no more SSH-ing, manual actions on VMs etc.</p>
<p>So what is GitOps? Many definitions are available. Often <a href="https://www.manning.com/books/gitops-and-kubernetes" target="_blank" rel="noreferrer noopener">tied to Kubernetes</a>
(K8s), but GitOps is broader than K8s.
The most concise I found at <a href="https://www.redhat.com/en/topics/devops/what-is-gitops" target="_blank" rel="noreferrer noopener">RedHat</a>
:</p>
<p><em>"GitOps is a set of practices to manage infrastructure and application configurations using Git, an open source version control system. GitOps works by using Git as a single source of truth for declarative infrastructure and applications."</em></p>
<p>I often quote here: <strong>"The Truth is in Git!"</strong>. Especially now in times of fake-news, I love speaking those words in presentations.
The quest of humanity is over: We Found The Truth, It Is In Git :-).
Pun aside, it takes effort to have 100% of your deployment sources and configs in Git. In particular all required credentials
(passwords, keys etc). Think of storing credentials in a public GitHub repo, how? Did some amazing discoveries in that area like
(combining) GitHub Secrets and <a href="https://docs.ansible.com/ansible/latest/cli/ansible-vault.html" target="_blank" rel="noreferrer noopener">Ansible Vault</a>
.</p>
<p>I was very lucky that my main three contracts in 2021 involved GitOps. One project sort of followed the other.</p>
<h3 id="geonovum">Geonovum</h3>
<p>It all began in June at <a href="https://www.geonovum.nl/themas/testbed-ogc-apis" target="_blank" rel="noreferrer noopener">Geonovum</a>
who started an OGC API Testbed as part of
their ongoing <a href="https://www.geonovum.nl/themas/kennisplatform-apis" target="_blank" rel="noreferrer noopener">Dutch API Strategy</a>
theme.</p>
<p>Together with <a href="https://geocat.net" target="_blank" rel="noreferrer noopener">GeoCat</a>
<a href="https://www.linkedin.com/in/paulvangenuchten/" target="_blank" rel="noreferrer noopener">Paul van Genuchten</a>
, developed a full-stack deployment for several <a href="https://ogcapi.ogc.org/features/" target="_blank" rel="noreferrer noopener">OGC API Features</a>
implementations.
The picture below depicts the stack elements:</p>
<div class="">
<a href="https://apitestdocs.geonovum.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/gitops/services-stack.png"
alt="GitOps Deployment - Operational Stack" />
</a>
</div>
<p>The challenge was to provide a testbed environment where third parties could further extend and experiment.
Eventually we developed a deployment fully based on <a href="https://github.com/Geonovum/ogc-api-testbed" target="_blank" rel="noreferrer noopener">GitOps within GitHub</a>
as a GitHub Template repo.
You can read details in the <a href="https://apitestdocs.geonovum.nl/" target="_blank" rel="noreferrer noopener">documentation</a>
.
The picture below depicts the GitOps workflow:</p>
<div class="">
<a href="https://apitestdocs.geonovum.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/gitops/schema.png"
alt="GitOps Workflow" />
</a>
</div>
<p>These results were presented at several occasions in 2021. See the Presentations below.</p>
<h3 id="geocat">Geocat</h3>
<p>For GeoCat (again with Paul van Genuchten), exploring and setting out how to further migrate to full GitOps
for the <a href="https://www.geocat.net/live/" target="_blank" rel="noreferrer noopener">GeoCat Live</a>
environment.</p>
<h3 id="european-commission---joint-research-centre">European Commission - Joint Research Centre</h3>
<p>Was already working on a contract with <a href="https://ec.europa.eu/info/departments/joint-research-centre_en" target="_blank" rel="noreferrer noopener">EC-JRC</a>
for pan-EU data sharing as a member of the "EU JRC ELISE Pool of Experts".
An honour to be (still) working with all these great minds from the international geospatial world!</p>
<p>My focus within the team was/is on data services deployment. My pitch and first deliverable, was called
<em>"The Power of Orchestrated Containerisation for Data Services"</em>.</p>
<p>The full outcome of the project will be published (publicly) as a JRC Technical Report (JRC127730?) titled:
<em>"Emerging approaches for data-driven innovation in Europe"</em>. For this report, wrote the
chapter <em>"Deployment of an OGC API Features web service using GitOps"</em></p>
<p>Apart from reporting in text, also provided a full-stack OGC API Features service based on <a href="https://pygeoapi.io" target="_blank" rel="noreferrer noopener">pygeoapi</a>
.
Using the <a href="https://github.com/Geonovum/ogc-api-testbed" target="_blank" rel="noreferrer noopener">Geonovum OGC API GitHub Template</a>
on a fresh Ubuntu VM provided by JRC, this was a great Proof-of-Concept: took just 2 hours to
get the complete stack, including stuff like SSL-certificates, <a href="https://traefik.io/" target="_blank" rel="noreferrer noopener">Traefik</a>
proxy (love Traefik!), OS-hardening etc, <a href="https://jrc.map5.nl/" target="_blank" rel="noreferrer noopener">up and running</a>
.
For now the GitHub repo is at <a href="https://github.com/justb4/ogc-api-jrc/" target="_blank" rel="noreferrer noopener">github.com/justb4/ogc-api-jrc/</a>
.</p>
<div class="">
<a href="https://apitestdocs.geonovum.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/gitops/pygeoapi-jrc.png"
alt="pygeoapi for EC JRC" />
</a>
</div>
<h2 id="4-osgeonl">4. OSGeo.nl</h2>
<h3 id="foss4g-nl-2021---irl">FOSS4G-NL 2021 - IRL</h3>
<p>Yes, it happened! In between COVID-Waves a LOC lead by the great <a href="https://www.linkedin.com/in/erikmeerburg/" target="_blank" rel="noreferrer noopener">Erik Meerburg</a>
achieved to organize
an in-real-life (IRL) <a href="https://foss4g.nl" target="_blank" rel="noreferrer noopener">4th OSGeo.nl FOSS4G-NL conference</a>
.
Abiding with COVID-regulations at the time. At ITC Enschede on October 19-20 2021. Using 2G access.
Zero COVID-cases after.</p>
<p>One day of Workshops followed by a one-day conference with great international keynotes
by <a href="https://www.linkedin.com/in/amelieagagnon/" target="_blank" rel="noreferrer noopener">Amelie A Gagnon from UNESCO-IIEP</a>
and
<a href="https://www.linkedin.com/in/kurtmenke" target="_blank" rel="noreferrer noopener">Kurt Menke from Septima DK</a>
, plus three parallel sessions.
We are still preparing the website to publish all presentations.</p>
<div class="">
<a href="https://foss4g.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/foss4gnl/group-photo.jpg"
alt="Group Photo FOSS4G-NL 2021" />
</a>
</div>
<p>Looks unreal this group photo, but this was in 2021!
Thanks to all who made this event a success!</p>
<h3 id="the-big-geo-show">The Big Geo Show</h3>
<p>This was already my 2020 highlight. You can watch back all shows via <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">tv.osgeo.nl</a>
</strong> or directly on
the <strong><a href="https://www.youtube.com/channel/UCvSAN6ur4RoGUqxtvmgsb8g" target="_blank" rel="noreferrer noopener">OSGeo.nl YouTube Channel</a>
</strong>. Hell, you can even <strong><a href="https://www.youtube.com/playlist?list=PLJMEnRQpAfZqCkhGh3lb3KUnXssK7Sk6C" target="_blank" rel="noreferrer noopener">binge-watch all episodes</a>
</strong>!</p>
<div class="">
<a href="https://youtu.be/nhlIqMyBIig">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/dggs/osm.jpg"
alt="Episode 15 - OpenStreetMap Special with Ilya Zverev" />
</a>
</div>
<p>In 2021 we did six shows, each a Special:</p>
<ul>
<li>Episode 14: <a href="https://www.youtube.com/watch?v=WNowA03zQSo" target="_blank" rel="noreferrer noopener">Newyear with QGIS-NL and OpenStreetMap-NL communities</a>
.</li>
<li>Episode 15: <a href="https://youtu.be/nhlIqMyBIig" target="_blank" rel="noreferrer noopener">OpenStreetMap Special</a>
. Special guest: Ilya Zverev a.k.a. <a href="https://wiki.openstreetmap.org/wiki/User:Zverik" target="_blank" rel="noreferrer noopener">Zverik</a>
!</li>
<li>Episode 16: <a href="https://youtu.be/iOiliIAorD4" target="_blank" rel="noreferrer noopener">GIS with your Keyboard</a>
</li>
<li>Episode 17: <a href="https://youtu.be/caOn96p3cYQ" target="_blank" rel="noreferrer noopener">On Geospatial Databases</a>
</li>
<li>Episode 18: <a href="https://youtu.be/8bPvPFFVlg8" target="_blank" rel="noreferrer noopener">Cartography Special</a>
</li>
<li>Episode 19: <a href="https://youtu.be/yMAxQEg0fSw" target="_blank" rel="noreferrer noopener">Ten Years of OSGeo.nl - Story in Pictures</a>
</li>
</ul>
<div class="">
<a href="https://youtu.be/iOiliIAorD4">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/dggs/gis-keyboard.jpg"
alt="Episode 16 - GIS with your Keyboard" />
</a>
</div>
<p>With a real instant-music-composer-performer, <a href="https://www.muziektaal.nl/" target="_blank" rel="noreferrer noopener">Luciën Greefkes</a>
in our team, we together made a <a href="https://www.youtube.com/watch?v=ZrfrcpWR29s" target="_blank" rel="noreferrer noopener">new DGGS leader video</a>
.</p>
<h3 id="osgeonl---ten-year-anniversary">OSGeo.nl - Ten Year Anniversary</h3>
<p>OSGeo.nl was established on Dec 2, 2011. So 10 years, still going strong!
On dec 2, 2021 we were in a new wave, so a small online event. Below the epic picture (by Arnulf Christl)
taken at the GIN Congres Utrecht 2011 where we got together to start OSGeo.nl.</p>
<div class="">
<a href="https://osgeo.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/osgeonl-founding.jpg"
alt="OSGeo.nl Founding Mothers and Fathers - 2011" />
</a>
</div>
<p>A special episode of the Big Geo Show presents: <a href="https://youtu.be/yMAxQEg0fSw" target="_blank" rel="noreferrer noopener">Ten Years of OSGeo.nl - Story in Pictures</a>
.</p>
<h2 id="5-geospatial-cloud-services">5. Geospatial Cloud Services</h2>
<p>Moved into providing Geospatial Cloud Services last few years,
both as a source of income and to support/strengthen underlying open source
projects with which they are developed. Warning: shameless ads below. All below was ongoing in 2021.</p>
<ul>
<li><strong><a href="https://www.map5.nl/" target="_blank" rel="noreferrer noopener">map5.nl</a>
</strong> is a subscription service for Dutch topographic, historical- and embellished hill-shade and arial maps I started to host in 2015. In 2021 work started for a map5.nl user portal.</li>
</ul>
<div class="">
<a href="https://www.map5.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/map5.nl_.png"
alt="" />
</a>
</div>
<ul>
<li><strong><a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
</strong> is a hosted <strong><a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
</strong> (GHC) service on a subscription basis. GHC is an uptime and QoS monitor for (OGC) web services. Customers get their own GHC instance. <strong><a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
</strong> saves the burden of self-hosting GHC. In 2021 started work on a GHC API, using <a href="https://fastapi.tiangolo.com/" target="_blank" rel="noreferrer noopener">FastAPI</a>
, and a new GHC-UI.</li>
</ul>
<div class="">
<a href="https://geoqos.com/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/geoqos.com_.png"
alt="" />
</a>
</div>
<ul>
<li><strong><a href="https://geotoko.nl" target="_blank" rel="noreferrer noopener">geotoko.nl</a>
</strong>, in short GeoToko, is basically a webshop to download Dutch open geo-datasets. One may ask: Dutch geospatial data like Topography, Addresses and Buildings, is already open and publicly available, mainly via <strong><a href="https://pdok.nl" target="_blank" rel="noreferrer noopener">Kadaster-PDOK</a>
</strong>. So why bother reselling? Explanation follows.</li>
</ul>
<div class="">
<a href="https://geotoko.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/opentopo-keyreg.png"
alt="" />
</a>
</div>
<p>Most Dutch Open geospatial datasets, available from governmental institutions like Kadaster, are provided in a neutral exchange-format. In practice: GML (Application Schema). Most users are not able to directly digest complex GML in their applications (or don't want to spend time on that). For over 10 years we run the Open Source project <strong><a href="https://github.com/nlextract/NLExtract" target="_blank" rel="noreferrer noopener">NLExtract</a>
</strong> to convert these datasets into formats like PostGIS and CSV that can be directly used in applications. NLExtract itself builds on <strong><a href="https://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl</a>
</strong>, an Open Source geospatial ETL-library in Python. NLExtract/Stetl is used a lot, but still will require users to install/maintain an NLExtract/Stetl installation, download datasets, run the ETL, check results etc. That may also be a bridge too far, if e.g. all a customer needs is a one-time CSV of say, all 10 million addresses in The Netherlands. So GeoToko taps into this niche, providing ready-to-use, often enriched Dutch datasets. Pricing is reasonable, compared to other providers, plus organizations and individuals may get highly reduced pricing when they work on Open data applications like OpenStreetMap.</p>
<div class="">
<a href="https://geotoko.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/bigpicture.png"
alt="" />
</a>
</div>
<p>As for development: the <strong><a href="https://geotoko.nl" target="_blank" rel="noreferrer noopener">GeoToko webshop</a>
</strong> was developed with Django, Flask/nginx (Download management), Stripe (backend payments) and <strong><a href="https://ckan.org" target="_blank" rel="noreferrer noopener">CKAN</a>
</strong>. The latter to provide a product-catalogue, sample data and metadata in general, all via <strong><a href="https://geocatalogus.nl" target="_blank" rel="noreferrer noopener">GeoCatalogus.nl</a>
</strong>.</p>
<div class="">
<a href="https://geotoko.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/geofabriek-keten.png"
alt="GeoFabriek - Production Chain" />
</a>
</div>
<p>Further development concerned automating the NLExtract/Stetl ETL-processes.
For this a framework called GeoFabriek, "GeoFactory", was developed. This automates the entire chain from checking new dataset versions at Kadaster, through downloading, conversions, packing for download and updating the metadata (in GeoCatalogus.nl).</p>
<h2 id="6-open-source-contributions">6. Open Source Contributions</h2>
<p>Continuous work as a contributor. Apart from some GitLab Projects, you can find/follow me best on <a href="https://github.com/justb4/" target="_blank" rel="noreferrer noopener">GitHub</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2022/github-contribs-2021.png" alt="GitHub contributions"></p>
<p>New in 2021:</p>
<ul>
<li><a href="https://github.com/justb4/fastapi-users-starter" target="_blank" rel="noreferrer noopener">FastAPI-Users starter Template repo</a>
- adding full user management to FastAPI.</li>
<li><a href="https://nlextract.nl/" target="_blank" rel="noreferrer noopener">NLExtract</a>
- added ETL for BAG version 2, my first PRs for GDAL!</li>
</ul>
<p>Ongoing in 2021:</p>
<ul>
<li><a href="https://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl</a>
- Geospatial ETL in Python, maintaining since 2011.</li>
<li><a href="https://geohealthcheck.org" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
- Service Status and QoS Checker for OGC Web Services.</li>
<li><a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
- a Python server implementation of the <a href="https://www.opengeospatial.org/blog/2996" target="_blank" rel="noreferrer noopener">OGC API suite of standards</a>
- joined this great project in 2019. Also in PSC.</li>
<li><a href="https://demo.pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi demo server</a>
- provided the (auto-)deployment stack for the pygeoapi demo server (<a href="https://github.com/geopython/demo.pygeoapi.io" target="_blank" rel="noreferrer noopener">code at GitHub</a>
). Added <a href="https://demo.pygeoapi.io/covid-19" target="_blank" rel="noreferrer noopener">COVID-19 NL data provider</a>
.</li>
<li><a href="https://nlextract.nl/" target="_blank" rel="noreferrer noopener">NLExtract</a>
- ETL for Dutch geospatial datasets.</li>
<li><a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
- geo-webclient framework based on <a href="https://vuejs.org/" target="_blank" rel="noreferrer noopener">Vue.js</a>
with OpenLayers started by <a href="https://github.com/chrismayer" target="_blank" rel="noreferrer noopener">Christian Mayer</a>
. Joined this great project in 2019. Unfortunately hardly contribs in 2021.</li>
<li><a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron MC</a>
- Web Mapping Client based on <a href="https://www.geoext.org/" target="_blank" rel="noreferrer noopener">GeoExt</a>
and OpenLayers. Yes, old tech but still in wide use. In time hope to migrate to <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
(see above). Migrating Heron to Wegue, see first example below.</li>
</ul>
<h3 id="more-contributions---handy-docker-images">More Contributions - Handy Docker Images</h3>
<p>To support many of the Cloud services and Open Source projects, I develop and maintain <a href="https://github.com/search?q=user%3Ajustb4+docker-" target="_blank" rel="noreferrer noopener">handy Docker Images</a>
, also available from <a href="https://hub.docker.com/u/justb4" target="_blank" rel="noreferrer noopener">my DockerHub</a>
.
Ongoing in 2021. In progress of moving all to Github Workflows build/push, replacing DockerHub Autobuild (still awaiting DockerHub Open Source waiver...):</p>
<ul>
<li><a href="https://github.com/justb4/docker-awstats" target="_blank" rel="noreferrer noopener">docker-awstats</a>
- AWStats in Docker, oldie, but very effective webstats. Deploy multiple instances in single Docker container. Highly configurable, e.g. also for <a href="https://docs.traefik.io/" target="_blank" rel="noreferrer noopener">Traefik</a>
access logs.</li>
<li><a href="https://github.com/justb4/docker-jmeter" target="_blank" rel="noreferrer noopener">docker-jmeter</a>
- <a href="https://jmeter.apache.org/" target="_blank" rel="noreferrer noopener">Apache JMeter</a>
wrapped in Docker. Over 10 million pulls at end of 2021. And 180 stars, but still <a href="https://www.paypal.com/donate/?cmd=_s-xclick&hosted_button_id=3QZW9SNGCWBM4" target="_blank" rel="noreferrer noopener">zero donations</a>
! C'mon people, not even using JMeter myself...</li>
<li><a href="https://github.com/justb4/docker-mapfish-print" target="_blank" rel="noreferrer noopener">docker-mapfish-print</a>
- for MapFish Print <strong>version 3</strong>. MapFish Print allows printing maps as PDFs.</li>
<li><a href="https://github.com/justb4/docker-mapfish-print2" target="_blank" rel="noreferrer noopener">docker-mapfish-print2</a>
- for MapFish Print <strong>version 2</strong> - This version is still used in quite some contexts, at least for <a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron</a>
and <a href="https://kadviewer.map5.nl/" target="_blank" rel="noreferrer noopener">KadViewer</a>
.</li>
<li><a href="https://github.com/justb4/docker-rclone" target="_blank" rel="noreferrer noopener">docker-rclone</a>
- Docker image to perform a <a href="http://rclone.org/" target="_blank" rel="noreferrer noopener">rclone</a>
sync based on a cron schedule, with <a href="https://healthchecks.io/" target="_blank" rel="noreferrer noopener">healthchecks.io</a>
monitoring.</li>
<li><a href="https://github.com/justb4/docker-pgbackup" target="_blank" rel="noreferrer noopener">docker-pgbackup</a>
- automated/scheduled PostgreSQL/PostGIS backups for all PostgreSQL-based Docker Containers in its Docker-network. I think it is nifty: just run and forget: your Postgres backups are taken care of. Inspired by: <a href="https://github.com/kartoza/docker-pg-backup">https://github.com/kartoza/docker-pg-backup</a>.</li>
<li><a href="https://github.com/justb4/docker-mapserver" target="_blank" rel="noreferrer noopener">docker-mapserver</a>
- Slim Docker Image for MapServer with Lighttpd FastCGI.</li>
<li><a href="https://github.com/justb4/docker-mapproxy" target="_blank" rel="noreferrer noopener">docker-mapproxy</a>
- MapProxy Docker Image adapted, slimmed, from the <a href="https://yagajs.org/" target="_blank" rel="noreferrer noopener">YAGA Development-Team</a>
. <a href="https://github.com/yagajs/docker-mapproxy/pull/18" target="_blank" rel="noreferrer noopener">Awaiting PR merge</a>
.</li>
<li><a href="https://github.com/justb4/docker-mapproxy-mapserver" target="_blank" rel="noreferrer noopener">docker-mapproxy-mapserver</a>
- Docker Image for MapProxy service with built-in MapServer binaries accessed directly (no MapServer service).</li>
<li><a href="https://github.com/justb4/docker-cron" target="_blank" rel="noreferrer noopener">docker-cron</a>
- Runs Unix cron, includes docker (compose) client for running remote Docker Images.</li>
</ul>
<h2 id="7-conferences---attended">7. Conferences - Attended</h2>
<ul>
<li>Feb 6-7 <a href="https://archive.fosdem.org/2021/" target="_blank" rel="noreferrer noopener">FOSSDEM 2021</a>
- Virtual</li>
<li>Sept 27- Oct 22 - <a href="http://2020.geopython.net/" target="_blank" rel="noreferrer noopener">FOSS4G Buenos Aires</a>
- Virtual</li>
<li>Oct 19-20 - <a href="https://info.crunchydata.com/en/postgis-day-2020-crunchy-data" target="_blank" rel="noreferrer noopener">FOSS4G-NL at ITC Enschede</a>
- In Real Life</li>
</ul>
<p><a href="http://2020.geopython.net/" target="_blank" rel="noreferrer noopener">FOSS4G Buenos Aires</a>
needs special mention. As an online conference, apart from amazing content, I was very impressed with the
technical and organizational setup. Even the social events were fun to join. Folks are tired of Zoom, Teams, etc but
the FOSS4G LOC had introduced tools like <a href="https://workadventu.re/" target="_blank" rel="noreferrer noopener">Work Adventure</a>
and <a href="https://streamyard.com/" target="_blank" rel="noreferrer noopener">StreamYard</a>
(also used by OSgeo.nl in Big Geo Show) to enable more "cozyness".
Offcourse nothing can replace a real event, but this came close. The Gala Dinner was really fun!</p>
<p>Jody Garnett words it very well in his Tweet:</p>
<div class="">
<a href="https://mobile.twitter.com/jodygarnett/status/1444022337968295938">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/foss4g/thankyou.png"
alt="Thank You!" />
</a>
</div>
<p>Never forget, always in our hearts: Malena Libman (1983-2021).
FOSS4G 2021 was also a conference dedicated to, and in memory of, Malena Libman.
She was a pillar of the FOSS4G community and a leading organizer of this conference.</p>
<div class="">
<a href="https://mappery.org/malena-libman/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/foss4g/malena.png"
alt="Malena - Photo from mappery.org" />
</a>
</div>
<p>Malena was the right and truthful winner of the Sol Katz Award 2021.
As the <a href="https://www.osgeo.org/foundation-news/malena-libman-receives-the-2021-sol-katz-award/" target="_blank" rel="noreferrer noopener">OSGeo.org website states</a>
:
<em>"Malena Libman is the Sol Katz Award winner 2021. The prize was presented at the FOSS4G closing session and handed over to her friends & family who accepted the award on her behalf since we sadly lost Malena in April 2021 to COVID-19."</em></p>
<h2 id="8-talks--workshops---provided">8. Talks & Workshops - Provided</h2>
<p>Below talks and workshops I provided in 2021. A complete list of <a href="https://justobjects.nl/presentations/">presentations</a>
is available.</p>
<ul>
<li>
<p><strong><em>"Ten Years of OSGeo.nl - A Story in Pictures" - De Grote Geo Show - Episode 19 (video) - December 2, 2021 - Online</em></strong> - <a href="https://www.youtube.com/watch?v=yMAxQEg0fSw" target="_blank" rel="noreferrer noopener">[Video recording on YouTube]</a>
- <a href="https://tv.osgeo.nl/episode/episode-0019/" target="_blank" rel="noreferrer noopener">[Episode Info]</a>
.</p>
</li>
<li>
<p><strong><em>"OGC API Features" - Geonovum - Kennisplatform APIs - <a href="https://www.geonovum.nl/over-geonovum/agenda/werkgroep-nederlandse-api-strategie-geo-extensie" target="_blank" rel="noreferrer noopener">Dutch API Strategy - Geo-extension WG</a>
- November 24, 2021 - Online</em></strong> - <a href="https://files.justobjects.nl/presentation/geonovum-2021-api-strategie/OGC-API-Features.pdf" target="_blank" rel="noreferrer noopener">[PDF Slides by OGC]</a>
.</p>
</li>
<li>
<p><strong><em>"FOSS4G-NL 2021 - Closing Words" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede</em></strong> - <a href="https://files.justobjects.nl/presentation/foss4gnl-2021/foss4gnl-2021-slot-just.pdf" target="_blank" rel="noreferrer noopener">[PDF Slides]</a>
.</p>
</li>
<li>
<p><strong><em>"Zo gemakkelijk kun je geo-webdiensten uitrollen!" - FOSS4G-NL 2021 - Oktober 20, 2021 - Enschede</em></strong> - On GitOps for OSGeo Software - <a href="https://files.justobjects.nl/presentation/foss4gnl-2021/foss4gnl-deploy-2021-just.pdf" target="_blank" rel="noreferrer noopener">[PDF Slides]</a>
.</p>
</li>
<li>
<p><strong><em>"GeoHealthCheck - A Quality of Service Monitor for Geospatial Web Services" - <a href="https://2021.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G 2021</a>
- September 30, 2021</em></strong> - <a href="https://geohealthcheck.org/presentation/" target="_blank" rel="noreferrer noopener">[HTML Slides]</a>
- <a href="https://callforpapers.2021.foss4g.org/foss4g2021/talk/9NBETK/" target="_blank" rel="noreferrer noopener">[Abstract]</a>
- <a href="https://www.youtube.com/watch?v=t7Y8FuaE-_M" target="_blank" rel="noreferrer noopener">[Video recording on YouTube]</a>
.</p>
</li>
<li>
<p><strong><em>"Doing Geospatial in Python" - <a href="https://2021.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G 2021</a>
- September 28, 2021</em></strong> - Workshop (4h): introduction to performing common GIS/geospatial tasks using Python geospatial tools such as OWSLib, Shapely, Fiona/Rasterio, and common geospatial libraries like GDAL, PROJ, pycsw, as well as other tools from the geopython toolchain. - <a href="https://geopython.github.io/geopython-workshop/" target="_blank" rel="noreferrer noopener">[HTML Startpage]</a>
- <a href="https://callforpapers.2021.foss4g.org/foss4g-2021-workshop/talk/7N3G3N/" target="_blank" rel="noreferrer noopener">[Abstract]</a>
.</p>
</li>
<li>
<p><strong><em>"Geonovum OGC API Testbed" - Open Geodag 2021 - September 14, 2021</em></strong> - <a href="https://www.youtube.com/watch?v=rl4-tPbfxOE" target="_blank" rel="noreferrer noopener">[PDF Slides]</a>
- <a href="https://youtu.be/apVe6dA5rqs?t=386" target="_blank" rel="noreferrer noopener">[Video Recording]</a>
.</p>
</li>
<li>
<p><strong><em>"Geonovum OGC API Testbed" - DiS Online: OGC en toegankelijke APIs - September 7, 2021</em></strong> - <a href="https://www.geobasisregistraties.nl/basisregistraties/documenten/publicatie/2021/09/07/dis-online-ogc-en-toegankelijke-apis" target="_blank" rel="noreferrer noopener">[Event en PDF Slides]</a>
.</p>
</li>
<li>
<p><strong><em>"Read BAG (Dutch Buildings & Addresses dataset) with Docker in 5 minutes" - 2021-04-08 - De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Ep. 16 "GIS met je Toetsenbord". Aired on april 8, 2021. <a href="https://youtu.be/iOiliIAorD4?t=3542" target="_blank" rel="noreferrer noopener">Video recording on YouTube</a>
.</em></strong></p>
</li>
<li>
<p><strong><em>"OSGeo.nl New Year's Party - Slides OSGeo.nl" - 2021-01-24 - Presentation I gave on behalf of OSGeo.nl on the online joint QGIS-NL, OpenStreetMap-NL and OSGeo.nl New Year's party on January 24, 2021.</em></strong>- <a href="https://www.slideshare.net/justb4/osgeonl-new-years-party-slides-osgeonl" target="_blank" rel="noreferrer noopener">[Slideshare]</a>
</p>
</li>
</ul>
<div class="">
<a href="https://callforpapers.2021.foss4g.org/foss4g-2021-workshop/talk/7N3G3N/">
<img class="lazy"
src="https://justobjects.nl/uploads/2022/foss4g/geopython-workshop-group.jpg"
alt="Geopython Workshop at FOSS4G Buenos Aires" />
</a>
</div>
<p>Not as much fun as in 2019 Bukarest, but providing the Geopython Workshop at FOSS4G Buenos Aires with these great minds
is still a 2021-highlight.</p>
<h2 id="9-resolutions-for-2022">9. Resolutions for 2022</h2>
<ul>
<li>Customer portal for <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
</li>
<li>Get back into Kubernetes</li>
<li>Advance further into full-lifecycle GitOps</li>
<li>Advance further into Vector Tiling</li>
<li>More effort into <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
project</li>
<li>Revive old GeoTracing projects like <a href="http://georambling.com" target="_blank" rel="noreferrer noopener">georambling.com</a>
</li>
<li>More favourite things</li>
<li>Whatever comes around.</li>
</ul>
-
Terraform - First Steps on Hetzner Cloud
https://justobjects.nl/terraform-first-steps/
Mon, 13 Sep 2021 13:24:25 +0000
https://justobjects.nl/terraform-first-steps/
<p>I use <a href="https://www.ansible.com/" target="_blank" rel="noreferrer noopener">Ansible</a>
already for years to provision server
instances and for subsequent CI/CD.
A recent example is the <a href="https://apitestdocs.geonovum.nl/" target="_blank" rel="noreferrer noopener">Geonovum OGC API Testbed</a>
.
Here (selective) Docker Containers are automatically deployed on GitHub pushes using Ansible called
from within a GitHub Workflows.</p>
<div class="">
<a href="https://apitestdocs.geonovum.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/09/ogcapi-testbed.png"
alt="Geonovum OGC API Testbed CI/CD Architecture" />
</a>
</div>
<p>Now investigating how <a href="https://www.terraform.io/" target="_blank" rel="noreferrer noopener">Terraform</a>
could play a key role in (Cloud) infrastructure management.
There is a small overlap between Ansible and Terraform but that is a matter of how they are applied in concert.</p>
<p>Ansible is more geared towards maintaining the OS and its running components e.g. Docker Containers on VM-instances.
Terraform is more geared to maintaining a Cloud infrastructure,
"in the large": acquiring VM-instances, networks, DNS.
If you are familiar with AWS, Google Cloud Platform, or in our case <a href="https://www.hetzner.com/cloud" target="_blank" rel="noreferrer noopener">Hetzner Cloud</a>
,
it is what you can do by clicking in their respective UIs or
via their APIs like Hetzner's <a href="https://github.com/hetznercloud/cli" target="_blank" rel="noreferrer noopener">hcloud</a>
.
And btw: both Ansible and Terraform are Open Source.</p>
<p>Quote from
<a href="https://www.cprime.com/resources/blog/terraform-and-ansible-tutorial-integrating-terraform-managed-instances-with-ansible-control-nodes/#:%5c~:text=Terraform%20and%20Ansible,-So%2C%20why%20do&text=Terraform%20is%20designed%20to%20provision,necessary%20applications%20on%20that%20machine." target="_blank" rel="noreferrer noopener">random web-search</a>
:
<em>"<strong>Terraform</strong> is designed to provision different infrastructure components. <strong>Ansible</strong> is a
configuration-management and application-deployment tool. It means that you'll use <strong>Terraform</strong> first
to create, for example, a virtual machine and then use <strong>Ansible</strong> to install necessary applications on that machine."</em></p>
<p>Both Ansible and Terraform are "declarative", i.e. configuration-based, where the configuration describes a desired state.
ctions are "idempotent", i.e. the same action can be applied multiple times, but when the desired state is reached it won't have effect.</p>
<p>As I plan to apply Terraform in other projects as well, I took a deep dive, following hands-on tutorials from the Terraform website.
<em><strong>In a very short time I was amazed by Terraform's power and elegance!</strong></em>
My ultimate goal was to manage (acquire, configure, access, destroy) the lifecycle of <a href="https://www.hetzner.com/cloud" target="_blank" rel="noreferrer noopener">Hetzner Cloud</a>
Virtual Machines (VMs, VPS's).
This all took about less than two hours, documenting my steps along the way.</p>
<p>I started at the <a href="https://learn.hashicorp.com/collections/terraform/aws-get-started" target="_blank" rel="noreferrer noopener">Getting Started</a>
page.
While there is a lot of stuff on AWS, Terraform Cloud, I basically stuck to these four steps.
You may even skip the third (GCP) step. Important is to learn the terminology and config conventions.</p>
<ol>
<li>Installation</li>
<li>Using Docker Provider</li>
<li>Using GCP Provider</li>
<li>Using Hetzner Cloud Provider
<br>
<em>Tip: in IntelliJ IDEA install the Terraform plugin. It will recognise/help with Terraform files!</em></li>
</ol>
<h3 id="step-1---installation">Step 1 - Installation</h3>
<p><a href="https://learn.hashicorp.com/tutorials/terraform/install-cli" target="_blank" rel="noreferrer noopener">learn.hashicorp.com/tutorials/terraform/install-cli </a>
</p>
<p>On a Mac with Homebrew install the Terraform CLI:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ brew tap hashicorp/tap
<span class="ln">2</span>
<span class="ln">3</span>$ brew install hashicorp/tap/terraform
<span class="ln">4</span>
<span class="ln">5</span>$ terraform -version
<span class="ln">6</span>
<span class="ln">7</span>Terraform v1.0.2
<span class="ln">8</span>on darwin_amd64
</code></pre></div><p>That's it!</p>
<h3 id="step-2---using-docker-provider">Step 2 - Using Docker Provider</h3>
<p><a href="https://learn.hashicorp.com/collections/terraform/docker-get-started" target="_blank" rel="noreferrer noopener">learn.hashicorp.com/collections/terraform/docker-get-started</a>
</p>
<p>I started project dirs under <code>~/project/terraform/learn/.</code></p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ mkdir -p ~/project/terraform/learn/terraform-docker-container
</code></pre></div><p>Create a file called main.tf:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>terraform {
<span class="ln"> 2</span> required_providers {
<span class="ln"> 3</span> docker = {
<span class="ln"> 4</span> source = "kreuzwerker/docker"
<span class="ln"> 5</span> version = "~ 2.13.0"
<span class="ln"> 6</span> }
<span class="ln"> 7</span> }
<span class="ln"> 8</span>}
<span class="ln"> 9</span>
<span class="ln">10</span>provider "docker" {}
<span class="ln">11</span>
<span class="ln">12</span>resource "docker_image" "nginx" {
<span class="ln">13</span> name = "nginx:latest"
<span class="ln">14</span> keep_locally = false
<span class="ln">15</span>}
<span class="ln">16</span>
<span class="ln">17</span>resource "docker_container" "nginx" {
<span class="ln">18</span> image = docker_image.nginx.latest
<span class="ln">19</span> name = "tutorial"
<span class="ln">20</span> ports {
<span class="ln">21</span> internal = 80
<span class="ln">22</span> external = 8000
<span class="ln">23</span> }
<span class="ln">24</span>}
<span class="ln">25</span>
</code></pre></div><p>This defines that we will use the Terraform Provider plugin named
"docker" with source <a href="https://registry.terraform.io/providers/kreuzwerker/docker/latest/docs" target="_blank" rel="noreferrer noopener">kreuzwerker/docker</a>
. `Terraform has a registry of official (Provider) plugins.</p>
<p>Now initialize and install the plugin:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>$ terraform init
<span class="ln"> 2</span>
<span class="ln"> 3</span> **Initializing the backend...**
<span class="ln"> 4</span>
<span class="ln"> 5</span> **Initializing provider plugins...**
<span class="ln"> 6</span>
<span class="ln"> 7</span>- Finding kreuzwerker/docker versions matching "~ 2.13.0"...
<span class="ln"> 8</span>
<span class="ln"> 9</span>- Installing kreuzwerker/docker v2.13.0...
<span class="ln">10</span>
<span class="ln">11</span>- Installed kreuzwerker/docker v2.13.0 (self-signed, key ID **24E54F214569A8A5**)
<span class="ln">12</span>
<span class="ln">13</span>. etc
<span class="ln">14</span>
</code></pre></div><p>You may validate your config:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ terraform validate
</code></pre></div><p>Moment of truth: create the resources:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ terraform apply
</code></pre></div><p>Verify the existence of the NGINX container
by visiting <a href="http://localhost:8000/" target="_blank" rel="noreferrer noopener">localhost:8000</a>
in your web browser or running docker ps to see the container.</p>
<p>That's it for Docker. Next is to use a real Cloud Provider.</p>
<h3 id="step-3---using-gcp-provider">Step 3 - Using GCP Provider</h3>
<p><a href="https://learn.hashicorp.com/collections/terraform/gcp-get-started" target="_blank" rel="noreferrer noopener">learn.hashicorp.com/collections/terraform/gcp-get-started</a>
</p>
<p>This was actually more elaborate than the Hetzner Cloud exercise.
I had a GCP account, so this went smooth: first creating a Network and later a VM Instance.
In this step also learned about using Terraform Providers, Resources, Variables(-files.)
I leave this as an option and skip to Hetzner Cloud which is the goal of this post.</p>
<h3 id="step-4---using-hetzner-cloud-provider">Step 4 - Using Hetzner Cloud Provider</h3>
<p>The Hetzner Cloud plugin provider was not in the tutorials,
but it was still not too hard to extrapolate from Step 3,
from various Hetzner tutorials and from the <code>hetznercloud/hcloud</code> Terraform Provider.</p>
<p>Links:</p>
<ul>
<li>hcloud Provider: <a href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest">https://registry.terraform.io/providers/hetznercloud/hcloud/latest</a></li>
<li><a href="https://community.hetzner.com/tutorials/howto-hcloud-terraform">https://community.hetzner.com/tutorials/howto-hcloud-terraform</a></li>
<li><a href="https://maddevs.io/blog/terraform-hetzner/">https://maddevs.io/blog/terraform-hetzner/</a></li>
<li>using TF-hcloud with Docker: <a href="https://github.com/insekticid/docker-terraform-hcloud-example/" target="_blank" rel="noreferrer noopener">github.com/insekticid/docker-terraform-hcloud-example</a>
</li>
</ul>
<p>Prerequisite is to have a Hetzner Cloud account and
thus login access to <a href="https://console.hetzner.cloud" target="_blank" rel="noreferrer noopener">console.hetzner.cloud</a>
.</p>
<p>Steps:</p>
<ul>
<li>create a new Project in <a href="https://console.hetzner.cloud/projects,">https://console.hetzner.cloud/projects,</a> e.g. <code>TerraformLearn</code></li>
<li>add your SSH public key to this project via "Security" menu link left</li>
<li>generate and copy an API Token for the project</li>
</ul>
<p>My goal was to create a Debian VM, login there with root and SSH key and destroy it afterwards.</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ mkdir -p ~/project/terraform/learn/terraform-hetzner
</code></pre></div><p>Create <code>main.tf</code> as follows:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>terraform {
<span class="ln"> 2</span> required_providers {
<span class="ln"> 3</span> hcloud = {
<span class="ln"> 4</span> source = "hetznercloud/hcloud"
<span class="ln"> 5</span> version = "1.27.2"
<span class="ln"> 6</span> }
<span class="ln"> 7</span> }
<span class="ln"> 8</span> }
<span class="ln"> 9</span>
<span class="ln">10</span> provider "hcloud" {
<span class="ln">11</span> token = var.hcloud_token
<span class="ln">12</span> }
<span class="ln">13</span>
<span class="ln">14</span> resource "hcloud_server" "node1" {
<span class="ln">15</span> name = "node1"
<span class="ln">16</span> image = "debian-9"
<span class="ln">17</span> server_type = "cx11"
<span class="ln">18</span> ssh_keys = ["[email protected]"]
<span class="ln">19</span> }
<span class="ln">20</span>
</code></pre></div><p>Create a file <code>variables.tf</code>:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span> # Set the variable value in *.tfvars file
<span class="ln">2</span> # or using the -var="hcloud_token=..." CLI option
<span class="ln">3</span> variable "hcloud_token" {
<span class="ln">4</span> sensitive = true # Requires terraform = 0.14
<span class="ln">5</span> }
<span class="ln">6</span>
</code></pre></div><p>Then a file called <code>terraform.tfvars</code>.
This is a file with "secrets" normally not checked-in a repo but there are
many other possiblities to deal with secrets/credentials:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span> hcloud_token = "the token string from Hetzner Cloud API Token"
</code></pre></div><p>Moment of truth: apply!</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ terraform init
<span class="ln">2</span>
<span class="ln">3</span>$ terraform apply -auto-approve
</code></pre></div><p>Using <code>-auto-approve</code> you skip the interactive approval-step.</p>
<p>Next check the Hetzner Cloud Console project page and see the new VM running!</p>
<p>Try to login on your new VM (IP may also be gotten from <code>output.tf</code>, another exercise):</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ ssh root@<Your VM IP>
</code></pre></div><p>Then destroy your VM:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>$ terraform destroy -auto-approve`
</code></pre></div><p>There is much more one can do with the Hetzner Provider:
basically everything that is available in the console
UI and <code>hcloud</code> API: creating Volumes, managing networks,
adding SSH-keys, snapshots, using <code>cloud-init</code> etc.
See the manpage in particular the Resources drop-down menu:
<a href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs" target="_blank" rel="noreferrer noopener">registry.terraform.io/providers/hetznercloud/hcloud/latest/docs</a>
</p>
<p>Beware that some Terraform actions are destructive: e.g. upgrading the OS
will destroy the existing VM and create a new.
For those cases Floating IPs and auto-provisioning with Ansible will help.
But in that case Ansible would be more suited to upgrade the OS.
One can always execute <code>terraform plan</code> first to see the execution plan.
My recommendation is to let Terraform handle the basics, and have Ansible manage the details on VMs.</p>
<p>Alternatives: TF with DigitalOcean using the DO Provider:
<a href="https://registry.terraform.io/providers/digitalocean/digitalocean/latest/docs" target="_blank" rel="noreferrer noopener">registry.terraform.io/providers/digitalocean/digitalocean/latest/docs</a>
.</p>
<p>All in all: Terraform can form a nice partnership with Ansible.</p>
-
That Was 2020
https://justobjects.nl/that-was-2020/
Wed, 20 Jan 2021 13:24:25 +0000
https://justobjects.nl/that-was-2020/
<p>Overview of my professional life in 2020. Highlights of living and working in the Open Source Geospatial and OSGeo(<a href="https://osgeo.nl/" target="_blank" rel="noreferrer noopener">.nl</a>
|.<a href="https://www.osgeo.org/" target="_blank" rel="noreferrer noopener">org</a>
)-world in 2020. Organized by “Theme”.</p>
<p>Yes, 2020 was a “special year” in pandemic and political terms. So much has been written and opinions expressed, that I don’t see a need to add to more here. Some of my activities below may make clear how I and we (OSGeo.nl community) dealt.</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/various-4eps.jpg"
alt="Screenshots of De Grote Geo Show" />
</a>
</div>
<!--
<div id="attachment_947" style="width: 796px" class="wp-caption aligncenter">
<a href="https://tv.osgeo.nl"><img aria-describedby="caption-attachment-947" loading="lazy" class="size-full wp-image-947" src="../uploads/2021/01/various-4eps.jpg" alt="Screenshots of De Grote Geo Show" width="786" height="432" srcset="https://justobjects.nl/wp-content/uploads/2021/01/various-4eps.jpg 786w, https://justobjects.nl/wp-content/uploads/2021/01/various-4eps-300x165.jpg 300w, https://justobjects.nl/wp-content/uploads/2021/01/various-4eps-768x422.jpg 768w, https://justobjects.nl/wp-content/uploads/2021/01/various-4eps-150x82.jpg 150w" sizes="(max-width: 786px) 100vw, 786px" /></a>
<p id="caption-attachment-947" class="wp-caption-text">
Screenshots of De Grote Geo Show
</p>
</div>
-->
<p>TL;DR. My absolute 2020-highlight was initiating and working on <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">De Grote Geo Show</a>
</strong> (“DGGS, The Big Geo Show”), a live-streamed webshow in “De Wereld Draait Door”-style. This was/is a great team-effort by Dutch Open Source Geo/OSGeo.nl community members. Kudos to Erik, Niene, Mariëlle, Jonna, Willem, Hans and many many more (about 30+ folks)! We streamed 13 shows in 2020 with a huge range of subjects and (international) guests. And we’re still going strong with a <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">DGGS New Year’s Show/Socializer</a>
</strong> on Sunday, Jan. 24, 2021.</p>
<p>Below follow 2020-highlights by theme.</p>
<h3 id="the-big-geo-show">The Big Geo Show</h3>
<p>Let’s start with the absolute 2020 highlight. It became clear in March 2020 that we had to cancel or virtualize our planned (June 2020, ITC Enschede) FOSS4G-NL conference and other OSGeo.nl events. I invested some time into “webinar streaming software”: Zoom, Jitsi, Teams WebEx etc. Those did not really appeal to me for what I had in mind.</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/backstage-view-by-niene.jpg"
alt="StreamYard - Backstage View" />
</a>
</div>
<!--
<div id="attachment_999" style="width: 830px" class="wp-caption alignnone">
<a href="https://tv.osgeo.nl"><img aria-describedby="caption-attachment-999" loading="lazy" class="size-large wp-image-999" src="../uploads/2021/01/backstage-view-by-niene-1024x522.jpg" alt="StreamYard - Backstage View" width="820" height="418" srcset="https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene-1024x522.jpg 1024w, https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene-300x153.jpg 300w, https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene-768x391.jpg 768w, https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene-1536x783.jpg 1536w, https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene-150x76.jpg 150w, https://justobjects.nl/wp-content/uploads/2021/01/backstage-view-by-niene.jpg 1902w" sizes="(max-width: 820px) 100vw, 820px" /></a>
<p id="caption-attachment-999" class="wp-caption-text">
<strong><em>StreamYard – Backstage View</em></strong>
</p>
</div>
-->
<p>For personal streaming, I was testing <strong><a href="https://obsproject.com/" target="_blank" rel="noreferrer noopener">OBS Studio</a>
</strong>, streaming to Twitch.tv and YouTube. I then stumbled on <strong><a href="https://www.learnwithjason.dev/" target="_blank" rel="noreferrer noopener">“Learn with Jason”</a>
</strong>. Very entertaining and deeply technical at the same time. I liked the format of having guests in the stream. That triggered me to do something similar with the Open Source Geo community in the Netherlands. Long story short: I had the idea to do a short, 1 hour webshow with various subjects and guests. Before I knew we had a Telegram group of now around 30 folks with a kernel of about 6-8 and started brainstorming. The other element which added to the success was my discovery of <strong><a href="https://streamyard.com" target="_blank" rel="noreferrer noopener">StreamYard.com</a>
</strong>.</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/combined2.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-997" src="../uploads/2021/01/combined2.png" alt="" width="800" height="214" srcset="https://justobjects.nl/wp-content/uploads/2021/01/combined2.png 800w, https://justobjects.nl/wp-content/uploads/2021/01/combined2-300x80.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/combined2-768x205.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/combined2-150x40.png 150w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<p>Now <strong><a href="https://streamyard.com" target="_blank" rel="noreferrer noopener">StreamYard</a>
</strong> is a whole story by itself. In short it differs from all the well-known video-conf programs that it is a Cloud-based Broadcast Studio. Conceptually it is like making a real TV-show. There is a Stage (main screen) which can be “branded” in all sorts of creative, interactive ways: backgrounds, overlays, banners/tickers, video clips (e.g. for intro/outro or soapboxes), screen layouts, chat messages from viewers. All is controlled by one or more “hosts” (Dutch: “regisseurs”). Guests join “backstage” first and can be added to the “stage” when their item starts. And last but not least: the entire Stage can be streamed to multiple destinations at the same time like YouTube, Twitch.tv, LinkedIn etc. but also to custom RTMP servers. Viewers who missed the show could watch back later. StreamYard works completely in the browser, no installs needed. Ok, StreamYard is not Open Source, but their registration is very minimal: one only needs to provide an email adres to which a temporary login code is sent, no profile-filling etc.</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/combined.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter wp-image-993" src="../uploads/2021/01/combined.png" alt="" width="800" height="232" srcset="https://justobjects.nl/wp-content/uploads/2021/01/combined.png 1024w, https://justobjects.nl/wp-content/uploads/2021/01/combined-300x87.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/combined-768x223.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/combined-150x44.png 150w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<p>So we just started, all of us not really familiar with providing online events and being first-timers in StreamYard. One learns quickest by just jumping into the deep, learning by doing, not bother about mistakes. We quickly adopted a format for a weekly one-hour show, always at the same time, Thursdays 4-5 PM. A central website <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">tv.osgeo.nl</a>
</strong> was quickly online (with GitHub and <strong><a href="https://gohugo.io/" target="_blank" rel="noreferrer noopener">Hugo</a>
</strong>!), so we can refer to a single, short URL.</p>
<p>The format of each show was basically to have 5-6 items in an hour. An “item” could be an interview with a guest, a technical presentation (or mixed interview/presentation), a column, community updates, a (hands-on) tutorial, a poem, or last but least a live-quiz (via Kahoot) where viewers can participate! After the first show (where we looked a bit like rabbits in headlights!), we quickly realized we need a presenter-host for the entire show. So all subsequent shows had one of us, well mainly Niene, being the talkshow-host. We also decided to do the show in Dutch, except with international guests, as to engage our local communities (OSGeo-NL, QGIS-NL, OpenStreetMap-NL).</p>
<div class="">
<a href="https://www.youtube.com/playlist?list=PLJMEnRQpAfZqCkhGh3lb3KUnXssK7Sk6C">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/episode-1-10.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-1001" src="../uploads/2021/01/episode-1-10.png" alt="" width="808" height="267" srcset="https://justobjects.nl/wp-content/uploads/2021/01/episode-1-10.png 808w, https://justobjects.nl/wp-content/uploads/2021/01/episode-1-10-300x99.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/episode-1-10-768x254.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/episode-1-10-150x50.png 150w" sizes="(max-width: 808px) 100vw, 808px" />
-->
<p>Enough theory, show me the show! As said you can watch back all shows via <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">tv.osgeo.nl</a>
</strong> or directly on the <strong><a href="https://www.youtube.com/channel/UCvSAN6ur4RoGUqxtvmgsb8g" target="_blank" rel="noreferrer noopener">OSGeo.nl YouTube Channel</a>
</strong>. Hell, you can even <strong><a href="https://www.youtube.com/playlist?list=PLJMEnRQpAfZqCkhGh3lb3KUnXssK7Sk6C" target="_blank" rel="noreferrer noopener">binge-watch all episodes</a>
</strong>!</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/combined3.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-1004" src="../uploads/2021/01/combined3.png" alt="" width="906" height="324" srcset="https://justobjects.nl/wp-content/uploads/2021/01/combined3.png 906w, https://justobjects.nl/wp-content/uploads/2021/01/combined3-300x107.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/combined3-768x275.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/combined3-150x54.png 150w" sizes="(max-width: 906px) 100vw, 906px" />
-->
<p>Many shows were organized around a theme, like 3D (<strong><a href="https://www.youtube.com/watch?v=UViczA8pvJs" target="_blank" rel="noreferrer noopener">ep. 3 offcourse</a>
</strong>), Heaven&Outer Space, Corona of the Sun (<strong><a href="https://www.youtube.com/watch?v=8qSTPE8Gkoc" target="_blank" rel="noreferrer noopener">ep. 6 on Ascension Day</a>
</strong>). We had some great guests in the show, also international guests, to name a few: Nadieh Bremer and Christian Mayer (<strong><a href="https://www.youtube.com/watch?v=zXIfCLbuVlI" target="_blank" rel="noreferrer noopener">ep. 7 – Visualization Special</a>
</strong>), Hugo Ledoux (TU Delft on 3D), Linda van den Brink (Geonovum) and Tom Kralidis (<strong><a href="https://www.youtube.com/watch?v=3l_a5Up8Rgc" target="_blank" rel="noreferrer noopener">ep 11 – on OGC API Special</a>
</strong>), Anita Graser (<strong><a href="https://www.youtube.com/watch?v=lkfVQcp3bpI" target="_blank" rel="noreferrer noopener">ep. 9 – QGIS Special</a>
</strong>), Naomi Bueno de Mesquita and Topi Tjukanov (<strong><a href="https://www.youtube.com/watch?v=rl4-tPbfxOE" target="_blank" rel="noreferrer noopener">ep 12. MapTime Special</a>
</strong>). The last show of 2020, <strong><a href="https://www.youtube.com/watch?v=HTgdEN3nb24" target="_blank" rel="noreferrer noopener">ep. 13 Christmas Edition, was one big PubQuiz</a>
</strong>.</p>
<div class="">
<a href="https://tv.osgeo.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/grote-geo-eiland-screenshot.jpg"
alt="NewYear on “Grote Geo Eiland”" />
</a>
</div>
<!--
<div id="attachment_1006" style="width: 830px" class="wp-caption aligncenter">
<a href="https://tv.osgeo.nl"><img aria-describedby="caption-attachment-1006" loading="lazy" class="size-large wp-image-1006" src="../uploads/2021/01/grote-geo-eiland-screenshot-1024x658.jpg" alt="" width="820" height="527" srcset="https://justobjects.nl/wp-content/uploads/2021/01/grote-geo-eiland-screenshot-1024x658.jpg 1024w, https://justobjects.nl/wp-content/uploads/2021/01/grote-geo-eiland-screenshot-300x193.jpg 300w, https://justobjects.nl/wp-content/uploads/2021/01/grote-geo-eiland-screenshot-768x494.jpg 768w, https://justobjects.nl/wp-content/uploads/2021/01/grote-geo-eiland-screenshot-150x96.jpg 150w, https://justobjects.nl/wp-content/uploads/2021/01/grote-geo-eiland-screenshot.jpg 1249w" sizes="(max-width: 820px) 100vw, 820px" /></a>
<p id="caption-attachment-1006" class="wp-caption-text">
<em><strong>NewYear on “Grote Geo Eiland”</strong></em>
</p>
</div>
-->
<p>All in all we did 13 shows in 2020. And 2021? We’ll start with a Newyear Special with OSGeo-NL, QGIS-NL and OpenStreetMap-NL communities. Only the plenary first half-hour in StreamYard, then next a new platform where all can participate and mingle. Just watch and participate on sun, jan 24, 2021, 3 PM (GMT+1) via <strong><a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">tv.osgeo.nl</a>
</strong>.</p>
<h3 id="geospatial-cloud-services">Geospatial Cloud Services</h3>
<p>Moving into providing Geospatial Cloud Services last few years, both as a source of income and to support/strengthen underlying open source projects with which they are developed. Warning: shameless ads below.</p>
<ul>
<li><strong><a href="https://www.map5.nl/" target="_blank" rel="noreferrer noopener">map5.nl</a>
</strong> is a subscription service for Dutch topographic, historical- and embellished hill-shade and arial maps I started to host in 2015. In 2020 the entire map5.nl server-infrastructure was moved from custom Ubuntu-installs to a complete <strong><a href="https://www.ansible.com/" target="_blank" rel="noreferrer noopener">Ansible</a>
</strong>/Docker-based setup.</li>
</ul>
<div class="">
<a href="https://www.map5.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/map5.nl_.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-917" src="../uploads/2020/01/map5.nl_.png" alt="" width="800" height="592" srcset="https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_.png 800w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-300x222.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-768x568.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-150x111.png 150w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<ul>
<li><strong><a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
</strong> is a hosted <strong><a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
</strong> (GHC) service on a subscription basis. GHC is an uptime and QoS monitor for (OGC) web services. Customers get their own GHC instance. <strong><a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
</strong> saves the burden of self-hosting GHC. Launch was in 2019, growing steady in 2020. Expect to work more on API and new UI for GeoHealthCheck (below) in 2021.</li>
</ul>
<div class="">
<a href="https://geoqos.com/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/geoqos.com_.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter wp-image-915 size-full" src="../uploads/2020/01/geoqos.com_.png" alt="" width="952" height="563" srcset="https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_.png 952w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-300x177.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-768x454.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-150x89.png 150w" sizes="(max-width: 952px) 100vw, 952px" />
-->
<ul>
<li>Feb 2020 – launched <strong><a href="https://geotoko.nl" target="_blank" rel="noreferrer noopener">geotoko.nl</a>
</strong>. geotoko.nl, in short GeoToko, is basically a webshop to download Dutch open geo-datasets. Here one may ask: Dutch geospatial data like Topography, Addresses and Buildings, is already open and publicly available, mainly via <strong><a href="https://pdok.nl" target="_blank" rel="noreferrer noopener">Kadaster-PDOK</a>
</strong>. So why bother reselling? This needs some explanation.</li>
</ul>
<div class="">
<a href="https://geotoko.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/opentopo-keyreg.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-953" src="../uploads/2021/01/opentopo-keyreg.png" alt="" width="558" height="188" srcset="https://justobjects.nl/wp-content/uploads/2021/01/opentopo-keyreg.png 558w, https://justobjects.nl/wp-content/uploads/2021/01/opentopo-keyreg-300x101.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/opentopo-keyreg-150x51.png 150w" sizes="(max-width: 558px) 100vw, 558px" />
-->
<p>Most Dutch Open geospatial datasets, as available from governmental institutions like Kadaster, are provided in a neutral exchange-format. In practice: GML (Application Schema). Most users are not able to directly digest complex GML in their applications (or don’t want to spend time on that). For over 10 years we run the Open Source project <strong><a href="https://github.com/nlextract/NLExtract" target="_blank" rel="noreferrer noopener">NLExtract</a>
</strong> to convert these datasets into formats like PostGIS and CSV that can be directly used in applications. NLExtract itself builds on <strong><a href="https://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl</a>
</strong>, an Open Source geospatial ETL-library in Python. NLExtract/Stetl is used a lot, but still will require users to install/maintain an NLExtract/Stetl installation, download datasets, run the ETL, check results etc. That may also be a bridge too far, if e.g. all a customer needs is a one-time CSV of say, all 10 million addresses in The Netherlands. So GeoToko taps into this niche, providing ready-to-use, often enriched Dutch datasets. Pricing is reasonable, compared to other providers, plus organizations and individuals may get highly reduced pricing when they work on Open data applications like OpenStreetMap.</p>
<div class="">
<a href="https://geotoko.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/bigpicture.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter wp-image-954 size-full" src="../uploads/2021/01/bigpicture.png" alt="GeoToko High-level Architecture" width="500" height="310" srcset="https://justobjects.nl/wp-content/uploads/2021/01/bigpicture.png 500w, https://justobjects.nl/wp-content/uploads/2021/01/bigpicture-300x186.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/bigpicture-150x93.png 150w" sizes="(max-width: 500px) 100vw, 500px" />
-->
<p>As for development: the <strong><a href="https://geotoko.nl" target="_blank" rel="noreferrer noopener">GeoToko webshop</a>
</strong> was developed with Django, Flask/nginx (Download management), Stripe (backend payments) and <strong><a href="https://ckan.org" target="_blank" rel="noreferrer noopener">CKAN</a>
</strong>. The latter to provide a product-catalogue, sample data and metadata in general, all via <strong><a href="https://geocatalogus.nl" target="_blank" rel="noreferrer noopener">GeoCatalogus.nl</a>
</strong>.</p>
<div class="">
<a href="https://geotoko.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/geofabriek-keten.png"
alt="GeoFabriek – Production Chain" />
</a>
</div>
<!--
<div id="attachment_956" style="width: 924px" class="wp-caption aligncenter">
<a href="https://geotoko.nl"><img aria-describedby="caption-attachment-956" loading="lazy" class="size-full wp-image-956" src="../uploads/2021/01/geofabriek-keten.png" alt="" width="914" height="287" srcset="https://justobjects.nl/wp-content/uploads/2021/01/geofabriek-keten.png 914w, https://justobjects.nl/wp-content/uploads/2021/01/geofabriek-keten-300x94.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/geofabriek-keten-768x241.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/geofabriek-keten-150x47.png 150w" sizes="(max-width: 914px) 100vw, 914px" /></a>
<p id="caption-attachment-956" class="wp-caption-text">
<strong>GeoFabriek – Production Chain</strong>
</p>
</div>
-->
<p>Further development concerned automating the NLExtract/Stetl ETL-processes. For this a framework called GeoFabriek, “GeoFactory”, was developed. This automates the entire chain from checking new dataset versions at Kadaster, through downloading, conversions, packing for download and updating the metadata (in GeoCatalogus.nl).</p>
<h3 id="contract-work">Contract Work</h3>
<p>As my focus is more and more on providing “Geospatial Cloud Services” (see above), did not take not too much contract work in 2020, though I am always open for offerings (but may say ‘no’)!</p>
<ul>
<li>
<p>For <strong><a href="https://www.geolicious.de/" target="_blank" rel="noreferrer noopener">Geolicious (Germany)</a>
</strong>. Developed a Wegue <strong><a href="https://map.naturpark-lueneburger-heide.de/" target="_blank" rel="noreferrer noopener">interactive map for the German National Park Luneburger Heide</a>
</strong>. This also gave a boost to the <strong><a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue Open Source project</a>
</strong> (see below). Was an honour to work together with the great <strong><a href="https://www.opengeospatial.org/blog/2996" target="_blank" rel="noreferrer noopener">Steve Bennett</a>
</strong> a.k.a. @stevage.</p>
<div class="">
<a href="https://map.naturpark-lueneburger-heide.de/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/screenshot-1.png"
alt="" />
</a>
</div>
</li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-971" src="../uploads/2021/01/screenshot-1.png" alt="" width="800" height="524" srcset="https://justobjects.nl/wp-content/uploads/2021/01/screenshot-1.png 800w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-1-300x197.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-1-768x503.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-1-150x98.png 150w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<p>Various new Widgets for Wegue were developed. To name a few: Enhanced Layer Tree, Enhanced Feature Info (see picture), Share Button, Routing, Multi-language, PDF Print, Download Features, Layout Improvements. Also introduced a “create-app” to have a starter app. We are in the process of merging back these new features into the Wegue core.</p>
<h3 id="open-source-contributions">Open Source Contributions</h3>
<p>Continuous work as a contributor. Apart from some GitLab Projects, you can find/follow me best on <a href="https://github.com/justb4/" target="_blank" rel="noreferrer noopener">GitHub</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2021/01/contribs-2020.png" alt="GitHub contributions"></p>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-965" src="../uploads/2021/01/contribs-2020.png" alt="" width="773" height="206" srcset="https://justobjects.nl/wp-content/uploads/2021/01/contribs-2020.png 773w, https://justobjects.nl/wp-content/uploads/2021/01/contribs-2020-300x80.png 300w, https://justobjects.nl/wp-content/uploads/2021/01/contribs-2020-768x205.png 768w, https://justobjects.nl/wp-content/uploads/2021/01/contribs-2020-150x40.png 150w" sizes="(max-width: 773px) 100vw, 773px" />
-->
<ul>
<li><a href="https://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl</a>
– Geospatial ETL in Python, maintaining since 2011.</li>
<li><a href="https://geohealthcheck.org" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
– Service Status and QoS Checker for OGC Web Services.</li>
<li><a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
– a Python server implementation of the <a href="https://www.opengeospatial.org/blog/2996" target="_blank" rel="noreferrer noopener">OGC API suite of standards</a>
– joined this great project in 2019. Also in PSC.</li>
<li><a href="https://demo.pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi demo server</a>
– provided the (auto-)deployment stack for the pygeoapi demo server (<a href="https://github.com/geopython/demo.pygeoapi.io" target="_blank" rel="noreferrer noopener">code at GitHub</a>
). Added <a href="https://demo.pygeoapi.io/covid-19" target="_blank" rel="noreferrer noopener">COVID-19 NL data provider</a>
.</li>
<li><a href="https://nlextract.nl/" target="_blank" rel="noreferrer noopener">NLExtract</a>
– ETL for Dutch geospatial datasets.</li>
<li><a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
– geo-webclient framework based on <a href="https://vuejs.org/" target="_blank" rel="noreferrer noopener">Vue.js</a>
with OpenLayers started by <a href="https://github.com/chrismayer" target="_blank" rel="noreferrer noopener">Christian Mayer</a>
. Joined this great project in 2019.</li>
<li><a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron MC</a>
– Web Mapping Client based on <a href="https://www.geoext.org/" target="_blank" rel="noreferrer noopener">GeoExt</a>
and OpenLayers. Yes, old tech but still in wide use. In time hope to migrate to <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
(see above). Migrating Heron to Wegue, see first example below.</li>
</ul>
<div class="">
<a href="https://wegue.heron-mc.org/">
<img class="lazy"
src="https://justobjects.nl/uploads/2021/01/screenshot-heron-wegue-1.jpg"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-974" src="../uploads/2021/01/screenshot-heron-wegue-1.jpg" alt="" width="800" height="568" srcset="https://justobjects.nl/wp-content/uploads/2021/01/screenshot-heron-wegue-1.jpg 800w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-heron-wegue-1-300x213.jpg 300w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-heron-wegue-1-768x545.jpg 768w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-heron-wegue-1-150x107.jpg 150w, https://justobjects.nl/wp-content/uploads/2021/01/screenshot-heron-wegue-1-285x201.jpg 285w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<ul>
<li><a href="https://mapproxy.org" target="_blank" rel="noreferrer noopener">MapProxy</a>
– joined Program Steering Committee (PSC)</li>
<li><a href="https://access.crunchydata.com/documentation/pg_tileserv/latest/" target="_blank" rel="noreferrer noopener">pg_tileserv</a>
Vector Tiles directly from PostGIS!<br>
Contributed <a href="https://github.com/CrunchyData/pg_tileserv/blob/master/Dockerfile.alpine" target="_blank" rel="noreferrer noopener">Dockerfile</a>
and <a href="https://github.com/CrunchyData/pg_tileserv/tree/master/examples/docker" target="_blank" rel="noreferrer noopener">Docker-example</a>
</li>
</ul>
<h3 id="more-contributions-8211-handy-docker-images">More Contributions – Handy Docker Images</h3>
<p>To support many of the Cloud services and Open Source projects, I develop and maintain <a href="https://github.com/search?q=user%3Ajustb4+docker-" target="_blank" rel="noreferrer noopener">handy Docker Images</a>
, also available from <a href="https://hub.docker.com/u/justb4" target="_blank" rel="noreferrer noopener">my DockerHub</a>
.</p>
<ul>
<li><a href="https://github.com/justb4/docker-awstats" target="_blank" rel="noreferrer noopener">docker-awstats</a>
– AWStats in Docker, oldie, but very effective webstats. Deploy multiple instances in single Docker container. Highly configurable, e.g. also for <a href="https://docs.traefik.io/" target="_blank" rel="noreferrer noopener">Traefik</a>
access logs.</li>
<li><a href="https://github.com/justb4/docker-jmeter" target="_blank" rel="noreferrer noopener">docker-jmeter</a>
– <a href="https://jmeter.apache.org/" target="_blank" rel="noreferrer noopener">Apache JMeter</a>
wrapped in Docker. Over 1 million pulls!</li>
<li><a href="https://github.com/justb4/docker-mapfish-print" target="_blank" rel="noreferrer noopener">docker-mapfish-print</a>
– for MapFish Print <strong>version 3</strong>. MapFish Print allows printing maps as PDFs.</li>
<li><a href="https://github.com/justb4/docker-mapfish-print2" target="_blank" rel="noreferrer noopener">docker-mapfish-print2</a>
– for MapFish Print <strong>version 2</strong> – This version is still used in quite some contexts, at least for <a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron</a>
and <a href="https://kadviewer.map5.nl/" target="_blank" rel="noreferrer noopener">KadViewer</a>
.</li>
<li><a href="https://github.com/justb4/docker-rclone" target="_blank" rel="noreferrer noopener">docker-rclone</a>
– Docker image to perform a <a href="http://rclone.org/" target="_blank" rel="noreferrer noopener">rclone</a>
sync based on a cron schedule, with <a href="https://healthchecks.io/" target="_blank" rel="noreferrer noopener">healthchecks.io</a>
monitoring.</li>
<li><a href="https://github.com/justb4/docker-pgbackup" target="_blank" rel="noreferrer noopener">docker-pgbackup</a>
– automated/scheduled PostgreSQL/PostGIS backups for all PostgreSQL-based Docker Containers in its Docker-network. I think it is nifty: just run and forget: your Postgres backups are taken care of. Inspired by: <a href="https://github.com/kartoza/docker-pg-backup">https://github.com/kartoza/docker-pg-backup</a>.</li>
</ul>
<p>New Docker images developed in 2020:</p>
<ul>
<li><a href="https://github.com/justb4/docker-mapserver" target="_blank" rel="noreferrer noopener">docker-mapserver</a>
– Slim Docker Image for MapServer with Lighttpd FastCGI.</li>
<li><a href="https://github.com/justb4/docker-mapproxy" target="_blank" rel="noreferrer noopener">docker-mapproxy</a>
– MapProxy Docker Image adapted, slimmed, from the <a href="https://yagajs.org/" target="_blank" rel="noreferrer noopener">YAGA Development-Team</a>
. <a href="https://github.com/yagajs/docker-mapproxy/pull/18" target="_blank" rel="noreferrer noopener">Awaiting PR merge</a>
.</li>
<li><a href="https://github.com/justb4/docker-mapproxy-mapserver" target="_blank" rel="noreferrer noopener">docker-mapproxy-mapserver</a>
– Docker Image for MapProxy service with built-in MapServer binaries accessed directly (no MapServer service).</li>
<li><a href="https://github.com/justb4/docker-cron" target="_blank" rel="noreferrer noopener">docker-cron</a>
– Runs Unix cron, includes docker (compose) client for running remote Docker Images.</li>
</ul>
<h3 id="osgeonl">OSGeo.nl</h3>
<p>As chair of the board, still involved in the OSGeo Dutch Local Chapter, <a href="https://osgeo.nl" target="_blank" rel="noreferrer noopener">OSGeo.nl</a>
since its establishment in 2011.</p>
<ul>
<li>Jan 12 – organizer – joint <a href="https://geoforum.nl/t/osgeo-nl-en-openstreetmap-nl-nieuwjaarsborrel-zo-13-jan-2019-hilversum/2170" target="_blank" rel="noreferrer noopener">OSGeo.nl and OpenStreetMap NL new-years party</a>
at Cafe Dudok in Hilversum. Our last IRL event in 2020….</li>
</ul>
<p>No FOSS4G-NL, plans are for okt 2021, but hey, 13 episodes of <a href="https://tv.osgeo.nl" target="_blank" rel="noreferrer noopener">De Grote Geo Show</a>
(see above)!</p>
<h3 id="conferences-8211-attended">Conferences – Attended</h3>
<ul>
<li>Feb 20-21 – <a href="https://vuejs.amsterdam/" target="_blank" rel="noreferrer noopener">Vue.js Amsterdam</a>
– last physically attended conf in 2020</li>
<li>Sept 21-22 – <a href="http://2020.geopython.net/" target="_blank" rel="noreferrer noopener">GeoPython 2019</a>
– Virtual</li>
<li>Nov 19 – <a href="https://info.crunchydata.com/en/postgis-day-2020-crunchy-data" target="_blank" rel="noreferrer noopener">PostGIS Day 2020</a>
– by crunchydata.com</li>
</ul>
<p>Had planned: FOSSGIS-DE Freiburg (Germany), FOSS4G-EU Valmiera (Latvia), GeoPython 2020 Bilbao (Spain), FOSS4G 2020 Calgary (Canada) and OSGeo Code Sprint Athens (Greece). Was studying train-tables for all EU events…Oh well.</p>
<h3 id="talks--workshops-8211-provided">Talks & Workshops – Provided</h3>
<p>Most of my slides on <a href="https://www.slideshare.net/justb4/" target="_blank" rel="noreferrer noopener">slideshare.net/justb4</a>
. Below my 2020 talks.</p>
<ul>
<li>Sept 21-22 – <a href="http://2020.geopython.net/schedule.html" target="_blank" rel="noreferrer noopener">Python Machine Learning &amp; GeoPython 2020</a>
– presented pygeoapi with Francesco Bartoli – [<a href="https://pygeoapi.io/presentations/default/#/frontpage" target="_blank" rel="noreferrer noopener">slides HTML</a>
]</li>
<li>Dec 7-9 – GeoNode Summit 2020 – <a href="https://summit.geonode.org/schedule/#session-106" target="_blank" rel="noreferrer noopener">presented pygeoapi</a>
with Francesco Bartoli, Tom Kralidis, Angelos Tzotsos – [<a href="https://pygeoapi.io/presentations/default/#/frontpage" target="_blank" rel="noreferrer noopener">slides HTML</a>
]</li>
<li>Nov 10 – GeoHealthCheck/<a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">geoqos.com</a>
workshop for <a href="https://www.rivm.nl/en" target="_blank" rel="noreferrer noopener">RIVM</a>
(National Institute for Public Health and the Environment)</li>
</ul>
<p>Probably some more, but cannot find back at this moment.</p>
<h3 id="resolutions-2021">Resolutions 2021</h3>
<ul>
<li>More effort into <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
project</li>
<li>Get back into Kubernetes and containerism in general</li>
<li>Revive old GeoTracing projects like <a href="http://georambling.com" target="_blank" rel="noreferrer noopener">georambling.com</a>
</li>
<li>HA projects: weather station, wildlife/bird cam, and AQ monitoring with <a href="http://www.home-assistant.io/" target="_blank" rel="noreferrer noopener">Home Assistant</a>
</li>
<li>More hiking and rambling</li>
<li>Whatever comes around.</li>
</ul>
-
Cheers to 2019
https://justobjects.nl/cheers-to-2019/
Fri, 17 Jan 2020 01:38:19 +0000
https://justobjects.nl/cheers-to-2019/
<p>Overview of my professional life in 2019. Inspired by the concise bullet-point-style of Tom Kralidis’ <a href="http://www.kralidis.ca/blog/2018/12/28/cheers-to-2018/" target="_blank" rel="noreferrer noopener">Cheers to 2018</a>
, on which I based <a href="https://justobjects.nl/cheers-to-2018/" target="_blank" rel="noreferrer noopener">my previous Cheers to 2018.</a>
Highlights of living and working in the Open Source Geospatial and OSGeo(<a href="https://osgeo.nl/" target="_blank" rel="noreferrer noopener">.nl</a>
|.<a href="https://www.osgeo.org/" target="_blank" rel="noreferrer noopener">org</a>
)-world in 2019. Organized by “Theme” i.s.o. by month.</p>
<p>TL;DR. My 2019 highlight was providing the <a href="https://geopython.github.io/geopython-workshop/" target="_blank" rel="noreferrer noopener">GeoPython Workshop</a>
(“Doing Geospatial with Python”) at the <a href="https://2019.foss4g.org/schedule/workshops/" target="_blank" rel="noreferrer noopener">FOSS4G in Bukarest</a>
. A really great team-effort: first remote collaboration to get the content done. Then at the spot, despite network failures, providing a hopefully inspiring workshop on modern GeoPython. Kudos to Tom, Angelos, Francesco, Jachym, Luis and Jorge!</p>
<p><img src="https://justobjects.nl/uploads/2020/01/geopython-workshop-team.jpg" alt="The GeoPython Workshop Team at FOSS4G 2019 – Bukarest"></p>
<!--
<div id="attachment_924" style="width: 830px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-924" loading="lazy" class="size-large wp-image-924" src="uploads/2020/01/geopython-workshop-team-1024x768.jpg" alt="" width="820" height="615" srcset="https://justobjects.nl/wp-content/uploads/2020/01/geopython-workshop-team-1024x768.jpg 1024w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-workshop-team-300x225.jpg 300w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-workshop-team-768x576.jpg 768w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-workshop-team-150x113.jpg 150w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-workshop-team.jpg 1200w" sizes="(max-width: 820px) 100vw, 820px" />
<p id="caption-attachment-924" class="wp-caption-text">
The GeoPython Workshop Team at FOSS4G 2019 – Bukarest.
</p>
</div>
-->
<p>My second highlight was joining the <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
project. Had a <a href="https://justobjects.nl/2nd-time-around-wfs-v3-pygeoapi/" target="_blank" rel="noreferrer noopener">love/hate relationship with WFS</a>
, but the new <a href="https://www.opengeospatial.org/standards/ogcapi-features" target="_blank" rel="noreferrer noopener">OpenAPI direction in OGC</a>
and the great team behind the pygeoapi project made me want to be part of this. BTW pygeoapi <span style="font-weight: 400;">has just (Jan. 2020) received </span><a href="https://www.opengeospatial.org/resource/products/details/?pid=1606" target="_blank" rel="noreferrer noopener"><span style="font-weight: 400;">OGC Compliance Certification and Reference Implementation Status</span></a>
<span style="font-weight: 400;">for OGC API – Features specification.</span></p>
<h3 id="geospatial-cloud-services">Geospatial Cloud Services</h3>
<p>Main focus. Seriously moving into hosting Geospatial Cloud Services, both as a source of income and to support/strengthen underlying open source projects. Warning: shameless ads below.</p>
<ul>
<li>
<p>Throughout 2019 – expanded <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
, a subscription service for Dutch topographic, historical- and embellished hill-shade and arial maps I started to host in 2015.</p>
<div class="">
<a href="https://map5.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/map5.nl_.png"
alt="" />
</a>
</div>
</li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-917" src="uploads/2020/01/map5.nl_.png" alt="" width="800" height="592" srcset="https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_.png 800w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-300x222.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-768x568.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/map5.nl_-150x111.png 150w" sizes="(max-width: 800px) 100vw, 800px" />
-->
<ul>
<li>
<p>January – launched <a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
, a Cloud-hosted <a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
(GHC) service on a subscription basis. GHC is is an uptime and QoS monitor for (OGC) web services. Customers get their own GHC instance. <a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
saves the burden of self-hosting GHC. Truly, I can’t do without GHC for any of my geospatial web-services (like <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
). Developed with <a href="https://www.djangoproject.com/" target="_blank" rel="noreferrer noopener">Python Django</a>
and <a href="https://stripe.com/" target="_blank" rel="noreferrer noopener">Stripe</a>
, deployed with <a href="https://www.ansible.com/" target="_blank" rel="noreferrer noopener">Ansible</a>
(can’t do without) and off course <a href="https://www.docker.com/" target="_blank" rel="noreferrer noopener">Docker</a>
.</p>
<div class="">
<a href="https://geoqos.com/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/geoqos.com_.png"
alt="" />
</a>
</div>
</li>
</ul>
<!--
[<img loading="lazy" class="aligncenter wp-image-915 size-full" src="uploads/2020/01/geoqos.com_.png" alt="" width="952" height="563" srcset="https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_.png 952w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-300x177.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-768x454.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos.com_-150x89.png 150w" sizes="(max-width: 952px) 100vw, 952px" />][13]
-->
<ul>
<li>Dec+ into 2020 – something big – more to be announced.</li>
</ul>
<h3 id="contract-work">Contract Work</h3>
<p>As my focus is more and more on providing “Geospatial Cloud Services” (see above), not too much contract work in 2019, though I am always open for offerings!</p>
<ul>
<li>
<p>For the <a href="https://ec.europa.eu/jrc/en" target="_blank" rel="noreferrer noopener">European Commission Joint Research Center</a>
(Ispra, Italy) -deployed five <a href="https://airsenseur.org/" target="_blank" rel="noreferrer noopener">AirSensEUR</a>
air-quality sensor-stations (NO2, NO, CO, O3) within the Netherlands, connecting these to the <a href="https://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
. <a href="https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/">This blog-post</a>
describes the overall setup and this <a href="https://justobjects.nl/emit-6-airsenseur-calibration">post the really good calibration results</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09.png" alt=" "></p>
</li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-large wp-image-880" src="uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09-1024x340.png" alt="" width="820" height="272" srcset="https://justobjects.nl/wp-content/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09-1024x340.png 1024w, https://justobjects.nl/wp-content/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09-300x100.png 300w, https://justobjects.nl/wp-content/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09-768x255.png 768w, https://justobjects.nl/wp-content/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09-150x50.png 150w" sizes="(max-width: 820px) 100vw, 820px" />
-->
<ul>
<li>
<p>For <a href="https://www.nijmegen.nl/" target="_blank" rel="noreferrer noopener">Gemeente Nijmegen</a>
– integrated <a href="https://luftdaten.info/" target="_blank" rel="noreferrer noopener">Luftdaten.info</a>
finedust (PM10, PM25) measurements within the <a href="https://data.smartemission.nl" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
via Harvesting.</p>
<div class="">
<a href="https://data.smartemission.nl/smartapp/">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/luftdaten-smartapp.png"
alt="" />
</a>
</div>
</li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-904" src="uploads/2020/01/luftdaten-smartapp.png" alt="" width="400" height="294" srcset="https://justobjects.nl/wp-content/uploads/2020/01/luftdaten-smartapp.png 400w, https://justobjects.nl/wp-content/uploads/2020/01/luftdaten-smartapp-300x221.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/luftdaten-smartapp-150x110.png 150w" sizes="(max-width: 400px) 100vw, 400px" />
-->
<ul>
<li>For <a href="https://www.stantec.com/" target="_blank" rel="noreferrer noopener">Stantec</a>
NL – custom <a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
Plugins (Probes/Checks) for ESRI ArcGIS Feature services, plus several Auth-facilities (Basic, Token) for auth-based web-services.</li>
<li>For <a href="https://coin.nl/nl/home" target="_blank" rel="noreferrer noopener">COIN NL</a>
– <a href="https://nlextract.nl/" target="_blank" rel="noreferrer noopener">NLExtract Project</a>
– added shortened streetname (<span style="color: #993300;">verkorteopenbareruimtenaam</span>, a Scrabble-winner, yes!) to <a href="https://zakelijk.kadaster.nl/bag" target="_blank" rel="noreferrer noopener">BAG</a>
(Dutch Adresses and Buildings Dataset) Postgres Dump and Address CSV.</li>
</ul>
<h3 id="open-source-contributions">Open Source Contributions</h3>
<p>Continuous work as a contributor on several Open Source Projects. Apart from some GitLab Projects, you can find/follow me best on <a href="https://github.com/justb4/" target="_blank" rel="noreferrer noopener">GitHub</a>
.</p>
<div class="">
<a href="https://github.com/justb4?tab=overview&from=2019-01-01&to=2019-12-31">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/gh-contribs-2019-justb4.png"
alt="" />
</a>
</div>
<!--
<img loading="lazy" class="size-full wp-image-901" src="uploads/2020/01/gh-contribs-2019-justb4.png" alt="GitHub Contribitions 2019" width="784" height="202" srcset="https://justobjects.nl/wp-content/uploads/2020/01/gh-contribs-2019-justb4.png 784w, https://justobjects.nl/wp-content/uploads/2020/01/gh-contribs-2019-justb4-300x77.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/gh-contribs-2019-justb4-768x198.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/gh-contribs-2019-justb4-150x39.png 150w" sizes="(max-width: 784px) 100vw, 784px" />
-->
<ul>
<li><a href="https://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl</a>
– Geospatial ETL in Python, maintaining since 2011. Highlight: Py2 to Py3 migration, many thanks: <a href="https://github.com/borrob" target="_blank" rel="noreferrer noopener">Rob van Loon</a>
.</li>
<li><a href="https://geohealthcheck.org" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
– Service Status and QoS Checker for OGC Web Services – <a href="https://github.com/geopython/GeoHealthCheck/pulls?q=is%3Apr+is%3Aclosed" target="_blank" rel="noreferrer noopener">39 PRs</a>
went in! Highlight: Py2 to Py3 migration, again many thanks: <a href="https://github.com/borrob" target="_blank" rel="noreferrer noopener">Rob van Loon</a>
. Even added ESRI Feature Server plugin!</li>
<li><a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
– a Python server implementation of the <a href="https://www.opengeospatial.org/blog/2996" target="_blank" rel="noreferrer noopener">OGC API suite of standards</a>
– joined this great project in 2019. Several contributions: a.o. the <a href="https://github.com/geopython/pygeoapi/blob/master/pygeoapi/provider/ogr.py" target="_blank" rel="noreferrer noopener">GDAL/OGR backend Provider</a>
.</li>
<li><a href="https://demo.pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi demo server</a>
– provided the (auto-)deployment stack for the pygeoapi demo server (<a href="https://github.com/geopython/demo.pygeoapi.io" target="_blank" rel="noreferrer noopener">code at GitHub</a>
).</li>
<li><a href="https://nlextract.nl/" target="_blank" rel="noreferrer noopener">NLExtract</a>
– ongoing development – data downloads attract quite some attention.</li>
<li><a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
– geo-webclient framework based on <a href="https://vuejs.org/" target="_blank" rel="noreferrer noopener">Vue.js</a>
with OpenLayers started by <a href="https://github.com/chrismayer" target="_blank" rel="noreferrer noopener">Christian Mayer</a>
. Joined this great project in 2019 with some small patches, adding <a href="https://github.com/meggsimum/wegue/blob/master/Dockerfile" target="_blank" rel="noreferrer noopener">Dockerfile</a>
, <a href="https://github.com/meggsimum/wegue/pull/82" target="_blank" rel="noreferrer noopener">projection-support</a>
and still working on <a href="https://github.com/meggsimum/wegue/pull/95" target="_blank" rel="noreferrer noopener">GeoCoder PR</a>
.</li>
<li><a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron MC</a>
– Web Mapping Client based on <a href="https://www.geoext.org/" target="_blank" rel="noreferrer noopener">GeoExt</a>
and OpenLayers. Yes, old tech but still in wide use. For one thing the <a href="https://kadviewer.map5.nl/" target="_blank" rel="noreferrer noopener">KadViewer</a>
within map5.nl attracts many customers. In time hope to migrate to <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
(see above).</li>
<li><a href="https://github.com/justb4/stademo" target="_blank" rel="noreferrer noopener">SensorThings API demo server</a>
– Geonovum organized a <a href="https://www.geonovum.nl/over-geonovum/agenda/hackathon-sensorthings-api" target="_blank" rel="noreferrer noopener">SensorThings API Hackathon</a>
in oct 2019. See <a href="https://github.com/Geonovum/SensorthingsAPI" target="_blank" rel="noreferrer noopener">results here</a>
. Used the <a href="https://www.gostserver.xyz/" target="_blank" rel="noreferrer noopener">Geodan GOST</a>
STA server. Also added a <a href="https://github.com/Geonovum/SensorthingsAPI/blob/master/sta-data-upload/postman-mqtt/README.md" target="_blank" rel="noreferrer noopener">workflow/pipeline</a>
for adding data for the <a href="https://www.sensemakers.sda-projects.nl/projects/mijnomgeving/" target="_blank" rel="noreferrer noopener">Amsterdam Sensemakers Water “mijnomgeving”</a>
measurements. This used a MQTT-NodeRed-STA mapping.</li>
<li><a href="https://geopython.github.io/OWSLib/" target="_blank" rel="noreferrer noopener">OWSLib</a>
– various <a href="https://github.com/geopython/OWSLib/commits?author=justb4" target="_blank" rel="noreferrer noopener">small contributions</a>
.</li>
</ul>
<h3 id="more-contributions-8211-handy-docker-images">More Contributions – Handy Docker Images</h3>
<p>To support many of the Cloud services and Open Source projects, I developed several handy Docker Images, also available from <a href="https://hub.docker.com/u/justb4" target="_blank" rel="noreferrer noopener">my DockerHub</a>
.</p>
<ul>
<li>
<p><a href="https://github.com/justb4/docker-awstats" target="_blank" rel="noreferrer noopener">docker-awstats</a>
– AWStats in Docker, oldie, but very effective webstats. Deploy multiple instances in single Docker container. Highly configurable, e.g. also for <a href="https://docs.traefik.io/" target="_blank" rel="noreferrer noopener">Traefik</a>
access logs.</p>
</li>
<li>
<p><a href="https://github.com/justb4/docker-jmeter" target="_blank" rel="noreferrer noopener">docker-jmeter</a>
– <a href="https://jmeter.apache.org/" target="_blank" rel="noreferrer noopener">Apache JMeter</a>
wrapped in Docker.</p>
<div class="">
<a href="https://hub.docker.com/u/justb4">
<img class="lazy"
src="https://justobjects.nl/uploads/2020/01/justb4-docker-images.png"
alt="" />
</a>
</div>
</li>
</ul>
<!--
<a href="https://hub.docker.com/u/justb4" target="_blank" rel="noopener noreferrer"><img loading="lazy" class="aligncenter wp-image-906 size-full" title="https://hub.docker.com/u/justb4" src="uploads/2020/01/justb4-docker-images.png" alt="" width="648" height="805" srcset="https://justobjects.nl/wp-content/uploads/2020/01/justb4-docker-images.png 648w, https://justobjects.nl/wp-content/uploads/2020/01/justb4-docker-images-241x300.png 241w, https://justobjects.nl/wp-content/uploads/2020/01/justb4-docker-images-121x150.png 121w" sizes="(max-width: 648px) 100vw, 648px" /></a>
-->
<p>Not too many Docker Image downloads, but look at <a href="https://github.com/justb4/docker-jmeter" target="_blank" rel="noreferrer noopener">docker-jmeter,</a>
over 1 million! Glad to give to <a href="https://en.wikipedia.org/wiki/Commons" target="_blank" rel="noreferrer noopener">The Commons</a>
.</p>
<ul>
<li><a href="https://github.com/justb4/docker-mapfish-print" target="_blank" rel="noreferrer noopener">docker-mapfish-print</a>
– for MapFish Print <strong>version 3</strong>. MapFish Print allows printing maps as PDFs.</li>
<li><a href="https://github.com/justb4/docker-mapfish-print2" target="_blank" rel="noreferrer noopener">docker-mapfish-print2</a>
– for MapFish Print <strong>version 2</strong> – This version is still used in quite some contexts, at least for <a href="https://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron</a>
and <a href="https://kadviewer.map5.nl/" target="_blank" rel="noreferrer noopener">KadViewer</a>
.</li>
<li><a href="https://github.com/justb4/docker-rclone" target="_blank" rel="noreferrer noopener">docker-rclone</a>
– Docker image to perform a <a href="http://rclone.org/" target="_blank" rel="noreferrer noopener">rclone</a>
sync based on a cron schedule, with <a href="https://healthchecks.io/" target="_blank" rel="noreferrer noopener">healthchecks.io</a>
monitoring.</li>
<li><a href="https://github.com/justb4/docker-pgbackup" target="_blank" rel="noreferrer noopener">docker-pgbackup</a>
– automated/scheduled PostgreSQL/PostGIS backups for all PostgreSQL-based Docker Containers in its Docker-network. I think it is nifty: just run and forget: your Postgres backups are taken care of. Inspired by: <a href="https://github.com/kartoza/docker-pg-backup">https://github.com/kartoza/docker-pg-backup</a>.</li>
</ul>
<h3 id="osgeonl">OSGeo.nl</h3>
<p>Now as chair of the board, still involved in the OSGeo Dutch Local Chapter, <a href="https://osgeo.nl" target="_blank" rel="noreferrer noopener">OSGeo.nl</a>
since its establishment in 2011. Thanks to our wonderful volunteers, we were able to organize several events.</p>
<ul>
<li>Jan 13 – organizer – joint <a href="https://www.meetup.com/OSGeoNL/events/256712195/" target="_blank" rel="noreferrer noopener">OSGeo.nl and OpenStreetMap NL new-years party</a>
at Cafe Dudok in Hilversum.</li>
<li>April – Docker (for geospatial) Workshop by <a href="https://www.merkator.com/" target="_blank" rel="noreferrer noopener">Merkator</a>
</li>
<li>June 20 – <a href="https://foss4g.nl/" target="_blank" rel="noreferrer noopener">FOSS4GNL</a>
– our yearly conference</li>
</ul>
<h3 id="conferences-8211-attended">Conferences – Attended</h3>
<ul>
<li>
<p>Jan 31-Feb 1 – <a href="https://www.thethingsnetwork.org/conference/" target="_blank" rel="noreferrer noopener">TheThingsNetwork Conference</a>
. About <a href="https://en.wikipedia.org/wiki/LoRa" target="_blank" rel="noreferrer noopener">Lora(WAN)</a>
.</p>
</li>
<li>
<p>Feb 21 – GeoDocker Workshop Merkator</p>
</li>
<li>
<p>Feb 14-15 – <a href="https://vuejs.amsterdam/" target="_blank" rel="noreferrer noopener">Vue.js Amsterdam</a>
</p>
</li>
<li>
<p>June 20 – <a href="https://foss4g.nl/" target="_blank" rel="noreferrer noopener">FOSS4G-NL</a>
.</p>
</li>
<li>
<p>June 24-25 – <a href="http://2019.geopython.net/" target="_blank" rel="noreferrer noopener">GeoPython 2019</a>
– Basel.</p>
</li>
<li>
<p>August 25-31 – <a href="https://2019.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G – Bukarest</a>
.</p>
</li>
<li>
<p>Sept 20 – Sensemakers Amsterdam</p>
</li>
<li>
<p>Okt 2 – Geo Gebruikers festival by Geonovum – Amersfoort</p>
</li>
</ul>
<h3 id="hackathons--code-sprints">Hackathons & Code Sprints</h3>
<p>I always love to go to hackathons. From software to hardware-hacking with the <a href="https://www.sensemakersams.org/" target="_blank" rel="noreferrer noopener">Sensemakers Amsterdam</a>
.</p>
<ul>
<li>April 5 – OGG Hackathon by <a href="https://www.opengeogroep.nl/" target="_blank" rel="noreferrer noopener">OpenGeoGroep</a>
. Random hacking day where I got involved in <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
the first time.</li>
<li>May 14-17 – OSGeo Codesprint (attended remotely).</li>
<li>June 3-6 – OGC API Features – “WFS3 Werkweek” by Geonovum – <a href="https://geonovum.github.io/wfs3-experiments/" target="_blank" rel="noreferrer noopener">report (HTML)</a>
</li>
<li>SensorThings API Hackathon by Geonovum – <a href="https://github.com/Geonovum/SensorthingsAPI/blob/master/wrapup/hackathon_results_justvandenbroecke.pdf" target="_blank" rel="noreferrer noopener">my contributions (PDF slides)</a>
– <a href="https://github.com/Geonovum/SensorthingsAPI/tree/master/sta-data-upload/postman-mqtt" target="_blank" rel="noreferrer noopener">How to upload data to STA</a>
– <a href="https://sta.map5.nl/gost/v1.0" target="_blank" rel="noreferrer noopener">STA Demo Server</a>
. And <a href="https://github.com/justb4/stademo" target="_blank" rel="noreferrer noopener">its setup in GitHub</a>
.</li>
</ul>
<h3 id="talks--workshops-8211-provided">Talks & Workshops – Provided</h3>
<p>On several of the above events I gave some presence with presentations and workshops. Most of my slides can be found on <a href="https://www.slideshare.net/justb4/" target="_blank" rel="noreferrer noopener">slideshare.net/justb4</a>
Below some links.</p>
<ul>
<li>
<p>June 20 – <a href="https://foss4g.nl/" target="_blank" rel="noreferrer noopener">FOSS4G-NL</a>
. Presented <a href="https://pygeoapi.io/presentations/default/#/frontpage" target="_blank" rel="noreferrer noopener">pygeoapi</a>
.</p>
</li>
<li>
<p>June 25 – GeoPython Basel – presented <a href="https://submit.geopython.net/geopython2019/talk/FYMUDV/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
.</p>
</li>
<li>
<p>August 25-31 – <a href="https://2019.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G – Bukarest</a>
. Provided GeoPython workshop. All together with the great GeoPython team as in picture above.</p>
<p><img src="https://justobjects.nl/uploads/2020/01/geopython-ws-foss4g19.png" alt=" "></p>
</li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-medium wp-image-923" src="uploads/2020/01/geopython-ws-foss4g19-143x300.png" alt="" width="143" height="300" srcset="https://justobjects.nl/wp-content/uploads/2020/01/geopython-ws-foss4g19-143x300.png 143w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-ws-foss4g19-72x150.png 72w, https://justobjects.nl/wp-content/uploads/2020/01/geopython-ws-foss4g19.png 402w" sizes="(max-width: 143px) 100vw, 143px" />
-->
<ul>
<li>August 25-31 – <a href="https://2019.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G – Bukarest</a>
. Presented <a href="https://submit.geopython.net/geopython2019/talk/FYMUDV/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
. With <a href="https://www.linkedin.com/in/tomkralidis/" target="_blank" rel="noreferrer noopener">Tom Kralidis</a>
.</li>
<li>Sept 20 – <a href="https://www.sensemakersams.org/" target="_blank" rel="noreferrer noopener">Sensemakers Amsterdam</a>
– on Open Sensor Networks – <a href="https://www.slideshare.net/justb4/open-sensor-networks-173718779" target="_blank" rel="noreferrer noopener">slides.</a>
</li>
<li>Okt 3 – at Stantec – GeoHealthCheck/GeoQoS workshop – <a href="https://geohealthcheck.org/presentation/index.html#/" target="_blank" rel="noreferrer noopener">slides</a>
.</li>
<li>Nov 1 – Odijk – Dutch Rentmeesters (steward: a person who manages an estate or farm for another person) – presented <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
.</li>
</ul>
<h3 id="logo-design">Logo Design</h3>
<p>I don’t consider myself as a designer, but with a little help from online logo-creation platforms and feedback from my co-workers, I created the following in 2019:</p>
<ul>
<li>For <a href="https://geoqos.com/" target="_blank" rel="noreferrer noopener">GeoQoS.com</a>
– hosted GeoHealthCheck in the Cloud.<br>
<img src="https://justobjects.nl/uploads/2020/01/geoqos-logo.png" alt=" "></li>
</ul>
<!--
<img loading="lazy" class="aligncenter wp-image-929" src="uploads/2020/01/geoqos-logo.png" alt="" width="433" height="93" srcset="https://justobjects.nl/wp-content/uploads/2020/01/geoqos-logo.png 765w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos-logo-300x64.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/geoqos-logo-150x32.png 150w" sizes="(max-width: 433px) 100vw, 433px" />
-->
<ul>
<li>For the <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
(OGC API Features, “WFS3”, in Python) Project<br>
<img src="https://justobjects.nl/uploads/2020/01/final-s.png" alt=" "></li>
</ul>
<!--
<img loading="lazy" class="aligncenter size-full wp-image-930" src="uploads/2020/01/final-s.png" alt="" width="300" height="82" srcset="https://justobjects.nl/wp-content/uploads/2020/01/final-s.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/final-s-150x41.png 150w" sizes="(max-width: 300px) 100vw, 300px" />
-->
<ul>
<li>For the <a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
project<br>
<img src="https://justobjects.nl/uploads/2020/01/medium.png" alt=" "></li>
</ul>
<!--
<img loading="lazy" class="aligncenter wp-image-931" src="uploads/2020/01/medium.png" alt="" width="419" height="73" srcset="https://justobjects.nl/wp-content/uploads/2020/01/medium.png 918w, https://justobjects.nl/wp-content/uploads/2020/01/medium-300x52.png 300w, https://justobjects.nl/wp-content/uploads/2020/01/medium-768x134.png 768w, https://justobjects.nl/wp-content/uploads/2020/01/medium-150x26.png 150w, https://justobjects.nl/wp-content/uploads/2020/01/medium-900x160.png 900w" sizes="(max-width: 419px) 100vw, 419px" />
-->
<h3 id="resolutions-2020">Resolutions 2020</h3>
<ul>
<li>More effort into <a href="https://github.com/meggsimum/wegue" target="_blank" rel="noreferrer noopener">Wegue</a>
project</li>
<li>Further expanding hosted Geospatial Cloud Services</li>
<li>Improve on and provide the GeoPython Workshop</li>
<li>Whatever comes around.</li>
</ul>
-
Second Time Around – WFS v3 and pygeoapi
https://justobjects.nl/2nd-time-around-wfs-v3-pygeoapi/
Fri, 31 May 2019 23:33:35 +0000
https://justobjects.nl/2nd-time-around-wfs-v3-pygeoapi/
<p>TLDR;</p>
<ul>
<li>tired of the OGC WFS/GML/INSPIRE complexity mess</li>
<li>new spirit from <a href="https://github.com/opengeospatial/WFS_FES" target="_blank" rel="noreferrer noopener">WFS v3 standard</a>
and OGC REST API movement</li>
<li>title of this blog refers (YouTube): <a href="https://www.youtube.com/watch?v=xz4YQZ01Q_A" target="_blank" rel="noreferrer noopener">Shalamar, The Second Time Around</a>
.</li>
<li><a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
implements OGC REST APIs in Python</li>
<li>I joined <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
</li>
<li><a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
can unlock/proxy existing WFS v1,v2 and any OGR Source!</li>
</ul>
<p>About 10 years ago, working on my first serious geospatial contract at <a href="https://www.kadaster.nl/" target="_blank" rel="noreferrer noopener">Dutch Kadaster</a>
, I was asked to investigate an emerging <a href="http://www.opengeospatial.org/" target="_blank" rel="noreferrer noopener">OGC</a>
standard called “WFS”, <a href="https://en.wikipedia.org/wiki/Web_Feature_Service" target="_blank" rel="noreferrer noopener">Web Feature Service</a>
. Providing a matrix of client-server interoperability was one of the expected outcomes. That is: which WFS-clients will happily fetch data from WFS server-products. I don’t have the actual outcome anymore, but I can recall: it was a very empty table: there were a few WFS server-implementations at the time: GeoServer, MapServer and deegree and some WFS clients, both Open Source and proprietary. In my recollection, only OpenLayers would interwork with the above three WFS servers. The rest had various interworking issues: around <a href="https://epsg.io/" target="_blank" rel="noreferrer noopener">EPSG</a>
-encoding (three possible encodings!) and other vague GML issues. In short, it was a mess: remember, all we wanted to exchange were flat, table-like <a href="https://www.opengeospatial.org/standards/sfa" target="_blank" rel="noreferrer noopener">Simple Features:</a>
records with attributes and a single geometry (Point, Line, Polygon).</p>
<p>Later on, things became even worse: multiple WFS and GML versions. And not to mention <a href="https://en.wikipedia.org/wiki/Geography_Markup_Language" target="_blank" rel="noreferrer noopener">GML Application Schemas</a>
like for <a href="http://inspire.ec.europa.eu/applicationschema" target="_blank" rel="noreferrer noopener">INSPIRE</a>
. Plenty of work for WFS/GML experts, which I was considered one, but I felt cynical at some point. From my background as a software engineer at <a href="https://en.wikipedia.org/wiki/Bell_Labs" target="_blank" rel="noreferrer noopener">AT&amp;T/Lucent/Bell Labs</a>
, where I developed embedded software for the public <a href="https://en.wikipedia.org/wiki/5ESS_Switching_System" target="_blank" rel="noreferrer noopener">5ESS telephone exchange</a>
, <strong><em>over-engineering was considered bad practice</em></strong>.</p>
<p>Also from the perspective of the emerging <a href="https://agilemanifesto.org/" target="_blank" rel="noreferrer noopener">Agile movement</a>
: <strong><em>“make the simplest thing that could possibly work”</em></strong>, this WFS/GML ‘beast’ seemed untamable, driven by theoreticians who modeled the geospatial world from behind UML tools like <a href="https://sparxsystems.com/products/ea/" target="_blank" rel="noreferrer noopener">Enterprise Architect</a>
. I attended numerous meetings between Dutch government organizations that had WFS/GML interworking problems. At some point I gave up: the intellectual challenges (and contracts!) were alluring, but the (geospatial) world wouldn’t progress like this. Though I used WFS and GML in many of my apps, I had a love-hate relationship with them, as a rule-of-thumb only the lowest denominator, WFS v1 and GLM2 had the most chances to interwork.</p>
<p>Until…back to the title of this blog. I grew up in the 70-s/80-s, so want to introduce this song. I hope the YouTube video below is embedded ok. Otherwise check out <a href="https://www.youtube.com/watch?v=xz4YQZ01Q_A" target="_blank" rel="noreferrer noopener">this link from Shalamar, The Second Time Around</a>
.</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/xz4YQZ01Q_A?controls=1&rel=0" loading="lazy"></iframe>
</div>
<p>Fast forward to 2019: we live in an API-world: REST, <a href="https://swagger.io/" target="_blank" rel="noreferrer noopener">Swagger</a>
, OpenApi. OGC became wiser: both in process and technology. Folks from the OSGeo Community, I must mention <a href="https://medium.com/@cholmes" target="_blank" rel="noreferrer noopener">Chris Holmes</a>
here, advocated/pushed for a more collaborative/development-like standardization process, e.g. using GitHub over Word documents. A landmark OGC/W3C study called <a href="https://www.w3.org/TR/sdw-bp/" target="_blank" rel="noreferrer noopener">“Spatial Data on the Web Best Practices”</a>
recommended a more web-friendly approach.</p>
<p>Fast-fast forward, the above issues resulted in a brand new <a href="https://github.com/opengeospatial/WFS_FES" target="_blank" rel="noreferrer noopener">WFS version 3 standard</a>
under development: using both GitHub and the latest REST (Open)API technologies. Forget about WFS v1 and v2, GML v.whatever, and OWS in general with their <strong><em>tired GetCapabilities</em></strong>, this is a new road taken. Kudos to <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://medium.com/@cholmes" target="_blank">Chris Holmes</a>, he words this better than me in this <a href="https://medium.com/@cholmes/wfs-3-0-get-excited-yes-8e904fdbcc0" target="_blank" rel="noreferrer noopener">Medium Article on WFS3</a>
.</p>
<p>So yes, what about this <strong><em>Second Time Around?</em></strong> Well I got attracted to WFS again (“in love” is not really appropriate for technology IMHO), for two reasons:</p>
<ol>
<li>the <a href="https://github.com/opengeospatial/WFS_FES" target="_blank" rel="noreferrer noopener">WFS spec</a>
and the folks/energy around it</li>
<li>Open Source implementation of this standard in Python: <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
</li>
</ol>
<p>Python IMHO makes much much sense within the Open Source geospatial domain. For similar reasons as described above, like over-engineering (and Oracle!), I abandoned Java about 8 years ago.</p>
<p>So I joined the <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
project. Not out of any commercial interest, but mainly out of an developer’s itch: this is useful, let’s see how this works out, a now small, but great community, several from my country, to work with.</p>
<p>About <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
: currently the project is focused on WFS v3, but in general we attempt to implement multiple OGC REST APIs. Also, the project provides a plugin architecture: via <strong><em>Providers</em></strong>, where data can be fetched from any backend: remote services or local files. Check out our <a href="https://demo.pygeoapi.io/" target="_blank" rel="noreferrer noopener">demo server</a>
which is updated constantly via the <a href="https://github.com/geopython/pygeoapi" target="_blank" rel="noreferrer noopener">pygeoapi GitHub repo</a>
and <a href="https://cloud.docker.com/u/geopython/repository/docker/geopython/pygeoapi/" target="_blank" rel="noreferrer noopener">DockerHub</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2019/05/image.png" alt=" "></p>
<!--
<figure class="wp-block-image">
<img loading="lazy" width="1024" height="578" src="https://justobjects.nl/wp-content/uploads/2019/05/image-1024x578.png" alt="" class="wp-image-886" srcset="https://justobjects.nl/wp-content/uploads/2019/05/image-1024x578.png 1024w, https://justobjects.nl/wp-content/uploads/2019/05/image-300x169.png 300w, https://justobjects.nl/wp-content/uploads/2019/05/image-768x433.png 768w, https://justobjects.nl/wp-content/uploads/2019/05/image-150x85.png 150w, https://justobjects.nl/wp-content/uploads/2019/05/image.png 1200w" sizes="(max-width: 1024px) 100vw, 1024px" /> </figure>
-->
<p>Diving a bit deeper, one of my first contributions to this project was to add an <a href="https://github.com/geopython/pygeoapi/blob/master/pygeoapi/provider/ogr.py" target="_blank" rel="noreferrer noopener">OGRProvider</a>
that would fetch from any <a href="https://gdal.org/drivers/vector/index.html" target="_blank" rel="noreferrer noopener">GDAL/OGR Vector source</a>
, in particular a backend WFS. Partly from a vision: <strong><em>“Let’s free all those tired WFS-es out there”</em></strong>, but also practically: the Python GDAL/OGR bindings are so powerful, performant and reusable, saving lots of development time over developing specific Providers for GeoJSON, GeoPackage, SpatialLite, PostGIS, and even ESRI Shapefiles and FeatureServers. In theory, any OGR (Vector) Source can now be exposed via WFS v3 in <a href="https://pygeoapi.io/" target="_blank" rel="noreferrer noopener">pygeoapi</a>
.</p>
<p>There’s lots of movement around WFS v3 and OGC REST APIs in general, where both standards makers and implementors interact. Like the <a href="https://www.opengeospatial.org/OGCAPI_Hack2019" target="_blank" rel="noreferrer noopener">OGC API Hackathon</a>
in June 2019, London (sold out now!). If you ever abandoned WFS v1/v2 it is time to reconcile now!</p>
-
Emit #6 – AirSensEUR Calibration
https://justobjects.nl/emit-6-airsenseur-calibration/
Tue, 19 Feb 2019 16:23:52 +0000
https://justobjects.nl/emit-6-airsenseur-calibration/
<p>This is Emit #6, in a <a href="https://justobjects.nl/categories/smartemission/">series of blog-posts around the Smart Emission Platform</a>
, an Open Source software component framework that facilitates the acquisition, processing and (OGC web-API) unlocking of spatiotemporal sensor-data, mainly for Air Quality and other environmental sensor-data like noise.</p>
<p>In <a href="https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/">Emit #5 – Assembling and Deploying 5 AirSensEURs…</a>
, I described how , with the great help of Jan Vonk from RIVM, we placed five <a href="https://airsenseur.org/" target="_blank" rel="noreferrer noopener">AirSensEUR</a>
(ASE) air quality stations at the RIVM reference site near the A2 Highway at Breukelen. For about 2.5 months raw data was gathered there while the RIVM station was gathering its data to be used as reference for calibration.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/airsenseur-deploy-combined-s.jpg" alt=" "></p>
<!--
<img loading="lazy" class="aligncenter wp-image-864" src="uploads/2019/02/airsenseur-deploy-combined-s-1024x537.jpg" alt="" width="555" height="291" srcset="https://justobjects.nl/wp-content/uploads/2019/02/airsenseur-deploy-combined-s.jpg 1024w, https://justobjects.nl/wp-content/uploads/2019/02/airsenseur-deploy-combined-s-300x157.jpg 300w, https://justobjects.nl/wp-content/uploads/2019/02/airsenseur-deploy-combined-s-768x403.jpg 768w, https://justobjects.nl/wp-content/uploads/2019/02/airsenseur-deploy-combined-s-150x79.jpg 150w" sizes="(max-width: 555px) 100vw, 555px" />
-->
<p>Now “calibration” is a huge and increasingly important topic when using inexpensive sensors for measuring Air Quality. Within the Smart Emission project we have been applying <a href="https://en.wikipedia.org/wiki/Artificial_neural_network" target="_blank" rel="noreferrer noopener">Artificial Neural Networks</a>
to calibrate the gas-sensors within the Josene stations. See also the <a href="https://smartplatform.readthedocs.io/en/latest/calibration.html" target="_blank" rel="noreferrer noopener">SE documentation</a>
. These sensors were so called <a href="https://sgx.cdistore.com/products/sgx-sensortech/metaloxide-gas-sensor" target="_blank" rel="noreferrer noopener">metaloxide (MICS) sensors from SGX Sensortech Limited</a>
.</p>
<p>The AirSensEURs contain <a href="http://www.alphasense.com/index.php/safety/products/" target="_blank" rel="noreferrer noopener">electrochemical sensors from AlphaSense</a>
. Several sources like <a href="https://www.samenmetenaanluchtkwaliteit.nl/sensoren-voor-no2" target="_blank" rel="noreferrer noopener">RIVM</a>
, state that these sensors are more accurate (than metaloxide sensors), but at the same time need per-sensor calibration.</p>
<p>Within the ASE boxes the following four gas-sensors were applied:</p>
<ul>
<li>NO2 (Nitrogen Dioxide): the <a href="http://www.alphasense.com/index.php/products/nitrogen-dioxide/" target="_blank" rel="noreferrer noopener">NO2-B43F</a>
, see <a href="http://www.alphasense.com/WEB1213/wp-content/uploads/2018/12/NO2B43F.pdf" target="_blank" rel="noreferrer noopener">datasheet</a>
.</li>
<li>NO (Nitric Oxide): the <a href="http://www.alphasense.com/index.php/products/nitric-oxide-safety/" target="_blank" rel="noreferrer noopener">NO-B4</a>
, see <a href="http://www.alphasense.com/WEB1213/wp-content/uploads/2015/03/NO-B4.pdf" target="_blank" rel="noreferrer noopener">datasheet</a>
.</li>
<li>O3 (Ozone): the <a href="http://www.alphasense.com/index.php/products/ozone-2/" target="_blank" rel="noreferrer noopener">OX_A431</a>
, see <a href="http://www.alphasense.com/WEB1213/wp-content/uploads/2018/12/OXA431.pdf" target="_blank" rel="noreferrer noopener">datasheet</a>
.</li>
<li>CO (Carbon Monoxide): the <a href="http://www.alphasense.com/index.php/products/carbon-monoxide-safety/" target="_blank" rel="noreferrer noopener">CO-A4</a>
, see <a href="http://www.alphasense.com/WEB1213/wp-content/uploads/2017/01/COA4.pdf" target="_blank" rel="noreferrer noopener">datasheet</a>
.</li>
</ul>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-12.jpg" alt="The Gang of Four Sensors"></p>
<!--
<div id="attachment_790" style="width: 358px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-790" loading="lazy" class=" wp-image-790" src="uploads/2018/08/asenl-unbox-assemble-deploy-12-300x243.jpg" alt="" width="348" height="282" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-300x243.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-768x623.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-150x122.jpg 150w" sizes="(max-width: 348px) 100vw, 348px" />
<p id="caption-attachment-790" class="wp-caption-text">
The Gang of Four Sensors
</p>
</div>
-->
<p>The calibration to be applied is based on <a href="https://en.wikipedia.org/wiki/Regression_analysis" target="_blank" rel="noreferrer noopener">Regression Analysis</a>
. This and other calibration-methods have been investigated and evaluated for many types/brands of sensors by the EC-JRC team. Read all in this <a href="https://www.sciencedirect.com/science/article/pii/S092540051500355X" target="_blank" rel="noreferrer noopener">landmark article</a>
and other references there.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/field-calibration-article.png" alt=" "></p>
<!--
<img loading="lazy" class="wp-image-862 aligncenter" src="uploads/2019/02/field-calibration-article.png" alt="" width="404" height="251" srcset="https://justobjects.nl/wp-content/uploads/2019/02/field-calibration-article.png 665w, https://justobjects.nl/wp-content/uploads/2019/02/field-calibration-article-300x186.png 300w, https://justobjects.nl/wp-content/uploads/2019/02/field-calibration-article-150x93.png 150w" sizes="(max-width: 404px) 100vw, 404px" />
-->
<p>The complete timeline was as follows. Each phase will be expanded below.</p>
<ol>
<li>
<p>Aug 1, 2018 – Okt 9, 2018<br>
Raw ASE and RIVM reference data collection (Breukelen site)</p>
</li>
<li>
<p>Okt 10, 2018 – Nov 2, 2018<br>
All ASEs deployed at their target locations</p>
</li>
<li>
<p>Nov/dec 2018<br>
Calibration performed by Michel Gerboles at the EC-JRC lab</p>
</li>
<li>
<p>Jan 2019<br>
Calibration formulas implemented in Smart Emission (SE) platform</p>
</li>
<li>
<p>Feb 2019<br>
All ASE calibrated gas-data continuously available via SE viewers/APIs</p>
</li>
<li>
<p>Feb 2019<br>
Analysis of the calibration (<a href="https://github.com/smartemission/docker-se-stetl/" target="_blank" rel="noreferrer noopener">SE Python Stetl</a>
) implementation results</p>
</li>
</ol>
<p><strong>ad 1)</strong> The five ASE Boxes were mounted on a horizontal pole and connected to WIFI and current. As end-result all boxes were publishing their raw data to the SE InfluxDB Data Collector and were visible in the <a href="https://data.smartemission.nl/grafana-dc/d/HVSBmbHmz/airsenseur-netherlands-deploy" target="_blank" rel="noreferrer noopener">SE Grafana raw data viewer</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-1.jpg" alt="Configured for InfluxDB Data Push visualized via Grafana"></p>
<p><strong>ad 2)</strong> The picture below shows ASE_NL_01 (left above) through _05 clockwise at their deployment sites. ASE_03 and 04 (right below) were at a single location.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/ase-nl-1-5-deployed.jpg" alt=" "></p>
<p>ASE_NL 01 was deployed at an RIVM site in Nijmegen. This allowed us to verify its calibration with different reference data as with which it was calibrated! See below.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/deployment-map-s.jpg" alt=" "></p>
<p><strong>ad 3)</strong> The calibration was performed by EC-JRC (M. Gerboles) using R and ShinyR webapp. All sources can be found in this <a href="https://github.com/ec-jrc/airsenseur-calibration" target="_blank" rel="noreferrer noopener">EC-JRC GitHub repo</a>
. This process is quite intricate and a bit hard to explain in the context of a blog-post paragraph. I’ll try a summary:</p>
<p>Sensor values are digital readings (0..65535). This is effected by the electrical circuitry within each ASE, for optimal gain. To calculate back to mV and nA a per-sensor (brand+type) calculation is required first before applying any regression formula. A bit is explained in the image below.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/readings-vref.jpg" alt=" "></p>
<p>The second outcome is a per-individual-sensor regression formula. This is for most sensors a linear equation. For O3 (OX_A431) the formula is polynomial, as O3 readings are influenced by NO2 concentration. Below is an example as later implemented in Python using <a href="https://github.com/smartemission/docker-se-stetl/" target="_blank" rel="noreferrer noopener">SE Stetl ETL</a>
</p>
<p><img src="https://justobjects.nl/uploads/2019/02/calibration-formulae.jpg" alt=" "></p>
<p>The main three outcomes of the calibration are:</p>
<ul>
<li>the parameters for digital to nA calculation (per sensor brand+type)</li>
<li>the linear (polynomial) equations for nA to concentration (ug/m3)</li>
<li>the per-individual-sensor parameters (a0-a3)</li>
</ul>
<p><img src="https://justobjects.nl/uploads/2019/02/scatterplots-asenl03.jpg" alt=" "></p>
<p>Above some scatterplots made for ASE Box 3 NO2 and O3.</p>
<p><strong>ad 4)</strong> Knowing all equations and their parameters from step 3 above, I attempted to integrate this in the continuous ETL within the Smart Emission Platform. Up to now the platform supported only a single sensor station type: the Intemo Jose(ne). As the platform is fed by harvesting raw data from a set of remote APIs provided by Data Collectors, it was relatively easy to add sensor(-station)-metadata and extend the Refiner ETL to apply calibration algorithms driven by that metadata.</p>
<p>So for Josene stations the existing ANN calibration would still be applied, while for ASE stations per-sensor linear equations would be performed. All parameterization was already configurable using the <a href="https://github.com/smartemission/docker-se-stetl/tree/mast" target="_blank" rel="noreferrer noopener">Device, DeviceDefs, DeviceFuncs</a>
abstractions in the <a href="https://github.com/smartemission/docker-se-stetl/" target="_blank" rel="noreferrer noopener">SE Stetl implementation</a>
. Recently, to allow stations that already send calibrated values, I introduced the <a href="https://github.com/smartemission/docker-se-stetl/blob/master/smartem/devices/vanilla.py" target="_blank" rel="noreferrer noopener">Vanilla Device</a>
starting with harvesting <a href="https://luftdaten.info" target="_blank" rel="noreferrer noopener">Luftdaten.info</a>
stations (more in a later post).</p>
<p>The formula’s as applied in Python SE Stetl are as follows:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>STEP 1a - Digital to Voltage (V)
<span class="ln">2</span>V = (Ref - RefAD) + (Digital+1) /2^16 x 2 x RefAD
</code></pre></div><div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>STEP 1b - Voltage (V) to Ampere (I) as Ri
<span class="ln">2</span>I = 10^9 V/(Gain x Rload)
</code></pre></div><div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>STEP 2 - Ampere (I) to concentration (ug/m3) - Example
<span class="ln">2</span>I=a0+a1*NO2+a2*T
</code></pre></div><div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>==> NO2 = (I - a0 - a2 * T) / a1
<span class="ln">2</span>a0-a2 has specific values for each NO2-B43F sensor.
</code></pre></div><p>Now that these formulas and their parameters were implemented, near-realtime values could be made visible in all SE apps (viewers) and APIs such as the <a href="https://data.smartemission.nl/smartapp/" target="_blank" rel="noreferrer noopener">SmartApp</a>
and the <a href="https://data.smartemission.nl/heron/" target="_blank" rel="noreferrer noopener">Heron Viewer</a>
.</p>
<div class="">
<a href="https://data.smartemission.nl/smartapp/">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/02/smartapp-ase01-nijmegen.png"
alt="" />
</a>
</div>
<p>Within the Heron Viewer we can compare for example NO2, not only with Josene measurements, but also with official RIVM values.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/heron1.png" alt=" "></p>
<p>Also the data is available through all <a href="https://data.smartemission.nl/data" target="_blank" rel="noreferrer noopener">SE OGC APIs</a>
, for example the <a href="https://en.wikipedia.org/wiki/SensorThings_API" target="_blank" rel="noreferrer noopener">SensorThings API</a>
.</p>
<p><strong>ad 6)</strong> The moment of truth! How well does the SE-based SE Stetl Python calibration results fit with the original RIVM values? One of the advantages of Data Harvesting (opposed to data push) is that we can switch back in time, i.e. restart harvesting from a given date. Harvesting and continuous calibration was restarted from august 1, 2018, the start of the calibration period at the RIVM station. Using a Grafana panel that displays both RIVM and SE-calculated values we can graphically see how well the data aligns.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/breukelen-grafana.jpg" alt=" "></p>
<p>What we can see from the above image, is that visually the data aligns very well, here for NO2. The purple graph is the official RIVM measurement. Only station ASE NL 02 is not completely aligned.</p>
<p>To also have some numeric proof and a more objective comparison, I dived in scatterplot and numerical analysis in Python. Apart from scatterplots that show calculated (Y) agains RIVM ref values (X) I calculated the <a href="https://en.wikipedia.org/wiki/Coefficient_of_determination" target="_blank" rel="noreferrer noopener">“R-squared”</a>
and “slope” for fitting indicator values. This was also my first serious use of Python libs like Scipy, Pandas, Seaborn and Numpy (you’re never too old to become a data-scientist!).</p>
<p>As all SE calibrated data is also stored in InfluxDB with RIVM refdata harvested from their SOS, it was easy to fetch values for the plots/calculations.</p>
<p>Objectivity could be effected since station ASE NL 01 was finally deployed (okt 2018) in Nijmegen, also next to an RIVM station. So the calibration calculations from RIVM refdata in Breukelen could be compared to “Nijmegen”. The implementation for making these scatterplots can be found <a href="https://github.com/smartemission/smartemission/tree/master/etl/calibration" target="_blank" rel="noreferrer noopener">here</a>
. Lets look at some results, mainly for NO2, as I consider this one of the most important AQ indicator gasses.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/nijm-ruy-no2-ASE_NL01-2018-12-25-2019-01-24.png" alt=" "></p>
<p>I like this image a lot as it shows an almost ideal alignment with an R2 of 0.976 and slope of almost 1. Mind: calibration was thus done at a very different site (about 80 km west) and AQ condition (highway) as the deployment (city street).</p>
<p><img src="https://justobjects.nl/uploads/2019/02/asenl01-se-pycal-plots.png" alt=" "></p>
<p>Above are plots for the other gasses as well. First row in Breukelen (no ref CO available in RIVM SOS), front row in Nijmegen. Only NO in Nijmegen is a bit problematic.</p>
<p><img src="https://justobjects.nl/uploads/2019/02/breuk-sw-no2-ASE_NL_All-2018-09-10-2018-10-09.png" alt=" "></p>
<p>To close off: this last image above shows NO2 fit at the Breukelen station for all five ASE boxes. Also quite good.</p>
<p>What to conclude? First of all AirSensEUR is a major step forward in affordable accurate AQ sensing. We hope to expand the community.</p>
<p>AlphaSense NO2 electrochemical sensors appear quite accurate, but calibration requires quite some effort, plus calibration formulas apply per individual sensor. Would automatic per-sensor ANN be less time-consuming and still accurate? Something I would like to investigate.</p>
<p>The Smart Emission project and platform is still going strong, running within a Kubernetes Cloud maintained by Dutch Kadaster.</p>
<p>Next emit will discuss how I integrated data from the amazing Luftdaten.info project for the municipality of Nijmegen.</p>
-
Cheers to 2018
https://justobjects.nl/cheers-to-2018/
Fri, 11 Jan 2019 02:04:54 +0000
https://justobjects.nl/cheers-to-2018/
<p>Never too late to look back. Inspired by the concise bullet-point-style of Tom Kralidis’ <a href="https://www.kralidis.ca/blog/2018/12/28/cheers-to-2018/" target="_blank" rel="noreferrer noopener">Cheers to 2018</a>
, my personal highlights of living and working in the Open Source Geospatial and OSGeo(.nl|org)-world in 2018. I’ll post 2019 reso’s later. Not just personal projects, but also events from the <a href="https://osgeo.nl" target="_blank" rel="noreferrer noopener">OSGeo.nl Local Chapter</a>
in which I am involved since its beginnings.</p>
<h3 id="january">January</h3>
<ul>
<li>One of my 2018-resolutions was to focus more on SensorWeb/Spatiotemporal inspired by working since 2015 on the <a href="https://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission project</a>
. Started a <a href="https://justobjects.nl/category/smartemission/" target="_blank" rel="noreferrer noopener">blog-post-series “Emits”</a>
.</li>
<li>With <a href="https://osgeo.nl" target="_blank" rel="noreferrer noopener">OSGeo.nl</a>
we always start the year (since 2011) with a joint <a href="https://osgeo.nl/2018/02/verslag-nieuwjaarsborrel-2018/" target="_blank" rel="noreferrer noopener">OSGeo.nl and OpenStreetMap NL new-years party</a>
at Cafe Dudok in Hilversum. Many of the 2018-plans discussed there became reality. Not at least the two QGIS-Days and FOSS4G-NL 2018 conference (see below).</li>
<li>The first Dutch QGIS Day hosted by the <a href="https://www.un-ihe.org/" target="_blank" rel="noreferrer noopener">IHE Delft Institute for Water Education</a>
. Very well attended (160?) by both regular QGIS-users and IHE-students. Big thanks to <a href="https://geo-academie.nl/erik/" target="_blank" rel="noreferrer noopener">Erik Meerburg</a>
(what would OSGeo.nl be without Erik?) and <a href="https://www.un-ihe.org/hans-van-der-kwast" target="_blank" rel="noreferrer noopener">Hans van der Kwast</a>
(IHE).</li>
<li>Started working on a contract for the <a href="https://ec.europa.eu/jrc/en" target="_blank" rel="noreferrer noopener">European Commission Joint Research Center</a>
(Ispra, Italy) to deploy five <a href="https://airsenseur.org/" target="_blank" rel="noreferrer noopener">AirSensEUR</a>
air-quality stations within the Netherlands, connecting these to the <a href="https://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
. More in <a href="https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/" target="_blank" rel="noreferrer noopener">this blog-post</a>
.</li>
</ul>
<h3 id="february">February</h3>
<ul>
<li>As first-timer went to <a href="https://www.thethingsnetwork.org/conference/" target="_blank" rel="noreferrer noopener">TheThingsNetwork Conference</a>
. What an amazing event! For some years already into <a href="https://en.wikipedia.org/wiki/LoRa" target="_blank" rel="noreferrer noopener">Lora(WAN)</a>
and maintaining a TTN Gateway, I learned so much, also from the workshops there, that I wrote a <a href="https://justobjects.nl/emit-3/" target="_blank" rel="noreferrer noopener">separate blog-post about this event</a>
.</li>
<li>Started contracting (eventually until sept 30, 2018) at Dutch Kadaster to migrate the entire Smart Emission (SE) Platform to Kadaster infrastructure. Initially just migrating the <a href="https://cloud.docker.com/u/smartemission/" target="_blank" rel="noreferrer noopener">SE Docker Images</a>
, normally two-weeks (to be honest, a day’s) work, BUT, as a (nice) surprise learned we had to move SE “To The Cloud”, that is to <a href="https://azure.microsoft.com/en-us/services/kubernetes-service/" target="_blank" rel="noreferrer noopener">Azure Kubernetes (AKS)</a>
, well that kept me and one Kadaster-colleague (learned a lot from you Gerwin!) busy till end of september at least 2-3 days a week!</li>
</ul>
<h3 id="march">March</h3>
<ul>
<li>As first-timer went to the <a href="https://wiki.osgeo.org/wiki/OSGeo_Code_Sprint_2018" target="_blank" rel="noreferrer noopener">OSGeo 2018 Code Sprint in Bonn</a>
. Mainly to work on <a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
. (GHC). Another highlight! Not the least since I finally met the great <a href="https://www.osgeo.org/member/tom-kralidis/" target="_blank" rel="noreferrer noopener">Tom Kralidis</a>
! I had been working on GHC with Tom a couple of years but we somehow never met IRL. Spent most of week on a single PR (<a href="https://github.com/geopython/GeoHealthCheck/pull/222" target="_blank" rel="noreferrer noopener">per-Resource scheduling</a>
) which required a re-architecture of GHC. But more importantly socializing with Tom and other folks there, well, Bonn, need I say more…Thanks organizers: <a href="https://wiki.osgeo.org/wiki/Astrid_Emde" target="_blank" rel="noreferrer noopener">Astrid Emde</a>
et al. for this great week!</li>
<li>In that same codesprint week I also attended several talks at the yearly <a href="https://fossgis-konferenz.de/2018/" target="_blank" rel="noreferrer noopener">FOSSGIS Conference</a>
there. The FOSSGIS Conference is a major event in Europe as it involves all German-speaking (DACH) countries, plus their OpenStreetMap communities. Even with my high-school German language knowledge very well to follow. As an exception to the rule I was grateful to be allowed to give a <a href="https://geohealthcheck.org/presentation/#/" target="_blank" rel="noreferrer noopener">GeoHealthCheck presentation</a>
in English. Thanks to <a href="https://www.ccc.de" target="_blank" rel="noreferrer noopener">CCC</a>
, FOSSGIS always videotapes all talks, so you can <a href="https://media.ccc.de/v/2018-5294-geohealthcheck" target="_blank" rel="noreferrer noopener">watch some of my stuttering here</a>
.</li>
</ul>
<h3 id="april">April</h3>
<ul>
<li>With OSGeo.nl we and our community-members more and more try to organize small, hands-on events that are lightweight to organize. The <a href="https://osgeo.nl/2018/04/verslag-nlextract-hacking-day/" target="_blank" rel="noreferrer noopener">NLExtract Hacking Day</a>
was a success! Thanks to organizer Frank Steggink (who also wrote the report), and a great space offered by <a href="https://www.webmapper.net/" target="_blank" rel="noreferrer noopener">WebMapper</a>
(Edward MacGillavry) in the <a href="https://sifutrecht.nl/" target="_blank" rel="noreferrer noopener">Social Impact Factory Utrecht.</a>
</li>
</ul>
<h3 id="may">May</h3>
<ul>
<li>What happened in this month? Luckily GitHub tracks my activity. Many commits for the Smart Emission Kubernetes challenge. I see a week without commits in May: made a great hiking trip with friends in Ireland: the <a href="http://www.kerryway.com/" target="_blank" rel="noreferrer noopener">Kerry Way</a>
. Rain? Only one evening when walking to the pub (always live music there)!</li>
</ul>
<div class="">
<a href="https://github.com/justb4?tab=overview&from=2018-01-01&to=2018-12-31">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/github-2018-just.png"
alt="" />
</a>
</div>
<h3 id="june">June</h3>
<ul>
<li>More work on <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
, a subscription service for Dutch topographic, historical- and embellished hillshade and arial maps I started to host in 2015. Plus ongoing Smart Emission stuff.</li>
<li>Gradually, a heat-wave was overtaking The Netherlands. Climate change?</li>
</ul>
<h3 id="july">July</h3>
<ul>
<li>Heat-wave more and more building up.. But that did not stop the great OSGeo.nl organizing team to host the second <a href="https://foss4g.nl/" target="_blank" rel="noreferrer noopener">FOSS4G-NL in Almere</a>
. July 10-11. Kudos to Erik Meerburg and team for two great days of workshops and conf!</li>
<li>At the FOSS4G-NL I had the honor to <a href="https://www.slideshare.net/justb4/osgeonlopeningfoss4gnl2018" target="_blank" rel="noreferrer noopener">open the conference</a>
and present the <a href="https://geoforum.nl/" target="_blank" rel="noreferrer noopener">national Dutch GeoForum: geoforum.nl.</a>
We jointly (Kadaster, WebMapper, OSGeo.nl) extended the existing Dutch Kadaster PDOK Forum. As OSGeo.nl we did not want to launch yet another forum but to bundle forces. Since that launch hundreds of folks registered.</li>
<li>Week after: off to <a href="https://foss4g-europe.osgeopt.pt/" target="_blank" rel="noreferrer noopener">FOSS4G-EU</a>
in <a href="https://en.wikipedia.org/wiki/Guimar%C3%A3es" target="_blank" rel="noreferrer noopener">Guimarães</a>
,Portugal. FOSS4G-EU is always something special. Folks gather from literally every country in Europe. From the Baltic States to Croatia. In a cosy atmosphere <a href="https://wiki.osgeo.org/wiki/Jorge_Gustavo_Rocha" target="_blank" rel="noreferrer noopener">Jorge Gustavo Rocha</a>
and the LOC made this into a great event. Great talks during the day and every evening (and sometimes nights) in this small-town university city it was easy to find and gather with the OSGeo-folks on the main square <a href="https://en.wikipedia.org/wiki/Historic_Centre_of_Guimar%C3%A3es" target="_blank" rel="noreferrer noopener">Largo da Oliveira</a>
.</li>
</ul>
<h3 id="august">August</h3>
<ul>
<li>Started to seriously deploy the five AirSensEUR (ASE) stations. First at an RIVM air quality station in Breukelen next to the main A2 highway. With the great help of Jan Vonk of RIVM we managed to attach and digitally connect these stations at their premises. To have an idea, read <a href="https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/" target="_blank" rel="noreferrer noopener">my blog post on this effort</a>
. The ASE stations will gather raw, uncalibrated, Air Quality data there for two months and use the onsite RIVM measurements as reference for later calibration.</li>
</ul>
<div class="">
<a href="https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/ase-rivm-breukelen.jpg"
alt="" />
</a>
</div>
<h3 id="september">September</h3>
<ul>
<li>Again a small gap in my GitHub commit history: two weeks with my family (and our great Labrador Jamie) in rural France: Auvergne and Ardeche. I always love to set out GPS-hikes at home, hacking some of the French IGN topo-maps for personal use, and see how these turn out on the spot.</li>
</ul>
<p><img src="https://justobjects.nl/uploads/2019/01/ign-trails.jpg" alt=" "></p>
<h3 id="oktober">Oktober</h3>
<ul>
<li>
<p>Contract with Dutch Kadaster ended on sept 30. Just in time we migrated the entire Smart Emission stack to Azure Kubernetes (AKS). Well, IMHO Kubernetes is just awesome, especially after a few years of ad-hoc Docker deployment. Yes, a learning curve and devops-paradigm shift. Though for smaller projects in 2019 I am now combining <a href="https://www.ansible.com/" target="_blank" rel="noreferrer noopener">Ansible</a>
with Docker (Compose).</p>
</li>
<li>
<p>After two months of gathering data for calibration, deployed the five AirSensEUR (ASE) stations at their target locations within The Netherlands. At each location challenges: an ASE needs 3 things: 1) to be firmly/rain-protected mounted physically, 2) 220V current and 3) WIFI. That combination turned out to be a challenge at each spot. But finally all five ASEs are now humming at their spots.</p>
<div class="">
<a href="https://data.smartemission.nl/grafana-dc/d/HVSBmbHmz/airsenseur-netherlands-deploy">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/ases-o3.jpg"
alt="" />
</a>
</div>
</li>
<li>
<p>And with OSGeo.nl we had the second 2018 QGIS-Day at <a href="https://www.ordina.nl/" target="_blank" rel="noreferrer noopener">Ordina Utrecht</a>
. Thanks to the LOC: Jonas, Coen et al. Observation: all hands-on workshops were overcrowded!</p>
</li>
</ul>
<h3 id="november">November</h3>
<ul>
<li>
<p>More work on <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">map5.nl</a>
and working on a next SAAS: <a href="https://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
in the Cloud, as a subscription service, under the name <a href="https://geoqos.com" target="_blank" rel="noreferrer noopener">geoqos</a>
(“Geo-Cause”) on <a href="https://geoqos.com" target="_blank" rel="noreferrer noopener">geoqos.com</a>
. Hopefully that makes the GeoHealthCheck project sustainable.</p>
</li>
<li>
<p>Presented Smart Emission at the <a href="https://www.geonovum.nl/over-geonovum/agenda/geo-gebruikersfestival-2018-en-sdinext" target="_blank" rel="noreferrer noopener">Geo Users Festival by Geonovum</a>
. Bit of beamer issues, <a href="https://www.slideshare.net/justb4/sensor-sdi-in-pdok-with-smart-emission-platform" target="_blank" rel="noreferrer noopener">my slides on Slideshare</a>
.</p>
<div class="">
<a href="https://www.geonovum.nl/over-geonovum/agenda/geo-gebruikersfestival-2018-en-sdinext">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/geofestival-2018.jpg"
alt="" />
</a>
</div>
</li>
<li>
<p>What’s that day after the <a href="https://www.gisday.com/" target="_blank" rel="noreferrer noopener">GIS Day</a>
? Yes, the <a href="https://postgisdag.nl/" target="_blank" rel="noreferrer noopener">PostGIS Day</a>
! That’s exactly what the LOC team (Steven M. Ottens and Tom van Tilburg) organized. Deliberately keeping it low-profile to attract high-profile PostGIS hackers. And how they did! The format was an <a href="https://en.wikipedia.org/wiki/Unconference" target="_blank" rel="noreferrer noopener">unconference</a>
. Even as a considered PostGIS veteran (starting with a PostGIS workshop from <a href="http://2006.foss4g.org/contributionDisplay0b28.html?contribId=47&sessionId=61&confId=1" target="_blank" rel="noreferrer noopener">Paul Ramsey in Lausanne 2006</a>
) my brain was crushing hearing stuff like from <a href="https://www.gdmc.nl/martijn/" target="_blank" rel="noreferrer noopener">Martijn Meijers</a>
- <em><a href="https://www.google.com/search?q=Indexing+AIS+data+%2F+point+clouds+with+a+Space+Filling+Curve&ie=utf-8&oe=utf-8&aq=t&channel=rcs" target="_blank" rel="noreferrer noopener">Indexing AIS data / point clouds with a Space Filling Curve</a>
</em>.</p>
<div class="">
<a href="https://postgisdag.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/postgisday-2018.jpg"
alt="" />
</a>
</div>
</li>
</ul>
<h3 id="december">December</h3>
<ul>
<li>
<p>Through my various projects thinking: All GeoData is Spatio-Temporal. Presented in Eindhoven at a Ruimteschepper event: <em>LoRaWAN TheThingsNetwork makers event</em>. <a href="https://www.slideshare.net/justb4/open-sensor-networks-with-lora-ttn-and-sensorthings-api" target="_blank" rel="noreferrer noopener">Slides are here</a>
. For one thing attempted to bridge the world of IoT with OGC SensorWeb via the <a href="https://en.wikipedia.org/wiki/SensorThings_API" target="_blank" rel="noreferrer noopener">SensorThings API</a>
in a PoC.</p>
<div class="">
<a href="https://www.slideshare.net/justb4/open-sensor-networks-with-lora-ttn-and-sensorthings-api">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/poc-ttn-sta.png"
alt="" />
</a>
</div>
</li>
<li>
<p>That’s almost it of 2018. A last brainwave on GeoHealthCheck usability made me to introduce interactive <a href="https://plot.ly/javascript/" target="_blank" rel="noreferrer noopener">Plotly.js</a>
based interactive history Run-graphs. Together with and thanks to Tom Kralidis reviewing that was my final GitHub commit in 2018. Ok finalized on jan 3, 2019 but what a year!</p>
<div class="">
<a href="https://geohealthcheck.org/">
<img class="lazy"
src="https://justobjects.nl/uploads/2019/01/plotly-ghc.png"
alt="" />
</a>
</div>
</li>
</ul>
-
Emit #5 – Assembling and Deploying 5 AirSensEURs – a Story in Pictures
https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/
Wed, 15 Aug 2018 21:01:46 +0000
https://justobjects.nl/emit-5-assembling-and-deploying-5-airsenseurs/
<p>This is Emit #5, in a <a href="https://justobjects.nl/category/smartemission/">series of blog-posts around the Smart Emission Platform</a>
, an Open Source software component framework that facilitates the acquisition, processing and (OGC web-API) unlocking of spatiotemporal sensor-data, mainly for Air Quality and other environmental sensor-data like noise.</p>
<p>Summer holidays and a heat-wave strikes The Netherlands. Time for some lighter material mainly told in pictures. As highlighted in <a href="https://justobjects.nl/emit-2/" target="_blank" rel="noreferrer noopener">Emit #2</a>
, I have the honor of doing a project for the <a href="https://ec.europa.eu/info/departments/joint-research-centre_en" target="_blank" rel="noreferrer noopener">European Union Joint Research Centre</a>
(EU JRC), to deploy five <a href="https://airsenseur.org/" target="_blank" rel="noreferrer noopener">AirSensEUR</a>
(ASE) boxes within The Netherlands, attaching these to the <a href="http://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
in cooperation with <a href="https://www.rivm.nl/" target="_blank" rel="noreferrer noopener">RIVM</a>
(National Institute for Public Health and the Environment). The ASE boxes measure four Air Quality (AQ) indicators: NO2 (Nitrogen Dioxide), NO (Nitrogen Monoxide), O3 (Ozone) and CO (Carbon Monoxide) plus meteo (Temp, Humidity, Air Pressure) and GPS. Read more on ASE <a href="http://publications.jrc.ec.europa.eu/repository/bitstream/JRC109337/jrc109337_airsenseur_part_c_jrc_technical_report_inspire.pdf" target="_blank" rel="noreferrer noopener">in this article</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/ase-arch.jpg" alt="ASE Architecture"></p>
<!--
<div id="attachment_801" style="width: 996px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-801" loading="lazy" class="size-full wp-image-801" src="uploads/2018/08/ase-arch.jpg" alt="" width="986" height="771" srcset="https://justobjects.nl/wp-content/uploads/2018/08/ase-arch.jpg 986w, https://justobjects.nl/wp-content/uploads/2018/08/ase-arch-300x235.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/ase-arch-768x601.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/ase-arch-150x117.jpg 150w" sizes="(max-width: 986px) 100vw, 986px" />
<p id="caption-attachment-801" class="wp-caption-text">
ASE Architecture
</p>
</div>
-->
<p>The ASE is an Open Hard/Software platform that can be configured with multiple brands/types of sensors. In the current case all four above mentioned AQ sensors are from <a href="http://www.alphasense.com/" target="_blank" rel="noreferrer noopener">AlphaSense</a>
. As these are relatively cheap sensors (< $100,-), the challenge is to have these calibrated before final deployment. This calibration is done by placing the ASE boxes first at an RIVM station, gather data for a month or two and then calibrate these sensors from official RIVM reference measurements at the same location. Using both the raw ASE data and the RIVM reference data the calibration “formulae” can be determined, before placing the ASEs at their final deployment locations around The Netherlands and have the Smart Emission Platform assemble/publish the (calibrated) data for the next year or so. More info on AirSensEUR via <a href="https://www.google.com/search?q=airsenseur" target="_blank" rel="noreferrer noopener">this Google Search</a>
.</p>
<p>Ok, picture time! Explanatory text is below each picture.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-9.jpg" alt="1. ASEs unboxed"></p>
<!--
<div id="attachment_787" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-787" loading="lazy" class="wp-image-787 size-full" src="uploads/2018/08/asenl-unbox-assemble-deploy-9.jpg" alt="" width="1024" height="768" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-9.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-9-300x225.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-9-768x576.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-9-150x113.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-787" class="wp-caption-text">
1. ASEs unboxed
</p>
</div>
-->
<p>Picture 1: Boxes arrived from EU JRC Italy on June 12, 2018. Assembling: upper left shows the (total of 20) AlphaSense sensors like “blisters” (Dutch “doordrukstrips”), the ASE box (with screwdrivers on top) and the protecting metal outer shield on the right.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-11.jpg" alt="2. placing AlphaSense sensors in sockets"></p>
<!--
<div id="attachment_788" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-788" loading="lazy" class="wp-image-788 size-full" src="uploads/2018/08/asenl-unbox-assemble-deploy-11.jpg" alt="" width="1024" height="784" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-11.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-11-300x230.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-11-768x588.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-11-150x115.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-788" class="wp-caption-text">
2. placing AlphaSense sensors in sockets
</p>
</div>
-->
<p>Picture 2: Very carefully placing the AlphaSense sensors in the ASE Sensor Shield (an Arduino-like board) without touching the top-membrane!</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-12.jpg" alt="3. All sensors firmly in their sockets"></p>
<!--
<div id="attachment_790" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-790" loading="lazy" class="size-full wp-image-790" src="uploads/2018/08/asenl-unbox-assemble-deploy-12.jpg" alt="" width="1024" height="830" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-300x243.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-768x623.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-12-150x122.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-790" class="wp-caption-text">
3. All sensors firmly in their sockets
</p>
</div>
-->
<p>Picture 3: all sensors placed, attach current and next to network and other configuring!</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-2.jpg" alt="4. Boxes humming and connected via WIFI to the LAN"></p>
<!--
<div id="attachment_791" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-791" loading="lazy" class="size-full wp-image-791" src="uploads/2018/08/asenl-unbox-assemble-deploy-2.jpg" alt="" width="1024" height="771" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-2.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-2-300x226.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-2-768x578.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-2-150x113.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-791" class="wp-caption-text">
4. Boxes humming and connected via WIFI to the LAN
</p>
</div>
-->
<p>Picture 4: On default startup (via touch buttons) the ASE will expose a default WIFI access point. This can be used to attach and to login at the “ASE Host Board”, a Raspberry Pi-like board running standard Linux Debian. SSH into each box and further configure e.g. the WIFI settings to become a WIFI client, first having all boxes connect to the local office WLAN.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-1.jpg" alt="5. configured for InfluxDB Data Push visualized via Grafana"></p>
<!--
<div id="attachment_792" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-792" loading="lazy" class="size-full wp-image-792" src="uploads/2018/08/asenl-unbox-assemble-deploy-1.jpg" alt="" width="1024" height="715" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-300x209.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-768x536.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-150x105.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-792" class="wp-caption-text">
5. configured for InfluxDB Data Push visualized via Grafana
</p>
</div>
-->
<p>Picture 5. Each box runs a Data Aggregator and can be configured to push data to a remote InfluxDB database. In our case we have setup a Smart Emission InfluxDB Data Collector where the (raw) data is received. This InfluxDB datastore is visualized using a Grafana Panel shown in the picture. We see the five boxes ASE_NL_01-05 sensing and pushing data!</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-4.jpg" alt="6. All packed and in trunk of my car"></p>
<!--
<div id="attachment_795" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-795" loading="lazy" class="wp-image-795 size-full" src="uploads/2018/08/asenl-unbox-assemble-deploy-4.jpg" alt="" width="1024" height="633" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-4.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-4-300x185.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-4-768x475.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-4-150x93.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-795" class="wp-caption-text">
6. All packed and in trunk of my car
</p>
</div>
-->
<p>Picture 6. A good start, but next we need to go out and place the boxes at the RIVM station for a period of calibration. So tearing down, packing, all into the trunk of my car. Up to the RIVM station! July 30, 2018, Still 35 degrees C outside.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-3.jpg" alt="7. The RIVM sensor station, right near the highway"></p>
<!--
<div id="attachment_796" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-796" loading="lazy" class="size-full wp-image-796" src="uploads/2018/08/asenl-unbox-assemble-deploy-3.jpg" alt="" width="1024" height="613" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-3.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-3-300x180.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-3-768x460.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-3-250x150.jpg 250w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-3-150x90.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-796" class="wp-caption-text">
7. The RIVM sensor station, right near the highway
</p>
</div>
-->
<p>Picture 7. July 30, 2018, 13:00. Arrived at the RIVM station. Now to figure out how to attach the five boxes. The lower horizontal iron pole seems the best option. Put all soft/hardware knowledge away, now real plumbing is required!</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-5.jpg" alt="8. Could not have made this without the great help of Jan Vonk (RIVM)"></p>
<!--
<div id="attachment_797" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-797" loading="lazy" class="size-full wp-image-797" src="uploads/2018/08/asenl-unbox-assemble-deploy-5.jpg" alt="" width="1024" height="598" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-5.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-5-300x175.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-5-768x449.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-5-150x88.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-797" class="wp-caption-text">
8. Could not have made this without the great help of Jan Vonk (RIVM)
</p>
</div>
-->
<p>Picture 8. Jan Vonk of RIVM, who also have deployed about 12 ASEs, placing the first boxes on the horizontal pole, so far so good.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-6-1.jpg" alt="9. All five boxes attached!"></p>
<!--
<div id="attachment_798" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-798" loading="lazy" class="size-full wp-image-798" src="uploads/2018/08/asenl-unbox-assemble-deploy-6-1.jpg" alt="" width="1024" height="596" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-6-1.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-6-1-300x175.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-6-1-768x447.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-6-1-150x87.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-798" class="wp-caption-text">
9. All five boxes attached!
</p>
</div>
-->
<p>Picture 9. All five boxes strapped to the pole. Jan Vonk doing the hard work. Next challenge: they need power and WIFI…</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-7.jpg" alt="10. Connecting to power…"></p>
<!--
<div id="attachment_803" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-803" loading="lazy" class="wp-image-803 size-full" src="uploads/2018/08/asenl-unbox-assemble-deploy-7.jpg" alt="" width="1024" height="649" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-7.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-7-300x190.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-7-768x487.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-7-150x95.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-803" class="wp-caption-text">
10. Connecting to power…
</p>
</div>
-->
<p>Picture 10. One cannot have enough power sockets.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-1-1.jpg" alt="11. Power supplies covered under plastic box"></p>
<!--
<div id="attachment_804" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-804" loading="lazy" class="size-full wp-image-804" src="uploads/2018/08/asenl-unbox-assemble-deploy-1-1.jpg" alt="" width="1024" height="714" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-1.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-1-300x209.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-1-768x536.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-1-1-150x105.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-804" class="wp-caption-text">
11. Power supplies covered under plastic box.
</p>
</div>
-->
<p>Picture 11. Covering all power supply stuff under tightened box shielded from rain.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/asenl-unbox-assemble-deploy-8.jpg" alt="12. Moment of truth starting up and attaching to local WIFI"></p>
<!--
<div id="attachment_805" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-805" loading="lazy" class="size-full wp-image-805" src="uploads/2018/08/asenl-unbox-assemble-deploy-8.jpg" alt="" width="1024" height="606" srcset="https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-8.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-8-300x178.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-8-768x455.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/asenl-unbox-assemble-deploy-8-150x89.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-805" class="wp-caption-text">
12. Moment of truth starting up and attaching to local WIFI
</p>
</div>
-->
<p>Picture 12. July 30, 2018, 17:00. Last challenge: booting up the boxes and have them connecting to the local RIVM station’s WIFI. I had pre-configured WLAN settings in each box, but this is always a moment of truth: will they connect? If they do they will start sampling and push their raw data to the Smart Emission Platform…Then we can start the calibration period. And success ... they connected!</p>
<p><img src="https://justobjects.nl/uploads/2018/08/grafana-all-pushing.jpg" alt="13. All boxes connected and sampling and pushing data"></p>
<!--
<div id="attachment_807" style="width: 1034px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-807" loading="lazy" class="size-full wp-image-807" src="uploads/2018/08/grafana-all-pushing.jpg" alt="" width="1024" height="323" srcset="https://justobjects.nl/wp-content/uploads/2018/08/grafana-all-pushing.jpg 1024w, https://justobjects.nl/wp-content/uploads/2018/08/grafana-all-pushing-300x95.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/grafana-all-pushing-768x242.jpg 768w, https://justobjects.nl/wp-content/uploads/2018/08/grafana-all-pushing-150x47.jpg 150w" sizes="(max-width: 1024px) 100vw, 1024px" />
<p id="caption-attachment-807" class="wp-caption-text">
13. All boxes connected and sampling and pushing data.
</p>
</div>
-->
<p>Picture 13. Now on August 15, 2018, with minor hickups, and with great help from the JRC folks Marco Signorini and Michel Gerboles, we have all five boxes sampling and pushing data for the calibration period. The above plot shows raw NO2 data, to be calibrated.</p>
<p><img src="https://justobjects.nl/uploads/2018/08/samenmeten-tweet.jpg" alt="A next step for the RIVM Program “Together Measuring Air Quality”"></p>
<!--
<div id="attachment_810" style="width: 596px" class="wp-caption aligncenter">
<img aria-describedby="caption-attachment-810" loading="lazy" class="size-full wp-image-810" src="uploads/2018/08/samenmeten-tweet.jpg" alt="" width="586" height="427" srcset="https://justobjects.nl/wp-content/uploads/2018/08/samenmeten-tweet.jpg 586w, https://justobjects.nl/wp-content/uploads/2018/08/samenmeten-tweet-300x219.jpg 300w, https://justobjects.nl/wp-content/uploads/2018/08/samenmeten-tweet-150x109.jpg 150w" sizes="(max-width: 586px) 100vw, 586px" />
<p id="caption-attachment-810" class="wp-caption-text">
A next step for the RIVM Program “Together Measuring Air Quality”.
</p>
</div>
-->
<p>So a good start! The heatwave is over, the next hard work is calibration. Why are we doing this? Well, like with meteorology, RIVM and others are stimulating Air Quality to be measured by basically anyone, from groups of civilians to individuals. For this RIVM has setup the program <a href="https://www.samenmetenaanluchtkwaliteit.nl/" target="_blank" rel="noreferrer noopener">“Samen meten aan Luchtkwaliteit”</a>
(“Together measuring air quality”). Measuring Air Quality is not an easy task. We need to learn by doing, make mistakes, and spread knowledge gained. Both AirSensEUR and Smart Emission are therefore Open. Below some further links:</p>
<p>Smart Emission: <a href="https://github.com/smartemission" target="_blank" rel="noreferrer noopener">GitHub</a>
, <a href="http://data.smartemission.nl" target="_blank" rel="noreferrer noopener">WebSite</a>
, <a href="http://data.smartemission.nl/docs" target="_blank" rel="noreferrer noopener">Documentation</a>
, and <a href="https://hub.docker.com/u/smartemission/" target="_blank" rel="noreferrer noopener">Docker Images</a>
.</p>
-
Emit #4 – Push or Pull?
https://justobjects.nl/emit-4-push-or-pull/
Thu, 05 Apr 2018 21:17:16 +0000
https://justobjects.nl/emit-4-push-or-pull/
<p>This is Emit #4, in a <a href="https://justobjects.nl/categories/smartemission/">series of blog-posts around the Smart Emission Platform</a>
, an Open Source software component framework that facilitates the acquisition, processing and (OGC web-API) unlocking of spatiotemporal sensor-data, mainly for Air Quality and other environmental sensor-data like noise.</p>
<p>A lot of things have happened with the project since the <a href="https://justobjects.nl/emit-3/" target="_blank" rel="noreferrer noopener">last Emit #3</a>
(feb 2018 already). The main news is that the SE Platform is now being migrated to the national Dutch GDI services infrastructure, <a href="https://www.pdok.nl/" target="_blank" rel="noreferrer noopener">PDOK</a>
. This infrastructure, entirely built with OSGeo software, is maintained by the <a href="https://www.kadaster.nl/" target="_blank" rel="noreferrer noopener">Dutch Kadaster</a>
and already (since 2010) provides open access to Dutch geospatial datasets and -services including the Dutch INSPIRE access point. Having the SE Platform hosted and maintained within PDOK gives a tremendous opportunity for long-term evolution and stability beyond its initial project-based funding.</p>
<p>There’s lots of exciting new technological details related to this migration, like the move from the current ad-hoc Docker-based deployment to a full-fledged <a href="https://kubernetes.io/" target="_blank" rel="noreferrer noopener">Kubernetes (K8s)</a>
Cloud roll-out. This could be the subject of an entire post by itself. For this we’re setting up a <a href="https://github.com/smartemission" target="_blank" rel="noreferrer noopener">fresh GitHub organization</a>
, plus <a href="https://hub.docker.com/r/smartemission/" target="_blank" rel="noreferrer noopener">SE Docker Images via DockerHub</a>
. All work-in-progress but first results are already tangible.</p>
<p>So what’s this Push or Pull about? As announced in <a href="https://justobjects.nl/emit-1/" target="_blank" rel="noreferrer noopener">Emit #1</a>
, I’ll try to explain the SE architecture “by following the data”. So sensor data is acquired by sensor stations constantly, 24 hours a day, and lots of it. To give somewhat of a figure: within the SE project the <a href="http://www.snuffle.org/doku.php?id=josene" target="_blank" rel="noreferrer noopener">Intemo Sensor Station “Josene”</a>
station provides more than 100 different sensor values, “indicators”. These range from meteo-data like Temperature, Humidity, Pressure, to Air Quality values for NO2 (Nitrogen Dioxide), CO2 (Carbon Dioxide), O3 (Ozone), finedust (PM) and audio/noise indicators (sound pressure) like Decibels (A) min/max/average for about 30 frequency bands (see image below).</p>
<p><img src="https://justobjects.nl/uploads/2018/04/sound-josene.png" alt=" "></p>
<p>Values for these about 100 indicators are sampled about every 20 seconds. Even with about one hundred Josene sensor stations currently one can imagine this will produce “a lot of data” to say the least. O yes, and we also like to keep all data, i.e. maintain history…Josene sensor-stations are connected via Wifi and will have to send their data to “somewhere”. Eventually, a user will need to view/consume sensor-data via an app, like the <a href="http://data.smartemission.nl/smartapp/" target="_blank" rel="noreferrer noopener">SmartApp</a>
(see image below), or web-services without being overwhelmed.</p>
<p><img src="https://justobjects.nl/uploads/2018/04/smartapp.png" alt=" "></p>
<p>So how does the SE Platform deal with this potential tsunami of data? Well, the keywords are Buffering, Harvesting and Aggregation. Back to following the data. Each Josense sensor station will constantly push its data to what is called a <em><strong>Data Collector</strong></em> within the SE architecture. A Data Collector is basically a buffer (server) that will store bulk “timeseries” data for each station. Data is stored in bulk, i.e. not every sample is stored in an individual record but as blobs. The granularity is one blob per station/per day/per hour. This already reduces data records to 24 blobs per day per station. A lot of storage but a reasonable number of records. Also remind: this is raw, unvalidated, unaggregated data. Best is to first show a picture from the global architecture as below.</p>
<p><img src="https://justobjects.nl/uploads/2018/04/se-global-arch.png" alt=" "></p>
<p>Starting at the bottom of the picture above: Sensors send (push) their raw data to their attached Data Collector. Currently this even happens outside the SE Platform: the two Data Collectors from the left are maintained by our partners Intemo (Intemo Data Collector) and CityGIS (CityGIS Data Collector) within their own premises. In the course of the project we have been adding another Data Collector based on InfluxDB for the AirSensEUR. Also the RIVM SOS is viewed as a Data Collector for purposes of Calibration (for another Emit).</p>
<p>The important notice is that the SE Platform deploys <em><strong>Harvesters</strong></em> that constantly <em><strong>Pull</strong></em> the raw data from a series of Data Collectors. So this is where the Push/Pull decoupling is realized. For this (Pull) we have developed a simple <a href="https://github.com/Geonovum/smartemission/tree/master/docs/specs/rawsensor-api" target="_blank" rel="noreferrer noopener">Raw Sensor REST API</a>
nicknamed the <em><strong>Whale API</strong></em>. Each Data Collector exposes its buffered bulk data (history) as JSON Blobs via this very simple API, independent from its internal storage technology. One may ask: why all this unnecessary indirection? Basically, here we have the Push/Pull decoupling realized. This gave us many advantages:</p>
<ul>
<li>Sensors only need “one IP-address to send to”</li>
<li>Data Collectors can be optimized for their purpose</li>
<li>The Whale API provides a thin interface while still allowing transfer of bulk data</li>
<li>The Whale API also provides “last/current” values, so a quick pass-through for near-realtime data</li>
<li>Harvesters can run at their own pace/schedule</li>
<li>The SE Platform can be deployed anywhere, in Test, Production phases and build-up its data</li>
<li>Development and testing can be done locally, on a development laptop even</li>
<li>Data Collectors provide bulk-data from “time zero” so when moving/migrating SE servers or re-architecting ETL or Calibration algorithms we can re-bootstrap from “time-zero”</li>
</ul>
<p>More details can be found within the ever-evolving <a href="http://smartplatform.readthedocs.io/en/latest/api.html" target="_blank" rel="noreferrer noopener">Smart Platform Technical documentation</a>
. As said, we are in the process of migration. Basically moving all software from a single <a href="https://github.com/Geonovum/smartemission" target="_blank" rel="noreferrer noopener">SE GitHub repo at Geonovum</a>
to our new <a href="https://github.com/smartemission" target="_blank" rel="noreferrer noopener">GitHub organization</a>
and <a href="https://hub.docker.com/r/smartemission/" target="_blank" rel="noreferrer noopener">SE DockerHub</a>
, so look into the latter for the current/upcoming status.</p>
<p>So within this Emit I tried to depict how sensor-data “gets into the platform without overwhelming it”. So the Harvesters keep on Harvesting, but what next? Like Neil Young sings in Harvest: “Dream up, dream up, let me fill your cup. With the promise of a man.” That is, I promise a next Emit #5 on After The Goldrush, sorry After The Harvest.</p>
-
Emit #3 – Things are Moving
https://justobjects.nl/emit-3/
Fri, 16 Feb 2018 23:03:57 +0000
https://justobjects.nl/emit-3/
<p>This is Emit #3, in a <a href="https://justobjects.nl/categories/smartemission/">series of blog-posts around the Smart Emission Platform</a>
, an Open Source software component framework that facilitates the acquisition, processing and (OGC web-API) unlocking of spatiotemporal sensor-data, mainly for Air Quality. In <a href="https://justobjects.nl/emit-1/">Emit #1</a>
, the big picture of the platform was sketched. Subsequent Emits will detail technical aspects of the SE Platform. “Follow the data” will be the main trail loosely adhered to.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/arch-big-picture.jpg" alt=" "></p>
<p>Three weeks ago since <a href="https://justobjects.nl/emit-2/">Emit #2</a>
. A lot of Things have been happening since:</p>
<ul>
<li>went to <a href="https://www.thethingsnetwork.org/conference/" target="_blank" rel="noreferrer noopener">The Things Conference</a>
, all about <a href="https://www.thethingsnetwork.org/" target="_blank" rel="noreferrer noopener">building a LoRaWAN network</a>
with an open global community.</li>
<li>started at Dutch Kadaster to migrate the Smart Emission Platform into the <a href="https://www.pdok.nl/en/about-pdok" target="_blank" rel="noreferrer noopener">Dutch National GDI PDOK</a>
.</li>
<li>Elaborated the <a href="http://www.opengeospatial.org/standards/sensorthings" target="_blank" rel="noreferrer noopener">OGC SensorThings API</a>
(STA) to see if it can replace <a href="http://www.opengeospatial.org/standards/sos" target="_blank" rel="noreferrer noopener">OGC Sensor Observation Service</a>
(SOS).</li>
</ul>
<p>A lot to expand on. Will try to briefly summarize on The Things Conference, LoRA and LoRaWAN and save the other tech for later Emits.</p>
<p>LoRA and LoRaWAN may sound like a sidestep, but are very much related to for example building a network of Air Quality and other environmental sensors. When deploying such sensors two issues always arise:</p>
<ul>
<li>power: need continuous electricity to keep sensors and their computing boards powered</li>
<li>transmission: need cabled Internet, WIFI or cellular data to transmit sensor-data</li>
</ul>
<p>In short, LoRa/LoRaWAN (LoRa=Long Range) is basically a wireless RF technology for long-<wbr />range, low-power and low-throughput communications. You may find many references on the web like from the <a href="https://www.lora-alliance.org" target="_blank" rel="noreferrer noopener">LoRa Alliance</a>
and <a href="https://www.semtech.com/technology/lora" target="_blank" rel="noreferrer noopener">SemTech</a>
. There is lots of buzz around LoRa. But just like the, <a href="https://wirelessleiden.nl/" target="_blank" rel="noreferrer noopener">Wireless Leiden project</a>
, who built a public WIFI network around the city, <a href="https://www.thethingsnetwork.org/" target="_blank" rel="noreferrer noopener">The ThingsNetwork</a>
has embraced LoRa technology to build a world-wide open, community-driven, “overlay network”:</p>
<div class="">
<a href="https://www.thethingsnetwork.org/">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/02/ttn-pics.jpg"
alt="" />
</a>
</div>
<p><em>“The Things Network is building a network for the Internet of Things by creating abundant data connectivity, so applications and businesses can flourish. The technology we use is called LoRaWAN and it allows for things to talk to the internet without 3G or WiFi. So no WiFi codes and no mobile subscriptions. It features low battery usage, long range and low bandwidth. Perfect for the internet of things.”</em></p>
<p>You may want to explore the worldwide map of TTN gateways below.</p>
<div class="">
<a href="https://www.thethingsnetwork.org/map">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/02/ttn-map.jpg"
alt="" />
</a>
</div>
<p>And <a href="https://www.thethingsnetwork.org/" target="_blank" rel="noreferrer noopener">The ThingsNetwork</a>
(TTN) was established in Amsterdam, The Netherlands. As an individual you can extend The Things Network by deploying a <a href="https://www.thethingsnetwork.org/docs/gateways/" target="_blank" rel="noreferrer noopener">Gateway</a>
. Via the TTN KickStarter project, I was one of the first backers, already in 2015. The interest was overwhelming, even leading to (Gateway) delivery problems. But a LoRa Gateway to extend TTN is almost a commodity now. You can even <a href="https://www.thethingsnetwork.org/docs/gateways/start/" target="_blank" rel="noreferrer noopener">build one yourself</a>
. TTN is very much tied to the whole “DIY makers movement”. All TTN designs and code (on <a href="https://github.com/thethingsnetwork" target="_blank" rel="noreferrer noopener">GitHub</a>
) are open. Below a global architecture picture from their site.</p>
<p><img src="https://justobjects.nl/uploads/2018/02/ttn-arch.jpg" alt=" "></p>
<p>So TTN organized <a href="https://www.thethingsnetwork.org/conference/" target="_blank" rel="noreferrer noopener">their first conference</a>
, off course in Amsterdam. For three days: it was an amazing success, more than 500 enthousiasts.</p>
<div class="">
<a href="https://www.thethingsnetwork.org/conference/">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/02/ttn-conf.jpg"
alt="" />
</a>
</div>
<p>The conf was very hands-on with lots of free workshops (with free takeaway hardware). Followed several workshops, which were intense (hardware+software hacking) but always rewarding (blinking green lights!). One to mention in particular (as a Python programmer) was on <a href="https://pycom.io/product/lopy/" target="_blank" rel="noreferrer noopener">LoPy</a>
a sort of Arduino board, very low cost (around $30), programmable with MicroPython that connects directly to TTN. Within an hour the board was happily sending meteo-data to the TTN.</p>
<div class="">
<a href="https://twitter.com/justb4/status/959113531449802752">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/02/lopy.jpg"
alt="" />
</a>
</div>
<p>All in all this conference made me eager to explore more of LoRA with TTN, in particular to explore possibilities for citizen-based sensor-networks for environmental, in particular air quality, data. I am aware that “IoT” has some bad connotations when it comes to security, especially from closed technologies. But IoT is a movement we cannot stop. With and end-to-end open technology like the TTN there is at least the possibility to avoid the “black box”-part and take things in our own hand.</p>
-
Emit #2 – On Air Quality Sensors
https://justobjects.nl/emit-2/
Fri, 26 Jan 2018 23:36:29 +0000
https://justobjects.nl/emit-2/
<p>This is Emit #2, in a <a href="https://justobjects.nl/categories/smartemission/">series of blog-posts around the Smart Emission Platform</a>
, an Open Source software component framework that facilitates the acquisition, processing and (OGC web-API) unlocking of spatiotemporal sensor-data, mainly for Air Quality. In <a href="https://justobjects.nl/emit-1/">Emit #1</a>
, the big picture of the platform was sketched. Subsequent Emits will detail technical aspects of the SE Platform. “Follow the data” will be the main trail loosely adhered to.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/arch-big-picture.jpg" alt=" "></p>
<p>In this Emit I will talk a bit about sensors as the data flow originates there. Mind, this is not my area of expertise, but much of the SE platform, in particular data processing (ETL), is built around challenges of dealing with (cheap) sensors for Air Quality.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/cheap-aq-sensors.jpg" alt=" "></p>
<p>Previously I posted about meteo sensors and weather stations (<a href="https://justobjects.nl/into-the-weather-part-1/">part 1</a>
, <a href="https://justobjects.nl/into-the-weather-part-2-fun-with-raspberry-pi/">part 2</a>
, <a href="https://justobjects.nl/into-the-weather-part-3/">part 3</a>
): how to connect a weather station to a Raspberry Pi and publish weather data “to the cloud”. Now this was relatively easy, due to the availability of:</p>
<ul>
<li>affordable hardware, like meteo sensors and <a href="https://www.wunderground.com/weatherstation/buyingguide.asp" target="_blank" rel="noreferrer noopener">Personal Weather Stations</a>
</li>
<li>Open Source software to collect, store and visualize data like <a href="http://www.weewx.com/" target="_blank" rel="noreferrer noopener">weewx</a>
</li>
<li>community cloud services like <a href="https://www.wunderground.com/" target="_blank" rel="noreferrer noopener">Weather Underground</a>
and <a href="https://wow.metoffice.gov.uk/" target="_blank" rel="noreferrer noopener">Met Office WOW</a>
to publish your data</li>
</ul>
<p>So without any programming, you can be “in business” quite quickly with your personal weather station. In addition: meteo sensors (temperature, humidity, pressure, wind, rain) in general produce relatively “clean/interpretable data”. From a cheap sensor like the <a href="https://www.adafruit.com/product/385" target="_blank" rel="noreferrer noopener">$9,95 DH22 Temperature Humidity Sensor</a>
, it is relatively straightforward to read-out temperature and humidity via an Arduino Board or Raspberry Pi. Personal Weather Stations provide even more internal software, so most meteo data comes out in well-known units (Fahrenheit/Celsius, HectoPascal, etc).</p>
<p>Now this is a whole different story for (cheap) Air Quality sensors. It begins with the fact that measuring Air Quality indicators like Carbon Monoxide/Dioxide (CO, CO2), Nitrogen Monoxide/Dioxide (NO, NO2), Particulate Matter (PM, i.e. fine dust), Ozone (O3) <em>requires many ways …, with both simple chemical and physical methods and with more sophisticated electronic techniques</em> (from <a href="http://www.enviropedia.org.uk/Air_Quality/Measuring.php" target="_blank" rel="noreferrer noopener">www.enviropedia.org.uk</a>
). While techniques for measuring weather data have evolved for maybe hundreds of years, measuring Air Quality is relatively new and mostly within the domain of “chemistry”, and when it comes to sensors, very expensive.</p>
<p>Recently, this has changed. Not only are governmental environmental agencies facing lowering budgets, but more importantly, a growing number of civilian initiatives want to “take things in their own hand” with respect to measuring Air Quality. As a result more and more affordable/cheap sensors and creative solutions like the <a href="http://ispex.nl/" target="_blank" rel="noreferrer noopener">iSpex</a>
(measure PM on your iPhone) are entering the market. But given the (chemical) complexity, <a href="https://www.sciencedirect.com/science/article/pii/S0160412016309989" target="_blank" rel="noreferrer noopener">how reliable are these sensors</a>
? Is the data that they produce readily usable? Like with Celsius to Fahrenheit, is it a matter of providing some simple formula?</p>
<p>IMHO unfortunately not, but things are getting better as time passes. It also depends on the chemical component you want to measure. For example, Nitrogen Dioxide (NO2) and Ozone (O3) appear to be much harder to measure than CO/CO2. Particulate Matter is a whole story by itself as one deals with, well, “dust” in many shapes and especially sizes (PM10, PM2.5, PM1).</p>
<p><img src="https://justobjects.nl/uploads/2018/01/cheapaq.jpg" alt=" "></p>
<p>There is ample research for the quest of finding cheap AQ sensors: their limitations, reliabilities, particularities. Within the Smart Emission Project, I am working with RIVM, the <a href="http://rivm.nl/" target="_blank" rel="noreferrer noopener">Dutch National Institute for Public Health and the Environment</a>
and the <a href="https://ec.europa.eu/jrc/en" target="_blank" rel="noreferrer noopener">European Union Joint Research Centre</a>
(JRC), who both did extensive research on cheap AQ sensors. I know there is much more, but forgot to mention that the main message of this Emit is that <em><strong>“measuring AQ has far more challenges than measuring weather data”</strong></em>. One of the main conclusions, again IMHO, is that, yes, it is possible (to use cheap AQ sensors), but one has to do <strong><em>Calibration</em></strong>. Below some links if you are interested in the state of RIVM and EU JRC research:</p>
<ul>
<li>RIVM – <a href="http://www.rivm.nl/dsresource?objectid=420a3851-7bae-4a1b-8c57-587422cb21b4&type=org&disposition=inline" target="_blank" rel="noreferrer noopener">Luchtkwaliteit meten met sensoren</a>
(Measuring AQ with Sensors), J. Wesseling et al, sorry in Dutch.</li>
<li>RIVM – <a href="https://www.samenmetenaanluchtkwaliteit.nl/apparaten-en-kits" target="_blank" rel="noreferrer noopener">AQ Kits</a>
, from the project/portal <a href="https://www.samenmetenaanluchtkwaliteit.nl/" target="_blank" rel="noreferrer noopener">“Together Measuring Air Quality”</a>
(sorry Dutch)</li>
<li>EU JRC – <a href="https://www.sciencedirect.com/science/article/pii/S092540051500355X" target="_blank" rel="noreferrer noopener">Field Calibration with Low Cost Sensors</a>
– L. Spinelle, M. Gerboles et al</li>
</ul>
<p>Though the latter EU JRC report may be a tough read, it is one of the most detailed and concise reports on the application of low-cost AQ sensors, and, I mention it again, different techniques for <em><strong>Calibration</strong></em>.</p>
<p>So back to the Smart Emission Platform, what sensors are we using currently? The SE Platform started with the <a href="http://smartemission.ruhosting.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Nijmegen Project</a>
, where participating citizens of the City of Nijmegen, would each have their own sensor station that publishes data to the SE Platform.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/intemo-sensor-josene5.jpg" alt=" "></p>
<p>Our partner in the project <a href="http://www.intemo.com/" target="_blank" rel="noreferrer noopener">Intemo</a>
, develops a sensor station, Josene nicknamed ‘Jose’, that measures not only AQ but also sound levels (Noise Pollution) and many other indicators, like light.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/smart-emission-citizens-measuring-air-quality-overview-6-638.jpg" alt=" "></p>
<p>In the course of the project I was fortunate to follow a workshop at <a href="https://ec.europa.eu/jrc/en" target="_blank" rel="noreferrer noopener">EU JRC</a>
for their amazing Open Hardware/Software product <a href="http://www.airsenseur.org/" target="_blank" rel="noreferrer noopener">AirSensEUR</a>
. At the spot each of us assembled a complete ASE, connecting these to standard web services like SOS. The ASE Open Hardware approach also allows it to embed a multitude of sensor types and brands. The workshop had participants from the major national environmental agencies in Europe. In fact RIVM is now <a href="https://www.samenmetenaanluchtkwaliteit.nl/airsenseur" target="_blank" rel="noreferrer noopener">deploying and testing about 18 AirSensEURs</a>
. Coming year I have the task to deploy five ASEs within The Netherlands. Two of them are already humming at my desk for testing.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/ase-jo.jpg" alt="AirSensEUR #2 at my desk"></p>
<p>Describing AirSensEUR would require a full post by itself. Quoting: <em>“AirSensEUR is an open framework focused on air quality monitoring using low cost sensors. The project started on 2014 from a group of passionate researchers and engineers. The framework is composed by a set of electronic boards, firmware, and several software applications.”</em></p>
<p><img src="https://justobjects.nl/uploads/2018/01/airsenseur.jpg" alt="EU JRC AirSensEURs"></p>
<p>So currently (jan 2018) the SE Platform supports both the Josene/Jose and AirSensEUR sensor devices.</p>
<div class="">
<a href="https://twitter.com/justb4/status/734789344389701632">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/01/ase-ws.jpg"
alt="" />
</a>
</div>
<p>The Air Quality sensor data coming out of these devices still requires cleanup and Calibration. This is part of the data handling within the SE platform, subject of one of the upcoming Emits.</p>
<p>This post was meant to give you a taste of the challenges around using (cheap) sensors for Air Quality and introduce the two sensor devices (Josene, AirSensEUR) currently used/supported by the Smart Emission Platform. Many details are still to be uncovered. These will be subjects of upcoming Emits.</p>
-
Emit #1 – Into Spatiotemporal
https://justobjects.nl/emit-1/
Sat, 20 Jan 2018 17:02:39 +0000
https://justobjects.nl/emit-1/
<p><img src="https://justobjects.nl/uploads/2018/01/se-google-photo.jpg" alt="Smart Emission Googled for Photos"></p>
<p>One of my new year’s resolutions for 2018 was to “blog more”. Not being very active on the well-known social media: a bit tired of Twitter, never really into Facebook, bit of LinkedIn. <a href="https://lists.osgeo.org/mailman/listinfo" target="_blank" rel="noreferrer noopener">OSGeo mailing lists</a>
, <a href="https://github.com/justb4" target="_blank" rel="noreferrer noopener">GitHub</a>
and <a href="https://gitter.im/geopython/" target="_blank" rel="noreferrer noopener">Gitter</a>
is where you can find me most (thanks Jody, for reminding!). And I read many blogs, especially on my <a href="https://en.wikipedia.org/wiki/Nexus_10" target="_blank" rel="noreferrer noopener">Nexus 10</a>
tablet and <a href="https://www.fairphone.com" target="_blank" rel="noreferrer noopener">Fairphone 2</a>
via the awesome <a href="https://feedly.com" target="_blank" rel="noreferrer noopener">Feedly</a>
App. If you have not heard of <a href="https://feedly.com" target="_blank" rel="noreferrer noopener">Feedly</a>
(or any other blog-feed collectors), stop here and check out <a href="https://feedly.com" target="_blank" rel="noreferrer noopener">Feedly</a>
! Most blogs (like this one) provide an <a href="https://justobjects.nl/feed/" target="_blank" rel="noreferrer noopener">RSS/Atom-feed</a>
. Via <a href="https://feedly.com" target="_blank" rel="noreferrer noopener">Feedly</a>
you can search/add RSS-feeds and thus create your own “reading table”. My favorite feeds are related to Open Source Geospatial, Python and IoT, like:</p>
<ul>
<li><a href="http://planet.osgeo.org/" target="_blank" rel="noreferrer noopener">planet.osgeo.org</a>
– the OSGeo blog aggregator, must-read if you’re in Open Geospatial</li>
<li><a href="https://sgillies.net/" target="_blank" rel="noreferrer noopener">Sean Gillies blog</a>
– developer at MapBox a.o. of <a href="http://toblerity.org/fiona/" target="_blank" rel="noreferrer noopener">Fiona</a>
, GeoPython, geospatial and personal stuff</li>
<li><a href="http://planetpython.org/" target="_blank" rel="noreferrer noopener">Planet Python</a>
– all Python</li>
</ul>
<div class="">
<a href="https://feedly.com/i/category/Geospatial">
<img class="lazy"
src="https://justobjects.nl/uploads/2018/01/feedly.png"
alt="Feedly shown in web browser" />
</a>
</div>
<p>Enough sidestepping, my goal is to share tech around the Open Source <a href="http://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
(SE Platform) in a series of posts, dubbed ‘Emits’. This is Emit #1. Since 2014 I have been working on several projects, often through <a href="https://www.geonovum.nl/" target="_blank" rel="noreferrer noopener">Geonovum</a>
, and recently via the <a href="https://ec.europa.eu/jrc/en" target="_blank" rel="noreferrer noopener">European Union Joint Research Centre</a>
(JRC), that deal with the acquisition, management, web-api-unlocking and visualization of environmental sensor-data, mainly for Air Quality (AQ).</p>
<p><img src="https://justobjects.nl/uploads/2018/01/se-google.jpg" alt="Smart Emission Googled"></p>
<p>What made these projects exciting for me is that they brought together many aspects and technologies (read: Open Source projects and <a href="https://www.osgeo.org/" target="_blank" rel="noreferrer noopener">OSGeo software</a>
) I had been <a href="https://justobjects.nl/projects/" target="_blank" rel="noreferrer noopener">working on</a>
through the years. Also, it was the first time I got back into Environmental Chemistry, for which I hold a master’s degree from the University of Amsterdam, <a href="https://justobjects.nl/publications/" target="_blank" rel="noreferrer noopener">co-authoring some publications</a>
, yes, many many years ago.</p>
<p>So what is the <a href="http://data.smartemission.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Platform</a>
and what makes it exciting and relevant? In a nutshell (<a href="http://smartplatform.readthedocs.io/" target="_blank" rel="noreferrer noopener">read the tech doc here</a>
): The goal of the SE Platform is to facilitate the acquisition (harvesting) of sensor-data from a multitude of sensor devices and make this data available via standardized formats and web-APIs (mainly: <a href="http://www.opengeospatial.org/" target="_blank" rel="noreferrer noopener">OGC Standards</a>
) and Viewers. The SE Platform originates, what is now called the <a href="http://smartemission.ruhosting.nl/visitors/" target="_blank" rel="noreferrer noopener">award-winning</a>
, <a href="http://smartemission.ruhosting.nl/" target="_blank" rel="noreferrer noopener">Smart Emission Nijmegen project</a>
in 2015-2017. Quoting from the paper <a href="http://www.ru.nl/publish/pages/774337/carton_etall_aesop-2015_v11_filling_thefeedback_gap_ofexternalities_insmartcities.pdf" target="_blank" rel="noreferrer noopener">“Filling the feedback gap of place-related externalities in smart cities”</a>
{.reference.external} :</p>
<p><em>“…we present the set-up of the pilot experiment in project “Smart Emission”,</em> <em>constructing an experimental citizen-sensor-network in the city of Nijmegen. This project, as part of</em> <em>research program ‘Maps 4 Society,’ is one of the currently running Smart City projects in the</em> <em>Netherlands. A number of social, technical and governmental innovations are put together in this</em> <em>project: (1) innovative sensing method: new, low-cost sensors are being designed and built in the</em> <em>project and tested in practice, using small sensing-modules that measure air quality indicators,</em> <em>amongst others NO2, CO2, ozone, temperature and noise load. (2) big data: the measured data forms</em> <em>a refined data-flow from sensing points at places where people live and work: thus forming a ‘big</em> _picture’ to build a real-time, in-depth understanding of the local distribution of urban air quality (3)_<em>empowering citizens by making visible the ‘externality’ of urban air quality and feeding this into a</em> <em>bottom-up planning process: the community in the target area get the co-decision-making control over</em> <em>where the sensors are placed, co-interpret the mapped feedback data, discuss and collectively explore</em> <em>possible options for improvement (supported by a Maptable instrument) to get a fair and ‘better’</em> <em>distribution of air pollution in the city, balanced against other spatial qualities. ….”</em></p>
<p><img src="https://justobjects.nl/uploads/2018/01/se-nijmegen.jpg" alt=" "></p>
<p>So from the outset the SE Platform is geared towards connecting citizen-owned sensor devices. Many similar programs and initiatives are currently evolving, often under the flag of Citizen Science and Smart Cities. Within the Netherlands, where the SE Nijmegen project originated, the <a href="http://rivm.nl" target="_blank" rel="noreferrer noopener">Dutch National Institute for Public Health and the Environment</a>
(RIVM) was an active project partner, and still stimulates citizens measuring Air Quality via a project and portal: <a href="https://www.samenmetenaanluchtkwaliteit.nl/" target="_blank" rel="noreferrer noopener">“Together Measuring Air Quality”</a>
. In the context of discussions on Air Quality, climate change and lowering budgets for governmental environmental institutions, citizen-participation becomes more and more relevant. A whole series of blogs could be devoted to social and political aspects of Citizen Science, but I will stick to tech-stuff here.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/se-partners.png" alt=":left"></p>
<p>What made working on the SE Nijmegen project exciting and challenging, is that I was given time and opportunity by the project partners (see pic) to not just build a one-time project-specific piece of software, but a reusable set of Open Source components: the Smart Emission Platform ([sources on GitHub][42]).</p>
<p>Having had some earlier experience within the <a href="http://sensors.geonovum.nl/" target="_blank" rel="noreferrer noopener">Geonovum SOSPilot project</a>
(2014-2015), investigating among others the <a href="https://en.wikipedia.org/wiki/Sensor_Observation_Service" target="_blank" rel="noreferrer noopener">OGC Sensor Observation Service</a>
to unlock <a href="http://www.lml.rivm.nl/" target="_blank" rel="noreferrer noopener">RIVM AQ data (LML)</a>
, I was aware of the challenges dealing with what can be called <em><strong>Spatiotemporal (Big) Data</strong></em>.</p>
<p> </p>
<p>The figure below shows The Big Picture of the SE Platform. Red arrows denote the flow of data: originating from sensor devices, going through Data Management (ETL), unlocked via various web-APIs, and finally “consumed” in client-apps and viewers.</p>
<p><img src="https://justobjects.nl/uploads/2018/01/arch-big-picture.jpg" alt=" "></p>
<p>There are many aspects of the SE Platform that can be expanded. These are for upcoming Emits. For now a summary of some of the challenges and applied technologies, to be detailed later:</p>
<ul>
<li>raw data from sensors: requires refinement: validation/calibration/aggregation</li>
<li>dealing with Big Data that is both spatial (location-based) and temporal (time-based)</li>
<li>applying an <a href="https://en.wikipedia.org/wiki/Artificial_neural_network" target="_blank" rel="noreferrer noopener">Artificial Neural Network</a>
(ANN) for sensor-data calibration</li>
<li>databases for Spatiotemporal data: PostGIS and <a href="https://www.influxdata.com/time-series-platform/" target="_blank" rel="noreferrer noopener">InfluxDB and TICK Stack</a>
</li>
<li>applying the <a href="http://www.stetl.org/" target="_blank" rel="noreferrer noopener">Stetl framework</a>
for all data management (ETL)</li>
<li>metadata for sensors and sensor networks, always a tough and often avoided subject</li>
<li>connecting the Open Hardware <a href="http://www.airsenseur.org/" target="_blank" rel="noreferrer noopener">EU JRC AirSensEUR</a>
AQ sensor-box to the SE Platform</li>
<li>using OGC WMS (with Dimensions for Time) and WFS for viewing and downloading sensor data</li>
<li>is <a href="http://www.opengeospatial.org/standards/sos" target="_blank" rel="noreferrer noopener">OGC Sensor Observation Service</a>
(SOS) and SWE still viable?</li>
<li>how powerful is the <a href="http://www.opengeospatial.org/standards/sensorthings" target="_blank" rel="noreferrer noopener">OGC SensorThings API</a>
(STA) standard?</li>
<li>deployment with Docker and <a href="https://docs.docker.com/compose/" target="_blank" rel="noreferrer noopener">Docker Compose</a>
</li>
<li>Docker and host systems monitoring: <a href="https://prometheus.io/" target="_blank" rel="noreferrer noopener">Prometheus</a>
+ Grafana</li>
<li>OGC Services Monitoring with <a href="http://geohealthcheck.org/" target="_blank" rel="noreferrer noopener">GeoHealthCheck</a>
</li>
<li>Visualizations: custom viewers with <a href="http://heron-mc.org/" target="_blank" rel="noreferrer noopener">Heron</a>
/Leaflet/OpenLayers, Grafana dashboards</li>
<li>from development to test and production: <a href="https://www.vagrantup.com/" target="_blank" rel="noreferrer noopener">Vagrant+VirtualBox</a>
, Ubuntu, Docker</li>
<li>using component-subsets of the platform for small deployments</li>
</ul>
<p><img src="https://justobjects.nl/uploads/2018/01/docker-monitoring.jpg" alt="Monitoring SE Docker Containers: Prometheus+cAdvisor+Grafana"></p>
<p>A lot of stuff to uncover, hopefully got your interest if you have read all the way to here. Will try to treat one aspect/technology in each subsequent Emit-blog post. And off course the entire <a href="https://github.com/geonovum/smartemission/" target="_blank" rel="noreferrer noopener">SE platform is Open Source</a>
(GNU GPL), so you are free to download and experiment, and maybe even would like to contribute.</p>
-
Dances with the GeoPython Tribe
https://justobjects.nl/dances-with-the-geopython-tribe/
Wed, 13 Jul 2016 23:06:10 +0000
https://justobjects.nl/dances-with-the-geopython-tribe/
<p>During June 21-24, 2016 I attended the very first <a href="http://www.geopython.net/" target="_blank" rel="noreferrer noopener">GeoPython Conference in Basel Switzerland</a>
. This event was organized by the <a href="http://www.fhnw.ch/habg/ivgi/" target="_blank" rel="noreferrer noopener">Institute of Geomatics Engineering of the FHNW – University of Applied Sciences and Arts Northwestern Switzerland</a>
and <a href="http://www.meetup.com/PyBasel-Basel-Python-Meetup/" target="_blank" rel="noreferrer noopener">PyBasel</a>
, the local Python User Group Northwestern Switzerland. In particular I should mention key-organizer <a href="https://www.linkedin.com/in/martinchristen" target="_blank" rel="noreferrer noopener">Martin Christen</a>
from FHNW. He and his team made this into a such a great event that GeoPython 2017 is already planned. About 130 people attended, most from Europe, but also from other continents. For a TL;DR the conference website <a href="http://www.geopython.net" target="_blank" rel="noreferrer noopener">www.geopython.net</a>
provides you all the details: not just the program, but also the “post-processing”: slides, photo’s (<a href="https://www.flickr.com/photos/144781014@N02/sets/72157667869241134/" target="_blank" rel="noreferrer noopener">on Flickr</a>
) etc. The conference also included time and resources (room, food, beverages) for code-sprints. One of the outcomes of the conference-survey was to establish a public GeoPython mailing list at python.org. Details: to subscribe, send mail to: <u><a href="mailto:[email protected]">[email protected]</a>
</u>with the keyword “subscribe” in the subject, or use the web-interface: <a href="https://mail.python.org/mm3/mailman3/lists/geopython.python.org/">https://mail.python.org/mm3/mailman3/lists/geopython.python.org/</a>.</p>
<div class="float_right">
<a href="http://www.geopython.net/">
<img class="lazy"
src="https://justobjects.nl/uploads/2016/07/gp1-right.jpg"
alt="" />
</a>
</div>
<p>So why a dedicated GeoPython conference? IMHO Python makes more and more sense for Open Source geospatial development. Not just for custom geo-scripting or glueing with e.g. <a href="http://gdal.org" target="_blank" rel="noreferrer noopener">GDAL</a>
, or for developing plugins for <a href="http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/plugins.html" target="_blank" rel="noreferrer noopener">QGIS</a>
and <a href="https://grass.osgeo.org/" target="_blank" rel="noreferrer noopener">GRASS</a>
, but more and more as a mature framework language for geospatial processing and OGC services. The projects <a href="http://pywps.org" target="_blank" rel="noreferrer noopener">PyWPS</a>
and <a href="http://pycsw.org" target="_blank" rel="noreferrer noopener">PyCSW</a>
are an example of the latter. To access OGC services from Python clients there is <a href="http://geopython.github.io/OWSLib/" target="_blank" rel="noreferrer noopener">OWSLib</a>
. Upcoming geospatial CMS frameworks like <a href="http://geonode.org" target="_blank" rel="noreferrer noopener">GeoNode</a>
and the very recent <a href="http://boundlessgeo.com/exchange/" target="_blank" rel="noreferrer noopener">Boundless Exchange</a>
, powered by GeoNode, show that Python has the potential to become “the new Java” within the Open Source geospatial world.</p>
<p>Did I say “Java”? Ok: did almost 20 years of Java, from the very first JDK somewhere in 95/96 (Applets!), through <a href="https://en.wikipedia.org/wiki/Microsoft_Java_Virtual_Machine" target="_blank" rel="noreferrer noopener">Sun vs Microsoft over Java</a>
, from the heaviness of <a href="https://en.wikipedia.org/wiki/Enterprise_JavaBeans" target="_blank" rel="noreferrer noopener">J2EE/EJB</a>
s, to the lighter weighings of <a href="https://en.wikipedia.org/wiki/Spring_Framework" target="_blank" rel="noreferrer noopener">Spring</a>
, the settling of Java as a backend/server technology. Sidestep: “Java” seems to be a central keyword in my family’s ancestry: <a href="https://en.wikipedia.org/wiki/Pieter_van_den_Broecke" target="_blank" rel="noreferrer noopener">my great-great grandfather was one of the first people in the world to drink a cup of Java</a>
and also was one of the first to set foot on the Indonesian island of Java, being on the same ship with <a href="https://nl.wikipedia.org/wiki/Jan_Pieterszoon_Coen" target="_blank" rel="noreferrer noopener">Jan Pieterszoon Coen</a>
. My grandfather lived for 20 years in <a href="https://en.wikipedia.org/wiki/Malang" target="_blank" rel="noreferrer noopener">Malang</a>
(East Java), working as a civil (Delft) engineer. But I am diverting. The colonial period was by times a violent (by the Dutch) episode in Dutch history, not to be proud of.</p>
<div class="float_left">
<a href="http://www.geopython.net/">
<img class="lazy"
src="https://justobjects.nl/uploads/2016/07/gp2-left.jpg"
alt="" />
</a>
</div>
<p>But times they are a-changing, technologies are evolving. I am happy these days to develop in Python (and JavaScript for the web-frontend). Like moving from C/C++ to Java back then, and now from Java to Python, what appeals to me: shortened development times, lesser lines of code to debug and maintain, ease of deployment, a central repository (<a href="https://pypi.python.org/pypi" target="_blank" rel="noreferrer noopener">PyPi</a>
) , an independent, vibrant community and possibly more. But again I am diverting, there are great and stable geo-products in Java like <a href="http://geoserver.org" target="_blank" rel="noreferrer noopener">GeoServer</a>
I use daily. Diversity in programming languages is good. Someone (<a href="https://www.linkedin.com/in/jodygarnett" target="_blank" rel="noreferrer noopener">Jody Garnett</a>
?) posted somewhere about the C-tribe and Java-tribe within the Open Source geospatial world, but can’t find the reference. Back to the subject of this post!</p>
<p> </p>
<p>Like said, the organizing team has done a great job postprocessing the event, to be found via <a href="http://www.geopython.net/" target="_blank" rel="noreferrer noopener">www.geopython.net</a>
so listing all talks/workshops will not add value here. My overall feeling was that this conference, like the very first <a href="http://2006.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G in Lausanne</a>
I attended in 2006, was the beginning of a global community. Above all this was also an event where folks with a shared interest met and conversed. Often at conferences one learns and shares the most during breaks and social gatherings. In short: I learned a lot, being a relative newcomer in the geospatial Python community: GeoNode, GeoDjango, Python with Grass, and much more. For example I learned about some general Python technologies like the <a href="http://jupyter.org/" target="_blank" rel="noreferrer noopener">Jupyter Notebook</a>
that many presenters used. I found that many (like me) are using <a href="http://flask.pocoo.org/" target="_blank" rel="noreferrer noopener">Flask</a>
for simple Python webapps/REST APIs. Flask expresses one of features I like about Python: minimalism.</p>
<p>I am grateful to the organizing team that I could <a href="http://www.slideshare.net/justb4/geospatial-etl-with-stetl-geopython-2016" target="_blank" rel="noreferrer noopener">present Stetl</a>
in the last session on the last day, since I submitted past the deadline. Luckily the room was still filled, though the cold beer was tempting just ahead. Mind: during the conference there was a heatwave, above 30 degrees C each day, yes in Switzerland. But all in all this was a very cool conference! Hope to see you at GeoPython 2017!</p>
-
Deploying Open Source Geospatial Software – Part 1: Challenges
https://justobjects.nl/deploying-open-source-geospatial-1/
Fri, 10 Jun 2016 01:03:36 +0000
https://justobjects.nl/deploying-open-source-geospatial-1/
<p>My blog has been quiet for some time. As many of us I’ve been busy doing projects, all involving Open Source Geo (OSGeo) software. Partly development, writing software, I love it, but also more and more in “what comes next”: deploying and maintaining “the application” with all of its dependencies. For this I have been using several “deployment strategies” I would like to share. To be specific and for a TL;DR : over the years I went through custom compiles/installs, Debian/Ubuntu(GIS) package installation, writing Debian/RPM packages, using Puppet (not yet Chef), and now sitting on the <a href="http://docker.com" target="_blank" rel="noreferrer noopener">Docker</a>
of the Bay. For many this last sentence may be gibberish, so I will try to sketch some context first. Calling this blog <em>Part 1</em> also hopefully keeps me attached to the subject and writing as I have very good news. But today, ‘helas’, the bad and the ugly.</p>
<p>In terms of architecture I always prefer a “best-of-breed” selection of Open Source Geospatial (OSGeo) software components, rather than select a single platform/”Suite”. Nothing against Suites, this is a domain where Open Source Geo providers, are, literally, “stacking up” against proprietary GIS providers. <a href="http://boundlessgeo.com/" target="_blank" rel="noreferrer noopener">Boundless</a>
, <a href="http://www.geo-solutions.it/" target="_blank" rel="noreferrer noopener">GeoSolutions</a>
, <a href="http://www.geomajas.org" target="_blank" rel="noreferrer noopener">Geomajas</a>
, to name a few, have great platforms you should check out. Because I like to dive deep into open source geospatial technology, trying to contribute where possible, even <a href="https://github.com/justb4" target="_blank" rel="noreferrer noopener">writing some myself</a>
, and having experienced the pros and cons of each individual component, I tend to go for a best fit in a project. For example, for WMS/WFS I may apply <a href="http://mapserver.org/" target="_blank" rel="noreferrer noopener">MapServer</a>
or <a href="http://geoserver.org" target="_blank" rel="noreferrer noopener">GeoServer</a>
or <a href="http://deegree.org" target="_blank" rel="noreferrer noopener">deegree</a>
, for web clients <a href="http://openlayers.org" target="_blank" rel="noreferrer noopener">OpenLayers</a>
or <a href="http://leaflet.org" target="_blank" rel="noreferrer noopener">Leaflet.</a>
As for tiling, well, to be honest, nothing beats <a href="http://MapProxy.org" target="_blank" rel="noreferrer noopener">MapProxy</a>
. <a href="http://gdal.org" target="_blank" rel="noreferrer noopener">GDAL</a>
, <a href="http://qgis.org" target="_blank" rel="noreferrer noopener">QGIS</a>
, <a href="http://grass.org" target="_blank" rel="noreferrer noopener">GRASS</a>
, <a href="http://geonetwork-opensource.org/" target="_blank" rel="noreferrer noopener">GeoNetwork</a>
or <a href="http://pycsw.org/" target="_blank" rel="noreferrer noopener">pycsw</a>
, I could go on. I am a huge fan of each of these projects, standing on the shoulders of giants when using their products. It depends on the project’s requirements what I choose.</p>
<p>But going for a “best-of-breed” architecture, where a selection of Open Source Geospatial components is made, usually extended with custom software and configurations, creates challenges in deployment and maintenance. With the latter I mean: going into production (live) and maintaining the system for an N number of years through modifications and updates. “Getting it working” on a single system will often succeed, possibly after a great number of Google searches, mailing list threads, then finally getting all components and dependencies installed, often by hand. In some cases even recompiling components, moving libraries, setting PATHs etc. At some point “it all works” but at the same time we enter the “don’t touch it” phase. We have an “upgrading issue”, but doable on a single system/server.</p>
<p>To worsen this situation: most professional IT-departments employ a multi-step deployment-strategy. There is not just a single system where the application runs, but several systems, each dedicated to, and named after their phase in deployment. For example, governmental projects within The Netherlands often deploy “OTAP”. <a href="https://nl.wikipedia.org/wiki/OTAP" target="_blank" rel="noreferrer noopener">OTAP</a>
(in English <a href="https://en.wikipedia.org/wiki/Development,_testing,_acceptance_and_production" target="_blank" rel="noreferrer noopener">DTAP</a>
) stands for Development, Test, Acceptance, Production. These are, often rigorously, separated computing infrastructures (servers, clients). An application with all its dependencies has to be deployed sequentially on/through each of these phases, sometimes called “pillars” (Dutch: zuilen). In many cases a direct connection between these systems is blocked by the IT-department. In the simplest case, we have a Test and Production system. Hence, our carefully handcrafted system will have a major challenge getting from one pillar to the next. But I am not finished yet, we have the “tribal thing” going on in Open Source Geospatial software. Let me expand.</p>
<p>Diversity is good. Also in software. Over the years Open Source Geospatial software has been developed using a plethora of programming languages. Each came with a variety of deployment systems. I am talking about Java, Python, JavaScript/NodeJS, C/C++, and recently Go. These languages usually have some kind of library and deployment technology. Take Java: for server side components we need to have an “J2EE Container”, in most cases Tomcat, and deploy <em>.war</em> files (e.g. GeoServer or GeoNetwork). For Python and “CGI-able” components like MapServer, we may just need a CGI-server like Apache or Nginx. Each of these products deploys in its own way, has its own method for maintaining its configuration and managing updates. In Dutch we call this a “Lappendeken”. The closest translation I found is a “patchwork”, that is to say a diverse deployment and maintenance system. Individual products may provide a “GUI” to manage configurations, stored in diverse ways, from single XML/YAML files to even databases. No way to manage these products in a uniform way. For an outsider, or a cynical proprietary GIS-provider, this all could be labeled, as “Open Source Geospatial (deployment) is a big mess”.</p>
<p>So dear readers, sketching this bag of problems, in a positive sense: challenges, how we go from here? As I indicated, there is good news. The answer, my friend, lies in _abstraction. _Abstraction is the way that software technology has always progressed: from machine instructions to assembly and programming languages, through data structures, objects and classes. To components and packages. Coupling and cohesion is another progressing force: maximizing cohesion (do one thing good) and minimizing coupling (reduce dependencies). All in all I have been finding solutions to the above problems using very accessible technologies. In the next two parts I hope to expand on these further as I am picking just two (Deployment Strategies) for now. The first is <a href="https://wiki.debian.org/Packaging" target="_blank" rel="noreferrer noopener">Debian Packaging</a>
(with some <a href="https://puppet.com/" target="_blank" rel="noreferrer noopener">Puppet</a>
), the second is <a href="https://www.docker.com/" target="_blank" rel="noreferrer noopener">Docker</a>
. In short: what to expect in my next two blogs (Part 2 and Part 3):</p>
<ul>
<li><em><strong>Debian Packaging:</strong></em> writing Debian packages to maintain software and configuration in a multi-step deployment environment</li>
<li><em><strong>Docker:</strong></em> building/maintaining Docker images but keep control (on host) over their configuration, state and functionality</li>
</ul>
<p>Also with some telling images, as these are lacking in this post!</p>
-
Unlocking Sensor Data – I’ll send an SOS to the world
https://justobjects.nl/unlocking-sensor-data-sos/
Thu, 01 Oct 2015 00:42:56 +0000
https://justobjects.nl/unlocking-sensor-data-sos/
<p>Revealing: the title refers, for the younger readers, to a great 1979-hit by The Police as expanded below. To be played at the loudest possible volume. If you don’t see anything here below try the YouTube link directly:</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/MbXWrmQW-OE?controls=1&rel=0" loading="lazy"></iframe>
</div>
<p>One of the main aspects that glues the OSGeo-world together are OGC-standards: WMS, WFS, WMTS, WCS and WPS are, at least for most insiders, not hollow acronyms. But who knows about and uses SOS? SOS stands for <em>“Sensor Observation Service”</em>, <a href="http://www.opengeospatial.org/standards/sos" target="_blank" rel="noreferrer noopener">an OGC standard</a>
within the elaborate framework of the <a href="http://www.opengeospatial.org/projects/groups/sensorwebdwg" target="_blank" rel="noreferrer noopener">OGC SensorWeb Enablement</a>
. SOS provides a standard to publish (SOS-T) and request time-based (meta-) data, mostly from “Sensors”. Its convention is similar to WMS/WFS (GetCapabilities, DescribeSensor, GetObservation etc). Think of weather or air quality measurements over time.</p>
<p>The Internet of <a href="https://en.wikipedia.org/wiki/Internet_of_Things" target="_blank" rel="noreferrer noopener">Things (IoT)</a>
is now gaining a strong momentum: <em>“The Internet of Things (IoT) is the network of physical objects or “things” embedded</em> <em>with electronics, software, sensors, and network connectivity, which enables these objects to collect and exchange data. The Internet of Things allows objects to be sensed and controlled remotely across existing network infrastructure, creating opportunities for more direct integration between the physical world and computer-based systems, and resulting in improved efficiency, accuracy and economic benefit Each thing is uniquely identifiable through its embedded computing system but is able to interoperate within the existing Internet infrastructure. Experts estimate that the IoT will consist of almost 50 billion objects by 2020.” (<a href="https://en.wikipedia.org/wiki/Internet_of_Things" target="_blank" rel="noreferrer noopener">from Wikipedia</a>
).</em></p>
<p>So how will SOS and the generic <a href="http://www.opengeospatial.org/projects/groups/sensorwebdwg" target="_blank" rel="noreferrer noopener">OGC SensorWeb Enablement</a>
will fit into this force? I really don’t know. For many of the OGC-standards like WMS, there are multiple implementations. For SOS I know about just two: the <a href="http://52north.org/communities/sensorweb/sos/" target="_blank" rel="noreferrer noopener">52 North SOS</a>
, and the <a href="http://istsos.org/" target="_blank" rel="noreferrer noopener">IstSOS</a>
. Both of these are powerful with their own strengths and limitations.</p>
<p>I have worked successfully with the <a href="http://52north.org/communities/sensorweb/sos/" target="_blank" rel="noreferrer noopener">52 North SOS</a>
within a <a href="http://sensors.geonovum.nl/" target="_blank" rel="noreferrer noopener">Dutch project on Air Quality</a>
. All details are in <a href="http://sospilot.readthedocs.org/en/latest/" target="_blank" rel="noreferrer noopener">this online document.</a>
In essence we are (still) publishing and emitting Dutch National Air Quality data via a SOS server. At the same time, via GeoServer, using WMS-Time <a href="http://sensors.geonovum.nl/heronviewer/" target="_blank" rel="noreferrer noopener">via this web-client</a>
. On the way I found that the OGC-SOS Standard is complex and quite cumbersome in its usage. 52 North has provided <a href="http://sensorweb.demo.52north.org/sensorwebclient-webapp-stable/api-doc/" target="_blank" rel="noreferrer noopener">a custom REST interface</a>
that appeals to be more usable. But SOS with its inner talk of “Procedures” and “Offerings” remains a Hot Potato.</p>
<div class="">
<a href="http://sensors.geonovum.nl/heronviewer/">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/10/heron-viewer-o3-ts.jpg"
alt="" />
</a>
</div>
<p>So the broader question is more about the OGC SOS standard and the related <a href="http://www.opengeospatial.org/projects/groups/sensorwebdwg" target="_blank" rel="noreferrer noopener">OGC SensorWeb Enablement</a>
: how we as an OSGeo-community think it should evolve within the expanding world of the IoT? My opinion is that we need to strive for more ease-of-use. SOS-as-standard is an academic challenge. A window to the future may be the OGC-effort initiated by Steve Liang: the SensorThings API not only provides a simplification from the original <a href="http://www.opengeospatial.org/projects/groups/sensorwebdwg" target="_blank" rel="noreferrer noopener">OGC SensorWeb Enablement</a>
but also a modern way of community-based cooperation of standards making via GtiHub: <a href="http://ogc-iot.github.io/ogc-iot-api/" target="_blank" rel="noreferrer noopener">http://ogc-iot.github.io/ogc-iot-api</a>
. Time will tell, a message in a bottle will also eventually arrive.</p>
-
GPS Hiking and The Evolution of Navigation
https://justobjects.nl/gps-hiking-evolution-navigation/
Wed, 26 Aug 2015 11:56:33 +0000
https://justobjects.nl/gps-hiking-evolution-navigation/
<p>On sunday August 25, 2015 I provided a talk/workshop for a local knowledge network of friends and acquaintances in Amsterdam. TLDR; see <a href="http://gps.justobjects.nl" target="_blank" rel="noreferrer noopener">gps.justobjects.nl</a>
. Originally, I would talk about <em>“Hiking with GPS”</em>. But as I started pondering about this subject I decided to broaden the content to include also the evolution of navigation. My reasoning was:</p>
<ul>
<li>by using GPS we unlearn navigational skills such as compass and map reading</li>
<li>my personal interest in human evolution for example the route humans followed out of Africa</li>
</ul>
<p><img src="https://justobjects.nl/uploads/2015/08/out-of-africa.jpg" alt="Routes (red) of Homo Sapiens out of Africa to populate the World"></p>
<p>So I combined all these elements into a new title/subject <em>“GPS Hiking and The Evolution of Navigation”</em>. See also <a href="http://www.slideshare.net/justb4/wandelen-met-gps-en-de-evolutie-van-navigatie" target="_blank" rel="noreferrer noopener">these slides</a>
.</p>
<p><strong>Natural Navigation</strong><br>
I was already reading and practicing the intriguing art of Natural Navigation which in short allows one to find direction without any device nor map. By reading and interpreting signs of nature one can determine direction, i.e. where The North is.</p>
<p>Natural Navigation is not just about finding direction by sun, moon and stars, but in particular by interpreting our natural surroundings, i.e. the land, the plants, the trees and animals. Natural forces shape these surroundings, making them asymmetric under the influence of their prevailing forces. This asymmetry in nature provides clues for example where the North is.</p>
<div class="">
<a href="http://www.naturalnavigator.com">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/08/tree-combined-wind-growth.jpg"
alt="Left tree shaped by wind, right by the sun (src: The Natural Navigator, Gooley)" />
</a>
</div>
<p>For instance: sun, rain and wind strongly shape the direction that trees and their branches grow. A prevailing south-western wind will bend trees slightly north-east.</p>
<p>But also mosses and lichen on trees give clues about direction. This has saved me once in finding my way when hiking on a cloudy day without GPS and an emptying smartphone.</p>
<p>To get started with Natural Navigation I recommend the books by Tristan Gooley via <a href="http://www.naturalnavigator.com" target="_blank" rel="noreferrer noopener">www.naturalnavigator.com</a>
.</p>
<div class="">
<a href="http://www.naturalnavigator.com">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/08/book-tnn-gooley.jpg"
alt="The Natural Navigator by Tristan Gooley" />
</a>
</div>
<p>I am also intrigued in how early humans navigated. As hunter/gatherers we had to cross enormous stretches of land, often going far out from a base camp for a hunt or food collection. Did these first humans use dogs for navigation, made landmarks?They sure must have been masters at Natural Navigation. We can get some clues from indigenous people today. Years ago I read a fascinating book <a href="https://en.wikipedia.org/wiki/The_Songlines" target="_blank" rel="noreferrer noopener">“The Songlines” by Bruce Chatwin</a>
, which in short told how Australian Aboriginals could navigate throughout their continent, or <a href="https://en.wikipedia.org/wiki/Songline" target="_blank" rel="noreferrer noopener">as Wikipedia explains</a>
:</p>
<p><em>“Within the animist belief system of Indigenous Australians, a songline, also called dreaming</em> <em>track, is one of the paths across the land which mark the route followed by localised ‘creator-beings’ during the Dreaming (creation time, JvdB). The paths of the songlines are recorded in traditional songs, stories, dance, and painting.</em></p>
<p><img src="https://justobjects.nl/uploads/2015/08/songlines.jpg" alt=" "></p>
<p><em>A knowledgeable person is able to navigate across the land by repeating the words of the song,</em> <em>which describe the location of landmarks, waterholes, and other natural phenomena.”</em></p>
<p><strong>Going Outdoors</strong></p>
<p><img src="https://justobjects.nl/uploads/2015/08/lezing-foto-1.jpg" alt="GeoCaching and Natural Navigation"></p>
<p>We ended the session by doing an exercise with both Natural Navigation and following a route and GeoCaching with GPS. Here my <a href="http://gps.justobjects.nl" target="_blank" rel="noreferrer noopener">Map5.nl</a>
<a href="http://app.map5.nl/nltopo/?key=gast&gpx=gpx/vliegenbos238.gpx" target="_blank" rel="noreferrer noopener">NLTopo App</a>
appeared useful for those without GPS.</p>
<div class="float_left">
<a href="http://app.map5.nl/nltopo/?key=gast&gpx=gpx/vliegenbos238.gpx">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/08/nltopo-app.jpg"
alt="The Route and Caches in the Map5.nl NLTopo App" />
</a>
</div>
<p> </p>
<p>It was fun afternoon, maybe something to reenact in another setting. Let me know if you are interested.</p>
<p>You can find the landing page of this talk/workshop at <a href="http://gps.justobjects.nl" target="_blank" rel="noreferrer noopener">gps.justobjects.nl</a>
and <a href="http://www.slideshare.net/justb4/wandelen-met-gps-en-de-evolutie-van-navigatie" target="_blank" rel="noreferrer noopener">slides on Slideshare</a>
. In Dutch but you may view the <a href="https://translate.google.com/translate?sl=nl&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fgps.justobjects.nl%2F&edit-text=&act=url" target="_blank" rel="noreferrer noopener">page via Google Translate</a>
.</p>
-
JPEG is Dead, Long Live JPEG!
https://justobjects.nl/jpeg-is-dead-long-live-jpeg/
Thu, 21 May 2015 23:31:46 +0000
https://justobjects.nl/jpeg-is-dead-long-live-jpeg/
<p>TL;DR: see <a href="https://github.com/map5nl/map5/issues/10" target="_blank" rel="noreferrer noopener">this GitHub issue</a>
for summary (Dutch but the pictures there and Figure 1 tell the story).</p>
<p><img src="https://justobjects.nl/uploads/2015/05/opentopo-png-vs-jpeg.png" alt="Figure 1. Image encoding-comparison for MapProxy-tiles"></p>
<p>Somewhere around 1995, building my first website, it was already quite a feat to embed images. Non-aware of image-formats I played with (animated!) GIF and JPEG. Naively image-editing I noticed that my JPEGs became worse and worse after each save…I learned quickly about lossless and lossy encodings back then. Later on came PNG. When I entered the geospatial domain there appeared to be a common convention that JPEG was to be used for arial/satellite images and PNG for rasterized vector renderings. So the years went by and all geo-folks, including me, followed that rule.</p>
<p>Recently I developed and launched <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">Map5.nl</a>
: a cloud service for topographic maps mostly made with Dutch Open Geo-data. See my previous post <a href="https://justobjects.nl/tales-from-topographic-lowlands/" target="_blank" rel="noreferrer noopener">Tales from the Topographic Lowlands</a>
how this service evolved. After quite some research I finally settled on an Open Source geo-stack with <a href="http://gdal.org" target="_blank" rel="noreferrer noopener">GDAL</a>
, <a href="http://mapserver.org" target="_blank" rel="noreferrer noopener">MapServer</a>
and <a href="http://mapproxy.org" target="_blank" rel="noreferrer noopener">MapProxy</a>
on <a href="http://www.ubuntu.com" target="_blank" rel="noreferrer noopener">Ubuntu</a>
. For raster-map serving this appeared to be a golden combination. The full stack, including pre-processing of TIFF-files (and some PNGs) is depicted below. The arrows denote the flow of data.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/geostack11.png" alt="Figure 2. Map5.nl dataflow for Raster data serving"></p>
<p>In a later post I will dive more into the details of this architecture. For now I should explicitly mention the work of <a href="https://www.linkedin.com/in/janwillemvanaalst" target="_blank" rel="noreferrer noopener">Jan-Willem van Aalst</a>
, who designed the <a href="http://opentopo.nl" target="_blank" rel="noreferrer noopener">OpenTopo-related maps</a>
provided on <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">Map5.nl</a>
using <a href="http://qgis.org" target="_blank" rel="noreferrer noopener">QGIS</a>
.</p>
<p>But this post is about JPEG and how/why I found it needs revival in the context of raster-data preparation and serving. I explicitly mention ‘preparation’ as JPEG was applied at two steps within the stack shown in Figure 2.</p>
<p><strong>JPEG Step 1 – Encoding in GeoTIFF</strong></p>
<p>Within the <a href="http://www.nlextract.nl" target="_blank" rel="noreferrer noopener">NLExtract project</a>
I was already preprocessing historical maps from PNG (with world files) to GeoTIFF. At some point I found out that GeoTIFF, as a container-image format, supports multiple image-encodings. Using JPEG encoding it appeared that the resulting GeoTIFF files were much smaller (about 50%) without hardly any loss in image quality. For the OpenTopo layers, I devised <a href="https://github.com/opengeogroep/NLExtract/blob/master/opentopo/bin/topotrans.sh" target="_blank" rel="noreferrer noopener">this shell-script</a>
to create GeoTIFFs ready to be served by any WMS-server. Without going into details of the script, this line, using <a href="http://www.gdal.org/gdal_translate.html" target="_blank" rel="noreferrer noopener">gdal_translate</a>
, does the actual JPEG-encoding:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>gdal_translate -b 1 -b 2 -b 3 -of GTiff
<span class="ln">2</span> -co TILED=YES
<span class="ln">3</span> -co PROFILE=Geotiff -co COMPRESS=JPEG
<span class="ln">4</span> -co JPEG_QUALITY=95
<span class="ln">5</span> -co PHOTOMETRIC=YCBCR -co BLOCKXSIZE=512 -co BLOCKYSIZE=512
<span class="ln">6</span> -a\_srs EPSG:28992 $src\_tif $dst_tif
</code></pre></div><p>Later on, I happily noticed, that others, like the great <a href="http://blog.cleverelephant.ca/2015/02/geotiff-compression-for-dummies.html" target="_blank" rel="noreferrer noopener">Paul Ramsey also advertised JPEG encoding in GeoTIFF</a>
. So JPEG lived up here.</p>
<p><strong>JPEG Step 2 – Encoding in WMS and Tiles</strong></p>
<p>Still within the rest of the geo-stack used for actual map serving with the MapServer/MapProxy combo I was still obeying the old rule to use PNG for serving non-arial/satellite images. I struggled and tested endless variations in configuration settings for PNG. My goal was to serve small tiles with just enough quality blazingly fast. Now PNG has many options, but broadly put one has to make a choice between PNG24 (24 bits) or PNG8 (8 bits, 256 colors). The latter uses a colormap encoding which for the rich color variations of the OpenTopo and coloured hillshading layers had quite noticeable degraded image quality. PNG24 on the other hand rendered great tiled images but with the penalty of significant tile-sizes. A <a href="http://en.wikipedia.org/wiki/Catch-22_%28logic%29" target="_blank" rel="noreferrer noopener">Catch-22 situation</a>
… Enter JPEG. Configuring MapProxy to serve JPEG-tiles gave much better results but needed some tweaking:</p>
<ul>
<li>use at least faktor 90 JPEG-compression (also in MapServer)</li>
<li>disable meta-tiling and buffering, i.e. request 256×256 JPEG maps from the MapServer source</li>
</ul>
<p>So some excerpts from the MapProxy config:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>opentopo_file_cache:
<span class="ln"> 2</span> grids: [geonovum_grid, opentopo_extent_grid]
<span class="ln"> 3</span> sources: [opentopo_wms]
<span class="ln"> 4</span> format: image/jpeg
<span class="ln"> 5</span> meta_buffer: 0
<span class="ln"> 6</span> meta_size: [1,1]
<span class="ln"> 7</span>
<span class="ln"> 8</span> opentopo_wms:
<span class="ln"> 9</span> type: wms
<span class="ln">10</span> req:
<span class="ln">11</span> url: http://ms.HOST_URL/go?
<span class="ln">12</span> layers: opentopo
<span class="ln">13</span> format: image/jpeg
<span class="ln">14</span> transparent: false
<span class="ln">15</span> coverage:
<span class="ln">16</span> bbox: [10000.000,299995.559,279997.956,625000.000]
<span class="ln">17</span> srs: 'EPSG:28992'
</code></pre></div><p>This gave optimal results. JPEG tiles were around 4.5 times as small as PNG24. See the results in Figure 1 above. For the <a href="http://app.map5.nl/nltopo/?base_layer=relief_struct" target="_blank" rel="noreferrer noopener">Hillshading layer</a>
the differences were striking especially when zoomed-in. See Figure 3 below.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/relief_struct-png-vs-jpeg.png" alt="Figure 3. Tiles and filesizes for different image encodings. Click image for full picture."></p>
<p>So my choice was to settle for JPEG for the topographic and hillshading maps. You can browse all Map5.nl layers in the <a href="http://app.map5.nl/nltopo/" target="_blank">NLTopo App</a>.</p>
<p>So yes, JPEG seems the most optimal for these type of map-layers, but am I missing something? Some proponed: <em>“Yes, but JPEG has no transparency nor alpha-channel”</em>. Hmm, true, but does this matter in most modern web-clients like <a href="http://openlayers.org" target="_blank" rel="noreferrer noopener">OpenLayers</a>
or <a href="http://leaflet.org" target="_blank" rel="noreferrer noopener">Leaflet</a>
? From what I observed, JPEG-layers will happily obey opacity-settings in these web-clients. For example, Figure 4 below shows the national Dutch Topographic map overlayed with the Map5.nl hillshading layer.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/kadaster-top25-relief-struct.jpg" alt="Figure 4 – Dutch 1:25000 raster map transparently overlayed with Map5.nl JPEG hillshading layer"></p>
<p>So what to conclude? Basically the title of this post should say it. Further I would again like to acknowledge Jan-Willem van Aalst for his outstanding work on OpenTopo maps and Frank Steggink for making the basic hillshading map from the free Dutch Lidar- pointcloud-data (<a href="http://www.ahn.nl" target="_blank" rel="noreferrer noopener">AHN2</a>
). And further the developers of MapServer and MapProxy, what an awesome combo. Even without pre-tiling maps are served blazingly fast! I am really fond of the Hillshading map. The Netherlands, known to be “flatland”, can now reveal also its past. See for example figure 5 below, a <a href="http://app.map5.nl/nltopo/#rd/relief_struct/12/175799.7/477431.2" target="_blank" rel="noreferrer noopener">Roman Fort</a>
from about 2000 years ago!</p>
<div class="">
<a href="http://app.map5.nl/nltopo/#rd/relief_struct/12/175799.7/477431.2">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/05/relief_struct_speuld_romfort.png"
alt="Figure 5 – Contours from a Roman Fort near Speuld" />
</a>
</div>
-
Tales from Topographic Lowlands
https://justobjects.nl/tales-from-topographic-lowlands/
Fri, 08 May 2015 00:20:28 +0000
https://justobjects.nl/tales-from-topographic-lowlands/
<p>As far as Open geo-data is concerned, the Netherlands goes through prosperous times: country-wide datasets for detailed topography (<a href="http://www.kadaster.nl/web/Themas/Registraties/brt.htm" target="_blank" rel="noreferrer noopener">BRT</a>
, <a href="http://www.kadaster.nl/web/Themas/Registraties/bgt.htm" target="_blank" rel="noreferrer noopener">BGT</a>
), buildings and addresses (<a href="https://www.kadaster.nl/bag" target="_blank" rel="noreferrer noopener">BAG</a>
), 0.5m elevation data (<a href="http://www.ahn.nl/" target="_blank" rel="noreferrer noopener">AHN2</a>
) and many more are available for download. Even <a href="https://www.pdok.nl/nl/producten/pdok-downloads/basis-registratie-topografie/top10nl-3d" target="_blank" rel="noreferrer noopener">3D-versions</a>
for several of these datasets are upcoming. These and other nuggets can be found and downloaded via our national geo-portal: <a href="https://www.pdok.nl" target="_blank" rel="noreferrer noopener">PDOK</a>
. The datasets are available also via OGC web services like WMS, WMTS, WCS and WFS, yes thanks also to <a href="http://inspire.ec.europa.eu/" target="_blank" rel="noreferrer noopener">INSPIRE</a>
. Having access to the raw data, in XML, GML and Lidar/LAS has stirred creativity and productivity within the Dutch OSGeo and OpenStreetMap communities. Thinking of the endless possibilities for combining these open datasets boggles the mind!</p>
<p>Without a fixed plan three activities have been emerging:</p>
<ol>
<li>Refinement: converting raw datasets to formats like PostGIS and GeoTIFF (ETL)</li>
<li>Creation: using and combining these refined datasets to maps and imports</li>
<li>Dissemination: making the results from 2) available as data and web-services</li>
</ol>
<p>Looks like a subsidized EU-project, but in reality this work has been done by enthusiastic, but unpaid volunteers. I will try to give some illustration below.</p>
<p><strong>1. Refinement</strong></p>
<p>The <a href="http://nlextract.nl" target="_blank" rel="noreferrer noopener">NLExtract</a>
project has emerged to provide not only tools, but also downloads for processable formats of the above datasets. For example the Dutch Addresses and Buildings dataset (BAG) is a quite involved semi-GML dataset monthly downloadable as a 1.8 GB zipfile. The NLExtract project provides a <a href="https://github.com/opengeogroep/NLExtract/tree/master/bag" target="_blank" rel="noreferrer noopener">tool</a>
to convert this dataset to a PostGIS schema and CSV (for addresses) and make these available for download at <a href="http://data.nlextract.nl/bag" target="_blank" rel="noreferrer noopener">data.nlextract.nl/bag</a>
. Also the Dutch Base Topography (BRT, Top10NL) is <a href="https://github.com/opengeogroep/NLExtract/tree/master/top10nl/etl" target="_blank" rel="noreferrer noopener">processed</a>
in the same fashion. Upcoming are GeoTIFFs for the Dutch elevation (yes we have some heights!) Lidar/LAS data.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/nlextract.png" alt=" "></p>
<p><strong>2. Creation</strong></p>
<p>This phase is more interesting, since here true creativity is happening. Two examples to illustrate.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/osm-bag.png" alt=" "></p>
<p>The Addresses and Buildings dataset was immediately of interest to the <a href="http://www.openstreetmap.nl" target="_blank" rel="noreferrer noopener">Dutch OpenStreetMap community</a>
. After much debate and careful planning a <a href="http://wiki.openstreetmap.org/wiki/BAGimport" target="_blank" rel="noreferrer noopener">structured import</a>
has been performed. This has enriched the Dutch OSM data enormously.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/opentopo-1600-otterlo.png" alt=" "></p>
<p>Now we get to even more visible results. <a href="https://www.linkedin.com/in/janwillemvanaalst" target="_blank" rel="noreferrer noopener">Jan-Willem van Aalst</a>
is a Dutch mapmaker who has combined “the best of” several Dutch open datasets and OpenStreetMap data into a compelling series of beautiful maps under the name <a href="http://opentopo.nl/" target="_blank" rel="noreferrer noopener">OpenTopo</a>
. His main tool is <a href="http://qgis.org" target="_blank" rel="noreferrer noopener">QGIS</a>
. You can see results on <a href="http://opentopo.nl/" target="_blank" rel="noreferrer noopener">his website</a>
. But the story does not stop there. Thanks to the work of Frank Steggink who has processed raw Dutch elevation data (AHN2) into GeoTIFFs, Jan-Willem is also producing a very detailed (up to 0.625 m/px about 1:5000 scale) hillshade-map of our entire country.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/relief-struct-500x300-1600.jpg" alt=" "></p>
<p>In addition OpenTopo topography has been overlayed on the free Dutch areal imagery from PDOK tiles. Who needs Google Maps ;)?</p>
<p><img src="https://justobjects.nl/uploads/2015/05/openlufo-500-300.jpg" alt=" "></p>
<p><strong>3. Dissemination</strong></p>
<p>So how are all these fine results shared? The OpenStreetMap database now contains all Dutch Addresses and Buildings and is always <a href="http://www.openstreetmap.org/#map=18/52.10386/5.04983" target="_blank" rel="noreferrer noopener">visible on OpenStreetMap</a>
.</p>
<p>The OpenTopo maps of Jan-Willem are exported as GeoTIFFs via <a href="http;//data.nlextract.nl/opentopo" target="_blank" rel="noreferrer noopener">data.nlextract.nl/opentopo</a>
. But many users may need an online map, or may want a standard web/tiling service to integrate maps into their applications or use them in their GIS.</p>
<p>To cater for the latter I have launched the <a href="https://map5.nl" target="_blank" rel="noreferrer noopener">Map5.nl</a>
platform: to provide Dutch topography as standard web-services. Using a geo-stack built with <a href="http://mapserver.org" target="_blank" rel="noreferrer noopener">MapServer</a>
and <a href="http://mapproxy.org" target="_blank" rel="noreferrer noopener">MapProxy</a>
, the <a href="https://map5.nl/kaarten.html" target="_blank" rel="noreferrer noopener">OpenTopo and several other Dutch topomaps</a>
, also historical, are now available in most of the common web mapping protocols: WMTS, TMS, Google/OSM tiling and even WMS.</p>
<div class="">
<a href="https://map5.nl">
<img class="lazy"
src="https://justobjects.nl/uploads/2015/05/map5.nl_.png"
alt="" />
</a>
</div>
<p>I have been thinking hard how to offer these services with respect to payment. Similar to what Andy Allan with <a href="http://www.thunderforest.com/" target="_blank" rel="noreferrer noopener">www.thunderforest.com</a>
provides for OSM maps, Map5.nl offers a free and a paid service. Part of the subscription fees will flow back into the OpenTopo development.</p>
<p><img src="https://justobjects.nl/uploads/2015/05/nltopo.png" alt=" "></p>
<p>The software and data behind the map5 services are still free. But to develop and maintain such a service and infrastructure has imminent costs. I hope that organizations within the Netherlands find the map5 services useful and are willing to pay the relatively small yearly fee to expose these maps on their websites publicly. The free service has some small advertisements now and then. After some stable amount of subscriptions these may also go away. You can view the current maps on the <a href="http://app.map5.nl/nltopo/" target="_blank" rel="noreferrer noopener">NLTopo App</a>
, which should work on most current (HTML5) mobiles/tablets.</p>
-
3D Geospatial – Open Standards – v0
https://justobjects.nl/3d-geospatial-open-standards-v0/
Thu, 29 Jan 2015 14:06:51 +0000
https://justobjects.nl/3d-geospatial-open-standards-v0/
<p>This is my evolving overview of 3D Geospatial Open Standards with a focus on web technology: services, HTML5, WebGL. These are notes I keep within <a href="http://evernote.com" target="_blank" rel="noreferrer noopener">Evernote</a>
as part of the 3D Geospatial Notebook whose public link is: <a href="https://www.evernote.com/pub/justb4ever/3d-geospatial">https://www.evernote.com/pub/justb4ever/3d-geospatial</a>. For sure, resources are missing, let me know. These notes are synced from Evernote using <a href="http://zapier.com" target="_blank" rel="noreferrer noopener">Zapier</a>
, so formatting is a bit messy, sorry for that, though I completely revised this blog on Jan 30, 2015.</p>
<p>The presentations by <a href="https://twitter.com/emmanuel_belo" target="_blank" rel="noreferrer noopener">Emanuel Belo</a>
(<a href="http://www.camptocamp.com/en/" target="_blank" rel="noreferrer noopener">Camp2Camp</a>
, links below) are an excellent start to get a global overview. He mentions the following categories of 3D standards and their organizations:</p>
<p><em><strong>Geo: OGC</strong></em></p>
<ul>
<li>3D Portrayal Services (Proposals: WVS WMS-Like & W3DS WFS-Like) (Just: Now 3DPS)</li>
<li>KML – XML/COLLADA</li>
<li>CityGML – representation, storage, and exchange</li>
<li>CZML (AGI/Cesium) ?</li>
</ul>
<p><em><strong>Geo: OSGeo</strong></em></p>
<ul>
<li>TMS (Cesium Terrain Server z.B)</li>
</ul>
<p> </p>
<p><em><strong>Web: Web3d</strong></em></p>
<ul>
<li>X3D – Extensible 3D Graphics</li>
</ul>
<p><em><strong>Graphics: Khronos Group</strong></em></p>
<ul>
<li>COLLADA – eXchange / interoperability</li>
<li>glTF – graphic language Transmission Format</li>
</ul>
<p> </p>
<p><em><strong>Other</strong></em></p>
<ul>
<li>TopoJSON</li>
</ul>
<p>In the short time I am spending within the 3D domain I think another categorization could be 3D Web Services Standards and 3D Content/Format Standards. A lot of focus is usually on the latter. Within 3D Content/Format Standards my view is that there are the sub-categories based on stages within a 3D rendering pipeline, i.e. from raw source data like CityGML up to 3D-scene encodings like COLLADA and glTF. I see “Terrain” as a separate content category with encodings like height maps and TINs.</p>
<p>I think/hope that 3D Web Services will gain importance: services to request Content in various encodings plus Terrain data. Techniques like tiling (TMS/WMTS) are already very common within the geospatial world. With the latest developments in vector-tiling I see much potential, especially for terrain and textures (raster) draped on a terrain. Streaming as in video streaming, is not common within the geospatial world but may become of use especially in HTML5 apps (via web sockets?).</p>
<p><strong>General</strong></p>
<p>The following presentations by Emanuel Belo from Camp2Camp hit the nail on the head. The content lives up to the title.</p>
<p>Two versions available, video and slides from FOSS4G 2013, Nottingham (missed that unfortunately) and a later 2014 version.</p>
<p><a href="https://www.youtube.com/watch?v=9CgU0zs8DlU">https://www.youtube.com/watch?v=9CgU0zs8DlU</a> 3D Web Services And Models For The Web: Where Do We Stand? Belo FOSS4G13</p>
<p><a href="http://www.slideshare.net/camptocamp/3-d-web-services-and-models-for-the-web">http://www.slideshare.net/camptocamp/3-d-web-services-and-models-for-the-web</a> Belo – ditto slides [1]</p>
<p><a href="http://www.geospatialworldforum.org/2014/presentation/geo3d/Emmanuel%20Belo%20M.pdf">http://www.geospatialworldforum.org/2014/presentation/geo3d/Emmanuel%20Belo%20M.pdf</a> Belo – ditto 2014 version slides</p>
<p><strong>Open Geospatial Consortium – OGC</strong></p>
<p>For open geospatial standards the Open Geospatial Consortium, OGC, is the first standards body to look at. Their standards<br>
are often aligned with corresponding ISO standards and more recently OGC started collaborating more closely with W3C.</p>
<p>At least two Standards Working Groups (WSGs) are dedicated to 3D. The more established 3D Information Management (3DIM, CityGML!)) and the probably lesser known 3D Portrayal SWG.</p>
<p>It should be noted and credited that most of the OGC 3D standards came out of the German <a href="http://www.gdi-3d.de" target="_blank" rel="noreferrer noopener">GDI-DE</a>
project.<br>
At this moment (Jan 2015) two main OGC standards are of importance for 3D: CityGML and the draft standard for 3D Portrayal web services: 3DPS.</p>
<p><em><strong>OGC – 3D Information Modeling – CityGML</strong></em><br>
<em>“CityGML is a common information model and XML-based encoding for the representation, storage, and exchange of virtual 3D city and landscape models. CityGML provides a standard model and mechanism for describing 3D objects with respect to their geometry, topology, semantics and appearance, and defines five different levels of detail. Included are also generalization hierarchies between thematic classes, aggregations, relations between objects, and spatial properties. CityGML is highly scalable and datasets can include different urban entities supporting the general trend toward modeling not only individual buildings but also whole sites, districts, cities, regions, and countries.”</em><br>
<a href="http://www.citygml.org">http://www.citygml.org</a></p>
<p>SWG: <a href="http://www.opengeospatial.org/projects/groups/3dimwg">http://www.opengeospatial.org/projects/groups/3dimwg</a> – 3D Information Management (3DIM) Domain Working Group<br>
CityGML: <a href="http://www.opengeospatial.org/standards/citygml">http://www.opengeospatial.org/standards/citygml</a></p>
<p>OGC candidate 3D Standards | GeoConnexion<br>
<a href="http://www.geoconnexion.com/articles/ogc-candidate-3d-standards">http://www.geoconnexion.com/articles/ogc-candidate-3d-standards</a><br>
<a href="http://www.w3ds.org/doku.php">http://www.w3ds.org/doku.php</a></p>
<p><em><strong>OGC – 3D Web Services – 3D Portrayal</strong></em></p>
<p>SWG: <a href="http://www.opengeospatial.org/projects/groups/3dpswg">http://www.opengeospatial.org/projects/groups/3dpswg</a> – 3D Portrayal SWG.<br>
Note the W3DS, WVS, WTS and WPVS are now all obsolete since they have been merged into a single web service standard: the 3D Portrayal Service (3DPS).</p>
<p>A draft version 1.0.0 is now (January 2015) out for public comment within OGC:<br>
<a href="http://www.opengeospatial.org/standards/requests/130">http://www.opengeospatial.org/standards/requests/130</a></p>
<p><em>“The 3D portrayal standard (3DPS) is an OGC service implementation specification targeting the delivery of 3D portrayals in an interoperable fashion. … When client and server(s) involved share a common set of capabilities, it becomes possible to view and analyze 3D geoinformation from diverse sources in a combined manner. …. The 3DPS combines the essential parts of the suggested W3DS and WVS into one common interface.”</em> and</p>
<p><em>“ The Open Geospatial Consortium (OGC) and the Web3D Consortium have both been working to address the need for interoperability, as well as the content challenges of volume, access speed, and diversity of devices. The Web3D Consortium has focused on open standards for real-time 3D visualization, including streaming, and their members developed a Geospatial Component extension for X3D. The OGC has focused on developing a service interface to provide interoperable access to 3D geospatial data servers. In 2012, a group of OGC members, building on work done in both organizations, completed the 3D Portrayal Interoperability Experiment (3DPIE) to develop and evaluate best practices for 3D portrayal services.”</em></p>
<p><em>“The candidate OGC 3D Portrayal Service Standard is designed to support both client and server side rendering. For client-side rendering, the client requests a 3D model from the server. The server extracts the requested model from the 3D geodata server and generates a 3D scene graph including geometry and textures. Depending on the server’s capabilities, data formats such as X3D, KML and COLLADA can be used to retrieve the scene graph. The rendering of the scene is done on the client side. In a web client, X3DOM and/or XML3D can be used to integrate the scene into an immersive HTML5 experience. However, there are no fixed format requirements, opening the service for other technologies such as JSON-based glTF. For server side rendering, the client passes the requested content and view parameters to the server. The server then generates layered image depictions of the 3D environment for display on the client. In either scenario, the client’s user can query and navigate through the 3D content.”</em></p>
<p>Link to 3DPS v1.0.0 for public review on jan 29, 2015:</p>
<p>Public comment request: <a href="http://www.opengeospatial.org/standards/requests/130" target="_blank"><a href="http://www.opengeospatial.org/standards/requests/130">http://www.opengeospatial.org/standards/requests/130</a></a><br>
Direct download 3DPS 1.0.0 draft: <a href="https://portal.opengeospatial.org/files/61884" target="_blank"><a href="https://portal.opengeospatial.org/files/61884">https://portal.opengeospatial.org/files/61884</a></a></p>
<p>For history reasons some notes kept on W3DS and WVS below.</p>
<p><em>A <strong>Web 3D Service (W3DS)</strong> is a portrayal service for three-dimensional geodata such as landscape models, city models, textured building models, vegetation objects, and streetfurniture. Geodata is delivered as scenes that are comprised of display elements, optimized for efficient real time rendering at high frame rates. 3D Scenes can be interactively displayed and explored by internet browsers with 3D plug-ins, or loaded into virtual globe applications.</em></p>
<p><em>The <strong>Web View Service (WVS)</strong> is a portrayal service for three-dimensional geodata suchas landscape models, city models, vegetation models, or transportation infrastructure models. A WVS server mainly provides 2D image representing a 3D view on a scene constructed from 3D geodata that is integrated and visualized by the WVS server. In addition to these color images of a 3D scene, a WVS server can advertise and deliver complementary image layers that include geometrical or thematic information: e.g., depth layers, surface normal data, or object id information.</em></p>
<p><em><strong>OGC – 3D Portrayal Interoperability Experiment (3DPIE)</strong></em><br>
The OGC …. <em>“3D Portrayal Interoperability Experiment (IE) successfully tested and demonstrated different mechanisms for the portrayal, delivery, and exploitation of 3D geodata based on open standards-based formats and services. …approaches for service-based 3D portrayal based on thedrafts for the candidate standards for 3D portrayal, Web 3D Service (W3DS) and WebView Service (WVS).”</em></p>
<p>Page: <a href="http://www.opengeospatial.org/projects/initiatives/3dpie">http://www.opengeospatial.org/projects/initiatives/3dpie</a><br>
Report: <a href="https://portal.opengeospatial.org/files/?artifact_id=49068">https://portal.opengeospatial.org/files/?artifact_id=49068</a></p>
<p>NB this ultimately lead to the development of the 3D Portrayal Standard which merged W3DS and WVS.</p>
<p><strong>Open Source Geospatial Foundation – OSGeo</strong><br>
Although <a href="https://osgeo.org" target="_blank" rel="noreferrer noopener">OSGeo.org</a>
is not a standards body, for years the Tiled Map Service TMS spec is intensively used, probably more than its OGC-counterpart WMTS…<br>
<a href="http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification">http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification</a></p>
<p>TMS is finding its use in the 3D landscape not just for tiles containing 2D raster maps to be draped over a terrain, but also more and more as a container for height and (compressed) vector tiles. This is an important development, see e.g. Cesium Terrain Server and its two format-encodings for terrain-tiles:<br>
<a href="https://github.com/AnalyticalGraphicsInc/cesium/wiki/Cesium-Terrain-Server">https://github.com/AnalyticalGraphicsInc/cesium/wiki/Cesium-Terrain-Server</a></p>
<p><strong>X3D – Web3D</strong><br>
<em>“Extensible 3D (X3D) Graphics and Humanoid Animation (H-Anim) include a coordinated set of steadily evolving ISO standards. “</em></p>
<p><a href="http://www.web3d.org/standards">http://www.web3d.org/standards</a> (mostly X3D, equiv with ISO/IEC 19775 (architecture and abstract capabilities), 19776 (encodings), and 19777 (API)))</p>
<p><strong>X3DOM</strong><br>
<em>“X3DOM is an adaption of the X3D standard to (X)HTML, ensuring declarative 3D can</em><br>
<em>be used inside standards-compliant browser. It aims to support a large browser base and</em><br>
<em>decent X3D feature coverage, while working towards a common declarative 3D standard</em><br>
<em>in the Declarative 3D community WG at the W3C. The reference implementation is</em><br>
<em>maintained by Fraunhofer IGD and available under the open-source license MIT.”</em><br>
<a href="http://www.x3dom.org/">http://www.x3dom.org/</a></p>
<p><strong>WebGL</strong><br>
<a href="https://www.khronos.org/webgl/">https://www.khronos.org/webgl/</a><br>
<em>“WebGL is a standard for programming in 3D with the browser as platform. The final</em><br>
<em>specification of the standard was published in 2010 and is defined by the Khronos Group,</em><br>
<em>a consortium which is also in charge of Open GL, Open CL and OpenGL ES (embedded</em><br>
<em>systems). WebGL provide a context into HTML5 canvas that is 3D Computer Graphics</em><br>
<em>capable without plug-in.”</em></p>
<p><strong>Cesium</strong><br>
Although <a href="http://cesiumjs.org" target="_blank" rel="noreferrer noopener">CesiumJS</a>
is mainly a client JavaScript framework based on WebGL and Dojo, some very useful standards are originating from the project.</p>
<p><a href="http://cesiumjs.org/data-and-assets/terrain/formats/quantized-mesh-1.0.html">http://cesiumjs.org/data-and-assets/terrain/formats/quantized-mesh-1.0.html</a> quantized mesh via TMS – for terrain services based on TINs.</p>
<p><a href="https://github.com/AnalyticalGraphicsInc/cesium/wiki/CZML-Guide">https://github.com/AnalyticalGraphicsInc/cesium/wiki/CZML-Guide</a> – CZML is a JSON schema for describing a time-dynamic graphical scene</p>
<p>NEW (august 2015) <em><strong>“3D Tiles”</strong></em> : <a href="https://cesiumjs.org/2015/08/10/Introducing-3D-Tiles">https://cesiumjs.org/2015/08/10/Introducing-3D-Tiles</a>. <em>“3D Tiles are an <a href="https://github.com/AnalyticalGraphicsInc/3d-tiles" target="_blank" rel="noreferrer noopener">open specification</a>
for streaming massive heterogeneous 3D geospatial datasets….3D Tiles define a spatial data structure and a set of tile formats designed for 3D and optimized for streaming and rendering. Tiles for 3D models use <a href="https://www.khronos.org/gltf" target="_blank" rel="noreferrer noopener">glTF</a>
, the WebGL runtime asset format developed by Khronos, which the Cesium team heavily contributes to….”</em></p>
<p><strong>Collada</strong><br>
<em>“COLLADA™ defines an XML-based schema to make it easy to transport 3D assets between applications – enabling diverse 3D authoring and content processing tools to be combined into a production pipeline. The intermediate language provides comprehensive encoding of visual scenes including: geometry, shaders and effects, physics, animation, kinematics, and even multiple version representations of the same asset.COLLADA FX enables leading 3D authoring tools to work effectively together to create shader and effects applications and assets to be authored and packaged using OpenGL® Shading Language, Cg, CgFX, and DirectX® FX.”</em></p>
<p><a href="https://www.khronos.org/collada/">https://www.khronos.org/collada/</a></p>
<p><strong>KML</strong><br>
Originally from Keyhole/Google, but since years an OGC Standard. May embed Collada.</p>
<p><em>“Keyhole Markup Language (KML) is an XML notation for expressing geographic annotation and visualization within Internet-based, two-dimensional maps and three-dimensional Earth browsers. KML was developed for use with Google Earth, which was originally named Keyhole Earth Viewer. It was created by Keyhole, Inc, which was acquired by Google in 2004. KML became an international standard of the Open Geospatial Consortium in 2008.[1][2] Google Earth was the first program able to view and graphically edit KML files. “</em><br>
Source: <a href="http://en.wikipedia.org/wiki/Keyhole_Markup_Language">http://en.wikipedia.org/wiki/Keyhole_Markup_Language</a></p>
<p>OGC Standard: <a href="http://www.opengeospatial.org/standards/kml">http://www.opengeospatial.org/standards/kml</a></p>
<p><em><strong>3D in KML:</strong></em><br>
<a href="https://developers.google.com/kml/documentation/models">https://developers.google.com/kml/documentation/models</a><br>
<a href="http://blog.thematicmapping.org/2008/04/proportional-3d-collada-objects-in-kml.html">http://blog.thematicmapping.org/2008/04/proportional-3d-collada-objects-in-kml.html</a><br>
<a href="http://blog.thematicmapping.org/2008/04/drawing-3d-bars-in-kml.html">http://blog.thematicmapping.org/2008/04/drawing-3d-bars-in-kml.html</a></p>
<p><strong>glTF</strong><br>
Final stage OpenGL Transmission Format to enable rapid delivery and loading of 3D content by WebGL, OpenGL, and OpenGL ES APIs.</p>
<p><em>“The “glTF” project aims to define a final stage OpenGL Transmission Format to enable rapid delivery and loading of 3D content by WebGL, OpenGL, and OpenGL ES APIs. glTF together with COLLADA comprise a standards-based content pipeline for rich 3D web and mobile applications. glTF Specification is a work-in-progress from the COLLADA Working Group; it is not an official Khronos-ratified specification yet. It is incomplete and subject to change. The draft specification and related converters and loaders are available on github.”</em></p>
<p><a href="https://www.khronos.org/" target="_blank"><a href="https://www.khronos.org/">https://www.khronos.org/</a></a><br>
<a href="https://www.khronos.org/gltf/" target="_blank"><a href="https://www.khronos.org/gltf/">https://www.khronos.org/gltf/</a></a></p>
<p><strong>TopoJSON</strong></p>
<p><em>“TopoJSON is an extension of GeoJSON that encodes topology. Rather than representing geometries discretely, geometries in TopoJSON files are stitched together from shared line segments called arcs. TopoJSON eliminates redundancy, offering much more compact representations of geometry than with GeoJSON; typical TopoJSON files are 80% smaller than their GeoJSON equivalents. In addition, TopoJSON facilitates applications that use topology, such as topology-preserving shape simplification, automatic map coloring, and cartograms.”</em> (i.e. compact TIN-representations!).<br>
<a href="https://github.com/mbostock/topojson">https://github.com/mbostock/topojson</a></p>
<p><strong>Germany: SIG 3D</strong><br>
Germany-based 3D interest group. Started already very early (2002) with 3D geospatial and has a long history. Many useful resources for both standards and implementations can be found via their websites.</p>
<p><em>“Three-dimensional models of cities and regions play an important role in major applications of architecture, urban planning, surveying, mobile telecommunication or facility management. In the environmental sector 3D city models enable the simulation of noise and exhaust emissions as well as predictions on city climate change affecting a city. Concerning disaster situations like floods, 3D landscape models can help to analyse the affected areas and buildings.”</em></p>
<p><em>“Since the beginning of 2010 SIG 3D is part of the German Spatial Data Infrastructure (GDI-DE) and coordinates this context the national and international network of 3D activities.”</em></p>
<p><a href="http://www.sig3d.org/">http://www.sig3d.org/</a><br>
Several standards like OGC CityGML W3DS, later 3DPS had their origins here.</p>
<p><strong>The Netherlands: Doorbraak 3D (Breakthrough 3D)</strong><br>
3D initiative within The Netherlands as a follow-up on earlier successful 3D projects like the 3D-Pilot: <a href="http://www.geonovum.nl/documenten/rapport-eindrapportages-3d-pilot-nl-eerste-fase">http://www.geonovum.nl/documenten/rapport-eindrapportages-3d-pilot-nl-eerste-fase</a></p>
<p>As a follow-up (2011) of the 3D-Pilot the public standard CityGML-IMGeo was developed. Basically this standardizes detailed smallscale/detailed topography using and extending CityGML: <a href="http://www.geonovum.nl/wegwijzer/standaarden/gegevenscatalogus-imgeo-versie-211">http://www.geonovum.nl/wegwijzer/standaarden/gegevenscatalogus-imgeo-versie-211</a></p>
<p>History of 3D geo in The Netherlands: <a href="http://www.geonovum.nl/onderwerpen/3d-geo-informatie/historie-3d-geo-informatie-nl">http://www.geonovum.nl/onderwerpen/3d-geo-informatie/historie-3d-geo-informatie-nl</a></p>
<p>Now (Jan 2015) starting up a next phase called “3D Doorbraak”, with a recent working conference for establishing the 3D roadmap for The Netherlands: <a href="http://www.geonovum.nl/nieuws/werkconferentie-voor-opstellen-roadmap-3d-doorbraak">http://www.geonovum.nl/nieuws/werkconferentie-voor-opstellen-roadmap-3d-doorbraak</a></p>
<p>The choice of standards plays an important part.</p>
<p>See also news items via Twitter (#doorbraak3D): <a href="https://twitter.com/search?q=%23doorbraak3D&src=typd">https://twitter.com/search?q=%23doorbraak3D&src=typd</a></p>
<p><strong>BIM – IFC</strong></p>
<p>Now that geospatial information is obtaining more detail and moving into 3D, integration with standards within the Building and infrastructure industry is one of the most obvious use-cases. Building Information Modelling or BIM is a huge standardization effort within the Building and infrastructure domain.</p>
<p>Geoff ,”Between The Poles”, Zeiss blogs regularly on BIM and Geospatial: <a href="http://geospatial.blogs.com/geospatial/bim/">http://geospatial.blogs.com/geospatial/bim/</a></p>
<p><em><strong>“Building information modeling</strong> (<strong>BIM</strong>) is a process involving the generation and management of digital representations of physical and functional characteristics of places. <strong>Building information models</strong> (<strong>BIMs</strong>) are files (often but not always in proprietary formats and containing proprietary data) which can be exchanged or networked to support decision-making about a place. Current BIM software is used by individuals, businesses and government agencies who plan, design, construct, operate and maintain diverse physical infrastructures, such as water, wastewater, electricity, gas, refuse and communication utilities, roads, bridges and ports, houses, apartments, schools and shops, offices, factories, warehouses and prisons.”</em></p>
<p>Source: <a href="http://en.wikipedia.org/wiki/Building_information_modeling">http://en.wikipedia.org/wiki/Building_information_modeling</a></p>
<p>For information exchange the Industry Foundation Classes are put forward.</p>
<p>“The <em>Industry Foundation Classes (IFC) data model is intended to describe building and construction industry data.</em></p>
<p><em>It is a platform neutral, open file format specification that is not controlled by a single vendor or group of vendors. It is an object-based file format with a data model developed bybuildingSMART (formerly the International Alliance for Interoperability, IAI) to facilitate interoperability in the architecture, engineering and construction (AEC) industry, and is a commonly used collaboration format in Building information modeling (BIM) based projects. The IFC model specification is open and available.[1] It is registered by ISO and is an official International Standard ISO 16739:2013.”</em><br>
Source: <a href="http://en.wikipedia.org/wiki/Industry_Foundation_Classes">http://en.wikipedia.org/wiki/Industry_Foundation_Classes</a></p>
<p>More: <a href="http://www.ifcwiki.org/index.php/Main_Page">http://www.ifcwiki.org/index.php/Main_Page</a></p>
-
3D Geospatial – Free and Open Source Software v0
https://justobjects.nl/3d-geospatial-foss-v0/
Wed, 28 Jan 2015 11:52:34 +0000
https://justobjects.nl/3d-geospatial-foss-v0/
<p>This is my list of FOSS products for 3D Geospatial with a focus on web technology/WebGL and Open Standards. These are notes I keep within Evernote as part of the 3D Geospatial Notebook whose public link is: <a href="https://www.evernote.com/pub/justb4ever/3d-geospatial">https://www.evernote.com/pub/justb4ever/3d-geospatial</a>. This is also the first experiment to auto sync Evernote to WordPress using Zapier, so beware of any glitches in formatting! I am sure I have missed many other great products. Let me know!</p>
<h3 id="oslandia">OSLandia</h3>
<p>OSLandia from Paris, France is one of the most active companies contributing to 3D Geospatial FOSS.
<a href="http://www.oslandia.com/">http://www.oslandia.com/</a></p>
<p><a href="http://www.oslandia.com/postgis-3d-foss4g-video-and-workshop-en.html">http://www.oslandia.com/postgis-3d-foss4g-video-and-workshop-en.html</a></p>
<p>– The latest PostGIS and QGIS 3D enhancements presented at <a href="http://2013.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G</a>
by Oslandia – GIS goes 3D : an OpenSource stack – Olivier Courtin – FOSS4G 2014</p>
<p><a href="http://vimeo.com/106846660">http://vimeo.com/106846660</a></p>
<h3 id="cesium">Cesium</h3>
<p>– General</p>
<p><em>“Cesium is a JavaScript library for creating 3D globes and 2D maps in a web browser without a plugin. It uses WebGL for hardware-accelerated graphics, and is cross-platform, cross-browser, and tuned for dynamic-data visualization. Cesium is open source under the Apache 2.0 license. It is free for commercial and non-commercial use.”</em></p>
<p><img src="https://justobjects.nl/uploads/2015/01/cesium.png" alt=" "></p>
<p><a href="http://cesiumjs.org/">http://cesiumjs.org/</a><br>
<a href="http://cesiumjs.org/presentations/Rendering%20the%20Whole%20Wide%20World%20on%20the%20World%20Wide%20Web.pdf">http://cesiumjs.org/presentations/Rendering%20the%20Whole%20Wide%20World%20on%20the%20World%20Wide%20Web.pdf</a> great presentation</p>
<p>– Cesium Terrain Service Preparation<br>
<a href="https://github.com/giohappy/gdal2cesium">https://github.com/giohappy/gdal2cesium</a><br>
<a href="https://github.com/kaktus40/Cesium-GeoserverTerrainProvider">https://github.com/kaktus40/Cesium-GeoserverTerrainProvider</a></p>
<p>– Creating 3D terrains with Cesium – Bjørn Sandvik<br>
<a href="http://blog.thematicmapping.org/2014/10/3d-terrains-with-cesium.html">http://blog.thematicmapping.org/2014/10/3d-terrains-with-cesium.html</a></p>
<p>– Migrating from Google Earth to Cesium<br>
<em>This is a guest post by Greg Angevine, Founder of <a href="http://cubecities.com/" target="_blank" rel="noreferrer noopener">Cube Cities Inc</a>
. His company has used the Google Earth plugin for years and has recently built impressive work with Cesium (like <a href="https://twitter.com/CesiumJS/status/552845347484364800" target="_blank" rel="noreferrer noopener">this</a>
)</em><br>
<a href="http://cesiumjs.org/2015/01/27/Migrating-from-Earth-to-Cesium/">http://cesiumjs.org/2015/01/27/Migrating-from-Earth-to-Cesium/</a></p>
<p>– Cesium and OpenLayers3<br>
Cesium has a real focus on (OGC) standards and integrating with software implementing OGC standards. I heard that the Cesium-folks are even proposing additions to OGC 3D standards (CZML? or the very compact terrain tiling using quantized mesh?). That would be great in concert with/as payload for the upcoming 3DPS (3D Portrayal Service) standard.</p>
<p><img src="https://justobjects.nl/uploads/2015/01/cesium-ol3.png" alt=" "></p>
<p><a href="http://openlayers.org/ol3-cesium/">http://openlayers.org/ol3-cesium/</a></p>
<p>– My Cesium experiments with Dutch OpenTopo tiles and Top10NL-3D Vector</p>
<p><img src="https://justobjects.nl/uploads/2015/01/cesium-just.png" alt=" "></p>
<p><a href="http://app.nlextract.nl/3d/">http://app.nlextract.nl/3d/</a></p>
<p>– My Cesium experiments with Cesium-OpenLayers3 Integration using Dutch Topo Top10NL-3D Vector</p>
<p><img src="https://justobjects.nl/uploads/2015/01/cesium-ol3-just.png" alt=" "></p>
<p><a href="http://app.nlextract.nl/3d/">http://app.nlextract.nl/3d/</a></p>
<h3 id="tangram">Tangram</h3>
<p>From the folks from <a href="http://mapzen.com" target="_blank" rel="noreferrer noopener">MapZen</a>
who make much more cool stuff: <em>“Tangram is a library for rendering 2D & 3D maps with WebGL, using GeoJSON/TopoJSON or binary vector tiles.”</em></p>
<p><img src="https://justobjects.nl/uploads/2015/01/tangram.png" alt=" "></p>
<p><a href="https://github.com/tangrams/tangram">https://github.com/tangrams/tangram</a><br>
<a href="https://mapzen.com/tangram">https://mapzen.com/tangram</a> (demo)</p>
<h3 id="d3js">D3JS</h3>
<p>– General</p>
<p><img src="https://justobjects.nl/uploads/2015/01/d3js.png" alt=" "></p>
<p><em>“ <strong>D3.js</strong> is a JavaScript library for manipulating documents based on data. <strong>D3</strong> helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.”</em><br>
<a href="http://d3js.org/">http://d3js.org/</a> –</p>
<p>– Kind of 3D with D3 – Maps for the Web<br>
<a href="http://www.web-maps.com/gisblog/?p=1370">http://www.web-maps.com/gisblog/?p=1370</a></p>
<p>– Creating Charts and Legends for 3D Atlas Maps – A Mashup of D3.js, osgEarth, and the Chromium Embedded Framework — Raimu<br>
<a href="http://vimeo.com/106234276">http://vimeo.com/106234276</a></p>
<h3 id="postgis">PostGIS</h3>
<p>PostGIS has many 3D facilities. Check these out.</p>
<p>– Lidar/Pointclouds in PostGIS:
<a href="https://github.com/pgpointcloud/pointcloud">https://github.com/pgpointcloud/pointcloud</a><br>
<a href="http://workshops.boundlessgeo.com/tutorial-lidar/">http://workshops.boundlessgeo.com/tutorial-lidar/</a> (tutorial)<br>
<a href="http://s3.cleverelephant.ca/foss4gna2013-pointcloud.pdf">http://s3.cleverelephant.ca/foss4gna2013-pointcloud.pdf</a> (paul ramsey slides)<br>
<a href="http://boundlessgeo.com/2013/11/manage-lidar-postgis/">http://boundlessgeo.com/2013/11/manage-lidar-postgis/</a></p>
<p>– TIN Support<br>
<a href="https://smathermather.wordpress.com/2013/12/18/2-5d-tins-in-postgis/">https://smathermather.wordpress.com/2013/12/18/2-5d-tins-in-postgis/</a> and<br>
<a href="https://github.com/smathermather/postgis-etc/blob/master/3D/AsTin.sql">https://github.com/smathermather/postgis-etc/blob/master/3D/AsTin.sql</a> – create TINs with <a href="http://postgis.net/docs/ST_DelaunayTriangles.html" target="_blank" rel="noreferrer noopener">ST_DelaunayTriangles</a>
<br>
<a href="http://www.oslandia.com/full-spatial-database-power-in-2-lines-en.html">http://www.oslandia.com/full-spatial-database-power-in-2-lines-en.html</a><br>
also check: GRASS: <a href="http://grass.osgeo.org/grass70/manuals/addons/v.delaunay3d.html">http://grass.osgeo.org/grass70/manuals/addons/v.delaunay3d.html</a></p>
<p>– X3D Generation<br>
<a href="http://postgis.net/docs/ST_AsX3D.html">http://postgis.net/docs/ST_AsX3D.html</a></p>
<p>– 3ddb for PostGIS (CityGML)<br>
<a href="http://www.3dcitydb.org/">http://www.3dcitydb.org/</a><br>
<a href="https://github.com/3dcitydb">https://github.com/3dcitydb</a> GitHub<br>
<a href="http://www.3dcitydb.org/3dcitydb/fileadmin/downloaddata/3dcitydb-v2_0_6-postgis-tutorial.pdf">http://www.3dcitydb.org/3dcitydb/fileadmin/downloaddata/3dcitydb-v2_0_6-postgis-tutorial.pdf</a></p>
<p>– A New Dimension To PostGIS : 3D – <a href="http://2013.foss4g.org/conf/programme/people/250/index.html" target="_blank" rel="noreferrer noopener">Olivier Courtin</a>
(Oslandia) with <a href="http://2013.foss4g.org/conf/programme/people/139/index.html" target="_blank" rel="noreferrer noopener">Hugo Mercier</a>
(Oslandia)<br>
<a href="http://2013.foss4g.org/conf/programme/presentations/7/index.html">http://2013.foss4g.org/conf/programme/presentations/7/index.html</a><br>
<a href="http://www.slideshare.net/SimeonNedkov/postgis-3d-implementation">http://www.slideshare.net/SimeonNedkov/postgis-3d-implementation</a><br>
<a href="https://www.youtube.com/watch?v=tQbE6B8JaHI">https://www.youtube.com/watch?v=tQbE6B8JaHI</a></p>
<p>– PostGIS and CGAL
CGAL <a href="https://www.cgal.org/">https://www.cgal.org/</a> and <a href="https://smathermather.wordpress.com/2013/12/05/postgis-with-sfcgal-videos-how-did-i-miss-these-videos/">https://smathermather.wordpress.com/2013/12/05/postgis-with-sfcgal-videos-how-did-i-miss-these-videos/</a><br>
SFCGAL <a href="http://www.sfcgal.org/">http://www.sfcgal.org/</a> “ <em>SFCGAL</em> is a C++ wrapper library around <em><a href="http://www.cgal.org" target="_blank" rel="noreferrer noopener">CGAL</a>
with the aim of supporting ISO 19107:2013 and <a href="http://www.opengeospatial.org/standards/sfa" target="_blank" rel="noreferrer noopener">OGC Simple Features Access 1.2</a>
for 3D operations.”</em></p>
<p>– Other PostGIS 3D Stuff<br>
<em>“This post explains how to setup a powerful spatial data store (PostGIS) with a wide range of features (SFCGAL, PgRouting, PostgreSQL PointCloud, PDAL)“</em><br>
<a href="http://www.oslandia.com/full-spatial-database-power-in-2-lines-en.html">http://www.oslandia.com/full-spatial-database-power-in-2-lines-en.html</a></p>
<p><a href="http://postgis3d.blogspot.nl/">http://postgis3d.blogspot.nl/</a> (Camp2Camp – 2007 – by Mathieu ..?)</p>
<h3 id="xnavigator">XNavigator</h3>
<p><em>“XNavigator is an interactive 3D viewer and integrated client for exploring virtual city and landscape models. Instead of defining its own proprietary communication protocols, open OGC standards are used. The Open Geospatial Consortium (OGC) defines standards for accessing spatial information over the internet. The main 3D content is downloaded from a Web 3D Service (W3DS).</em></p>
<p><em>Additional OGC services which can be accessed include:</em><br>
<em>- Web Map Service (WMS)</em><br>
<em>- Open Location Services (OpenLS) including Route Service, Directory Service, and Geocoder</em><br>
<em>- Catalog Service for Web (CSW) ebRIM profile</em><br>
<em>- Web Feature Service (WFS) serving GML3 and CityGML content “</em></p>
<p><a href="http://xnavigator.sourceforge.net/doku.php">http://xnavigator.sourceforge.net/doku.php</a></p>
<h3 id="threejs">ThreeJS</h3>
<p><em>“The aim of the project is to create a lightweight 3D library with a very low level of complexity — in other words, for dummies. The library provides <canvas>, <svg>, CSS3D and WebGL renderers.”</em></p>
<p><img src="https://justobjects.nl/uploads/2015/01/threejs.png" alt=" "></p>
<p><a href="http://threejs.org">http://threejs.org</a></p>
<h3 id="w3ds">W3DS</h3>
<p>W3DS (Web 3D Service) is a portrayal service for 3D scenes. Early OGC discussion documents. Now superseded by the 3DPS, the 3D Portrayal Service, now (jan 2015) out for public comment in OGC. This is an early W3DS implementation in GeoServer that started from the dissertation work by Nuno Miguel Carvalho Oliveira (professor: Jorge Gustavo Rocha) at the University of Minho (Portugal).</p>
<p><a href="http://mei.di.uminho.pt/sites/default/files/dissertacoes//eeum_di_dissertacao_pg18391.pdf">http://mei.di.uminho.pt/sites/default/files/dissertacoes//eeum_di_dissertacao_pg18391.pdf</a> – Dissertation<br>
<a href="https://github.com/geoserver/geoserver/tree/master/src/community/w3ds">https://github.com/geoserver/geoserver/tree/master/src/community/w3ds</a> – GeoServer community module<br>
<a href="http://osgeo-org.1560.x6.nabble.com/W3DS-Implementation-up-and-running-td4665127.html">http://osgeo-org.1560.x6.nabble.com/W3DS-Implementation-up-and-running-td4665127.html</a> – email on GeoServer list</p>
<h3 id="qgishorao">QGIS/Horao</h3>
<p>By OSLandia</p>
<p><em>“A simple viewer built around OpenSceneGraph … designed to listen to commands on its standard input. … The other piece is a Python plugin that is used to connect QGIS signals to the viewer (in another process) to allow loading of QGIS layers with 3D geometries.”</em></p>
<p><img src="https://justobjects.nl/uploads/2015/01/horao.png" alt=" "></p>
<p><a href="https://github.com/Oslandia/horao">https://github.com/Oslandia/horao</a><br>
<a href="http://www.openscenegraph.org/">http://www.openscenegraph.org/</a></p>
<h3 id="cuardo">Cuardo</h3>
<p>Again By OSLandia, watch these guys!<br>
<em>“ Cuardo is an OpenSource WebGL 3D data viewer, focusing on urban data analysis and visualization … a 3D GIS web framework based on Three.js and WebGL, oriented toward urban visualization.“</em></p>
<p><img src="https://justobjects.nl/uploads/2015/01/cuardo.png" alt=" "></p>
<p><a href="https://github.com/Oslandia/cuardo">https://github.com/Oslandia/cuardo</a>
<a href="http://www.oslandia.com/oslandia-releases-cuardo-3d-gis-viewer-en.html">http://www.oslandia.com/oslandia-releases-cuardo-3d-gis-viewer-en.html</a></p>
<h3 id="suggestions-from-readers">Suggestions from Readers</h3>
<p>After the first version I got quite some suggestions. Thanks!</p>
<p>– Glob3Mobile – <a href="https://twitter.com/DiegoGomezDeck" target="_blank" rel="noreferrer noopener">@DiegoGomezDeck</a>
<br>
<em>“ G3M is a framework developed and designed to:</em><br>
<em>- Develop mobile maps apps in 2D, 2,5D and 3D</em><br>
<em>- Work with real time data</em><br>
<em>- Integrate any kind of data (format,size)</em><br>
<em>- Be integrated on any legacy system</em><br>
<em>- High performance mobile native development</em><br>
<em>- Multi Touch screens</em><br>
<em>- Face the problem of the mobile performance as an integrated problem between server & client</em></p>
<p><em>Works on iOS, Android devices and HTML5 environments. “</em></p>
<p><a href="http://www.glob3mobile.com/">http://www.glob3mobile.com/</a></p>
-
Into the Weather – Part 3 – Publishing Data to the Cloud – 1
https://justobjects.nl/into-the-weather-part-3/
Mon, 15 Dec 2014 23:19:17 +0000
https://justobjects.nl/into-the-weather-part-3/
<p>In my last post, <a href="https://justobjects.nl/into-the-weather-part-2-fun-with-raspberry-pi/">Into the Weather – Part 2</a>
, I outlined a global architecture of a <a href="http://www.davisnet.com/weather/products/vantage-pro-professional-weather-stations.asp" target="_blank" rel="noreferrer noopener">Davis Vantage Pro2</a>
weather station connected to a <a href="http://www.raspberrypi.org/" target="_blank" rel="noreferrer noopener">Raspberry Pi</a>
(RPi) running <a href="http://www.weewx.com/" target="_blank" rel="noreferrer noopener">weewx</a>
weather software to capture raw weather data. Here I will try to depict how to bring this weather data “from the <a href="http://en.wikipedia.org/wiki/Little_Fluffy_Clouds" target="_blank" rel="noreferrer noopener">fluffy clouds</a>
into the digital cloud”. Finally, at the end, also some geospatial content. The image below shows the weather station sensors at the Geonovum building rooftop (was quite hazardous replacing a faulty temperature sensor there!) and the Davis console connected to the Raspberry Pi (transparent enclosure). All documentation and code can be found via: <a href="http://sospilot.readthedocs.org" target="_blank" rel="noreferrer noopener">sospilot.readthedocs.org</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2014/12/davis-pws-geonovum-pics1.jpg" alt=" "></p>
<p>To recap: the <a href="http://www.davisnet.com/weather/" target="_blank" rel="noreferrer noopener">Davis Weather Station</a>
continuously captures raw weather data through its sensors: temperature (out/in), pressure, wind (speed, direction), rainfall and even UV-radiation. This data is initially gathered within the local console display. This is fine for personal/local usage, but for capturing history, deriving trends and in particular for external sharing this is quite limited. The real fun starts with getting access to the raw data and go from there.</p>
<p><img src="https://justobjects.nl/uploads/2014/11/weather-hw-setup.png" alt=" "></p>
<p>This is where the Raspberry Pi with weewx and later <a href="http://www.stetl.org" target="_blank" rel="noreferrer noopener">Stetl</a>
, PostGIS, GeoServer and the 52North SOS come in, but I’ll go step-by-step. Let’s first see how we can publish weather data with just weewx.</p>
<p>My first post <a href="https://justobjects.nl/into-the-weather-part-1/">Into the Weather – Part 1</a>
in this series introduced weewx, a Python framework for capturing, storing and publishing weather data. The Davis weather station is connected via USB to the RPi. The RPi runs <a href="http://www.weewx.com/" target="_blank" rel="noreferrer noopener">weewx</a>
to gather and store weather data (in a SQLite DB) from the weather station. But weewx can do more than this: it can also publish weather data to a variety of services. As any well-designed framework, weewx is basically a kernel, the <em>weewx engine</em> with configurable plugins, all specified and parameterized from a single configuration file <em>weewx.conf</em>, like in <a href="https://github.com/Geonovum/sospilot/blob/master/src/weewx/davis/weewx.conf" target="_blank" rel="noreferrer noopener">this example</a>
. The <em>weewx daemon</em> process runs forever in a main loop continuously calling on all plugins.</p>
<p>First there are <em>weewx station-drivers</em> that continuously capture raw data from most common weather stations. Although there are many brands of weather stations, many will share common hardware and protocols. The second class of plugins are <em>archiving drivers</em>, where/how to store raw weather data. Two standard archiving drivers are available: SQLite and MySQL. My choice: SQLite. For publication from archived data, a <em>standard reporting driver</em> generates a plain HTML website using an extensible <em>skin containing (HTML) templates</em>. By configuring an FTP or <em>rsync</em> destination, the generated HTML can be published to a remote webserver. This is the first connection to the digital cloud. Off course the skin and templates are highly configurable <a href="https://github.com/Geonovum/sospilot/tree/master/src/weewx/davis/byteweather" target="_blank" rel="noreferrer noopener">as in this example</a>
. Many examples can be found on the web. I found the nice <a href="http://davies-barnard.co.uk/2014/01/weewx-byteweather-template" target="_blank" rel="noreferrer noopener">byteweather-template by Chris Davies-Barnard</a>
. Below is the result as can be found at: <a href="http://sensors.geonovum.nl/weewx" target="_blank" rel="noreferrer noopener">sensors.geonovum.nl/weewx</a>
.</p>
<div class="">
<a href="http://sensors.geonovum.nl/weewx/">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/12/weewx-geonovum-screen1.png"
alt="Weewx Standard Report" />
</a>
</div>
<p>In addition, I’ve added even a more dynamic weather display like the <a href="http://wiki.sandaysoft.com/a/SteelSeries_Gauges" target="_blank" rel="noreferrer noopener">Steelseries Gauges</a>
, as seen below and via the link <a href="http://sensors.geonovum.nl/weewx/gauges/" target="_blank" rel="noreferrer noopener">sensors.geonovum.nl/weewx/gauges</a>
.</p>
<div class="">
<a href="http://sensors.geonovum.nl/weewx/gauges/index.html">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/12/weewx-geonovum-gauges-screen1.png"
alt="" />
</a>
</div>
<p>Just like other crowd-sourced projects like OpenStreetMap and WikiPedia there are various weather communities where you can join and publish your weather data via RESTful APIs. weewx provides drivers for most common communities like <a href="http://www.wunderground.com/wundermap?lat=52.152&lon=5.372&zoom=13" target="_blank" rel="noreferrer noopener">Weather Underground</a>
and <a href="http://pwsweather.com/" target="_blank" rel="noreferrer noopener">PWSWeather</a>
. For example, I registered the Geonovum weather station as <a href="http://www.wunderground.com/personal-weather-station/dashboard?ID=IUTRECHT96" target="_blank" rel="noreferrer noopener">Geonovum IUTRECHT96</a>
as below.</p>
<div class="">
<a href="http://www.wunderground.com/personal-weather-station/dashboard?ID=IUTRECHT96">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/12/wu-geonovum-pws1.png"
alt="" />
</a>
</div>
<p>Weather Underground also provides various apps and a map, the <a href="http://www.wunderground.com/wundermap?lat=52.152&lon=5.372&zoom=13" target="_blank" rel="noreferrer noopener">WunderMap</a>
. Here you can view your station, together with all others that jointly provide weather data. As can be seen there is already quite some coverage within The Netherlands.</p>
<div class="">
<a href="http://www.wunderground.com/wundermap?lat=52.152&lon=5.372&zoom=13">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/12/wundermap-nl1.png"
alt="" />
</a>
</div>
<p>All in all, there is a fascinating world to explore once you get into the weather domain and its many communities.</p>
<p>So why am I doing all of this? Apart from having the opportunity to develop this as part of the <a href="http://sospilot.readthedocs.org" target="_blank" rel="noreferrer noopener">SOSPilot Project at Geonovum</a>
, I think that “geospatial” is moving from 2D to “N-dimensional”: not only more and more “3D” is hitting the shores (just see the recent 2014 blogs at <a href="http://planet.osgeo.org/" target="_blank" rel="noreferrer noopener">planet.osgeo.org</a>
), but also location-based sensor data (like Air Quality and weather data) and the <a href="http://en.wikipedia.org/wiki/Internet_of_Things" target="_blank" rel="noreferrer noopener">Internet of Things</a>
drives a need to deal with time-series data: management, storage, services and visualization. Within the Open Source geospatial world I happily see that many frameworks and tools are extended to deal with 3D, like <a href="http://openlayers.org/ol3-cesium/" target="_blank" rel="noreferrer noopener">OpenLayers/Cesium</a>
(one of my next posts) and <a href="http://boundlessgeo.com/2013/11/manage-lidar-postgis/" target="_blank" rel="noreferrer noopener">PostGIS/PDAL</a>
and with Time like in <a href="http://docs.geoserver.org/latest/en/user/services/wms/time.html" target="_blank" rel="noreferrer noopener">GeoServer Dimension</a>
support. Also the <a href="http://www.opengeospatial.org/ogc/markets-technologies/swe" target="_blank" rel="noreferrer noopener">OGC Sensor Web Enablement</a>
and its lighter-weight version <a href="http://ogc-iot.github.io/ogc-iot-api/" target="_blank" rel="noreferrer noopener">OGC SensorThings</a>
is gaining more attention.</p>
<p>Not yet done with the weather. Next post I will dive into further unlocking weather data via OGC services like WMS and SOS. That would be “Publishing Data to Cloud 9” ;-)</p>
-
Open Source and The Theory on Brontosauruses by Anne Elk (Miss)
https://justobjects.nl/open-source-and-the-theory-on-brontosauruses-by-anne-elk-miss/
Thu, 20 Nov 2014 01:55:41 +0000
https://justobjects.nl/open-source-and-the-theory-on-brontosauruses-by-anne-elk-miss/
<p>Preparing for a talk on our <a href="https://osgeo.nl/osgeo-nl-dag-2014/" target="_blank" rel="noreferrer noopener">OSGeo.nl Day</a>
at the Dutch <a href="http://geobuzz.nl/" target="_blank" rel="noreferrer noopener">GeoBuzz</a>
Conference, I am trying to put in some slides on Free and Open Source Software (FOSS) for geospatial: why “FOSS is good” and why I live by it. The usual arguments on licensing, (not) price, feature comparison, collaboration contrasted with proprietary source are to me a, to be honest, a past, boring station.</p>
<p>Interlude: some younger readers (and non Python-programmers), may be puzzled: who the !&$# is <a href="http://en.wikipedia.org/wiki/Anne_Elk%27s_Theory_on_Brontosauruses" target="_blank" rel="noreferrer noopener">Anne Elk</a>
? Ok watch this Monty Python video first:</p>
<div class="">
<a href="https://www.youtube.com/watch?v=cAYDiPizDIs">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/11/anne-elk-300x179.png"
alt="" />
</a>
</div>
<p>From my humble developer’s point of view software is always built on assembled and shared human knowledge codified in programming language lines, ultimately compiled into zeroes and ones running on a machine. Sharing knowledge has been always been key to human evolution. Someone invented building a fire long ago. I am always wondering how <em>that</em> knowledge was shared. Was it “licensed” with other tribes. How? Per fire? For the duration of the fire, or N fires per month? Was the original inventor awarded?</p>
<p>No well-established software developer team will start from a real “scratch”, i.e. first designing and “baking” all hardware, developing CPU-instructions, boot-loaders, operating systems, programming languages, libraries, frameworks etc, in order to develop their application. In general they use available knowledge, i.e. software, from the “giants” that developed those components before. The obvious metaphor is the pyramid: would placing the last stone at the top make the entire structure “yours”, as would be the proprietary software case? From the 1980’s on, for many a marketing manager, still the entire pyramid is sold a piece, as software became a valuable asset.</p>
<p>So in real-life, any smart developer team will Google for, and use any freely (FOSS) software available “out there”, usually being aware of any licensing constraints. Developing “from scratch” is something we did in the dark ages, or even further back, at the time of the brontosauruses.</p>
<p>Well, not that long ago. My career started in the 1985 working for <a href="htCtp://en.wikipedia.org/wiki/AT%26T">AT&amp;T</a>
, later called <a href="http://en.wikipedia.org/wiki/Lucent" target="_blank" rel="noreferrer noopener">Lucent</a>
, for 11 years, working on software for the <a href="ttp://en.wikipedia.org/wiki/5ESS_switch">5ESS public telephone exchange</a>
. I am still grateful for that opportunity. From what I gathered at the time, both software and hardware were all developed “in-house”: the chips, the Unix operating system, the C, later C++ language, its compilers, the whole lot. Well, <em>that</em> is really “from scratch”. Luckily all these goods were later shared with the world. That is why we have Linux and Mac OSX (via BSD, NEXT, but that is another story) today.</p>
<p>But still, who is Anne Elk and what does The Theory on Brontosauruses have to do with all of this? My point is that, although in practice software is developed on the shoulders of “the pyramid builders”, i.e. “the giants”, proprietary software is often still traded in the high spirits of Anne Elk. Although some may be uttering: “we love Open Source, we use it all the time”, as to sell a fire… Only if you are like AT&T and many others at the time, “from scratch” comes close to the truth and may not itch…But for the true humans among us, sharing is us and where we came from.</p>
-
Into the Weather – Part 2 – Fun with Raspberry Pi
https://justobjects.nl/into-the-weather-part-2-fun-with-raspberry-pi/
Thu, 06 Nov 2014 01:35:30 +0000
https://justobjects.nl/into-the-weather-part-2-fun-with-raspberry-pi/
<p>This is a follow-up to <a href="https://justobjects.nl/into-the-weather-part-1/">“Into the Weather – Part 1 – Exploring weewx”</a>
. Sorry, still almost no geospatial content for now. To recap: I am trying to setup an infrastructure where measurements from a <a href="http://www.davisnet.com/weather/products/vantage-pro-professional-weather-stations.asp" target="_blank" rel="noreferrer noopener">Davis Vantage Pro2 Weather Station</a>
are collected and exposed to web services, most notably OGC Services like <a href="http://docs.geoserver.org/latest/en/user/services/wms/time.html" target="_blank" rel="noreferrer noopener">WMS-Time</a>
and SOS, the <a href="http://www.opengeospatial.org/standards/sos" target="_blank" rel="noreferrer noopener">Sensor Observation Service</a>
. The TLDR; /impatient can view results at <a href="http://sensors.geonovum.nl" target="_blank" rel="noreferrer noopener">sensors.geonovum.nl</a>
, sources in the <a href="https://github.com/Geonovum/sospilot/tree/master/src" target="_blank" rel="noreferrer noopener">GitHub project</a>
and in general <a href="http://sospilot.readthedocs.org/en/latest/" target="_blank" rel="noreferrer noopener">the documentation</a>
.</p>
<p>As this setup needs to be run from within my client’s local intranet with available servers “in the Cloud” there is a need for a “relaying middleman”.</p>
<p>The <a href="http://www.raspberrypi.org/" target="_blank" rel="noreferrer noopener">Raspberry Pi</a>
(RPi) was my first choice. The RPi is a credit-card sized computer that can run Linux-es like <a href="http://www.raspbian.org/" target="_blank" rel="noreferrer noopener">Raspbian</a>
, a Linux OS based on <a href="https://www.debian.org/" target="_blank" rel="noreferrer noopener">Debian</a>
. As the Davis weather station console has a USB-interface and <a href="http://www.weewx.com/" target="_blank" rel="noreferrer noopener">weewx</a>
supports read-outs from Davis weather stations, choosing the RPi was obvious. The combination RPi, Raspbian, weewx (try <a href="https://www.google.nl/search?q=RPi%2C+Raspbian%2C+weewx" target="_blank" rel="noreferrer noopener">this Google search</a>
) is becoming more and more popular for setting up public and community-based weather stations.</p>
<p>By now it is time to depict the overall architecture as in the image below.</p>
<p><img src="https://justobjects.nl/uploads/2014/11/weather-hw-setup1.png" alt=" "></p>
<p>The Davis weather station will connect via USB to the RPi. The RPi will run <a href="http://www.weewx.com/" target="_blank" rel="noreferrer noopener">weewx</a>
to gather and store weather data (in a SQLite DB) from the weather station. In addition the RPi will run ETL software based on <a href="http://www.stetl.org" target="_blank" rel="noreferrer noopener">Stetl</a>
to publish weather data to a PostgreSQL/PostGIS database in a cloud server. Using <a href="http://geoserver.org" target="_blank" rel="noreferrer noopener">Geoserver</a>
and the <a href="http://52north.org/communities/sensorweb/sos/" target="_blank" rel="noreferrer noopener">52North SOS</a>
the cloud server will expose the weather data via web services like WMS (Time), WFS and SOS and plain HTML using custom weather web templates like the fabulous <a href="http://wiki.sandaysoft.com/a/SteelSeries_Gauges" target="_blank" rel="noreferrer noopener">steelseries gauges</a>
. I will expand on the software setup in the next post.</p>
<p>For now I will focus on acquiring and setting up the RPi, as this was a fun-exercise by itself. I ordered a kit with a Raspberry Pi Model B+ with power unit, casing, micro SD and WIFI USB adapter (<a href="http://www.kiwi-electronics.nl/raspberry-pi/raspberry-pi-accessoires/wi-pi-draadloze-usb-adapter-voor-raspberry-pi" target="_blank" rel="noreferrer noopener">WiPi</a>
) at <a href="http://www.kiwi-electronics.nl/" target="_blank" rel="noreferrer noopener">Kiwi Electronics</a>
. The whole package arrived the next day.</p>
<p><img src="https://justobjects.nl/uploads/2014/11/rasp-pi-all1.jpg" alt=" "></p>
<p>From unboxing to having everything installed with the <a href="https://justobjects.nl/into-the-weather-part-1/">weewx Simulator (see Part 1)</a>
went smooth. It would take too far to describe all the install steps and gotcha’s. I’ve summarized these here in the <a href="http://sospilot.readthedocs.org/en/latest/raspberrypi-install.html" target="_blank" rel="noreferrer noopener">RPi installation doc</a>
. Apart from a standard Raspbian install, I paid in particular attention to two aspects:</p>
<ul>
<li>running unattended as a headless server, i.e. monitoring and self-healing</li>
<li>having SSH access outside the LAN via reverse SSH-tunneling</li>
</ul>
<p>Monitoring and self-healing are non-neglectable aspects, in particular the weewx server may go down for some reason, as well as the WIFI network and any of the SSH-tunnels. These aspects are described <a href="http://sospilot.readthedocs.org/en/latest/raspberrypi-install.html" target="_blank" rel="noreferrer noopener">in the documentation</a>
.</p>
<p>All in all this step was to get weewx running, still in simulator mode, storing raw weather data in a SQLite database and publishing HTML reports.</p>
<p>So the final result is an RPi humming silently, weewx reporting regularly. In general having a stable system for the next steps: gathering and publishing the weather data to the OGC services like WMS, WFS and SOS. The <a href="http://www.stetl.org" target="_blank" rel="noreferrer noopener">Python-based Stetl framework</a>
, again proved to be instrumental to this effort, both on the RPi and on the Linux server in the Cloud. This will be a subject for my next post. See the architecture below.</p>
<p><img src="https://justobjects.nl/uploads/2014/11/weather-sw-setup.png" alt=" "></p>
<p>Summarizing: A Stetl process (<a href="https://github.com/Geonovum/sospilot/tree/master/src/weather/weewx2pg" target="_blank" rel="noreferrer noopener">Stetl sync</a>
) continuously gathers and publishes weather data to a remote PostgreSQL/PostGIS server. Through PostgreSQL-VIEWs refined weather data is immediately available to GeoServer as WMS(Time) and WFS sources, and via another Stetl process (<a href="https://github.com/Geonovum/sospilot/tree/master/src/weather/pg2sos" target="_blank" rel="noreferrer noopener">Stetl SOS</a>
) published via SOS-T (ala WFS-T) to the SOS. The weewx engine has a plugin to publish a flat HTML website via <em>rsync</em> using configurable templates.</p>
<p>There is much to expand still. I’m excited to see this whole infrastructure work in such a short time <em><strong>thanks to</strong> <strong>all these developers that produce all of the Open Source software involved here</strong></em>: from the core Debian/Raspbian OSs, the weewx weather software, PostgreSQL/PostGIS, GeoServer and the 52North SOS. The proverbial sentence is: <em><strong>I am just standing on the shoulders of you giants</strong></em>. This is the way humanity should evolve regarding soft/hardware-technology. Thanks again, and if you have read this far, I hope to see you in my next post !</p>
-
Into the Weather – Part 1 – Exploring weewx
https://justobjects.nl/into-the-weather-part-1/
Fri, 17 Oct 2014 01:21:02 +0000
https://justobjects.nl/into-the-weather-part-1/
<div class="float_right">
<a href="http://lib.heron-mc.org/heron/latest/examples/simpletimeslider/">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/wms-time-heron-knmi-300x181.png"
alt="WMS Time Example with GeoServer in Heron" />
</a>
</div>
<p>Tagging this post as “Part 1” is ambitious. Beware: there is hardly any “geo” for now. In the coming time I hope to share some technical experiences with weather stations, weather software and ultimately exposing weather data via some open geospatial standards like <a href="http://mapserver.org/ogc/wms_time.html" target="_blank" rel="noreferrer noopener">OGC WMS(-Time)</a>
as in <a href="http://lib.heron-mc.org/heron/latest/examples/simpletimeslider/" target="_blank" rel="noreferrer noopener">example image right</a>
, WFS and in particular <a href="http://en.wikipedia.org/wiki/Sensor_Observation_Service" target="_blank" rel="noreferrer noopener">SOS (Sensor Observation Service)</a>
. The context is an exciting project with <a href="http://www.geonovum.nl/" target="_blank" rel="noreferrer noopener">Geonovum</a>
in the Netherlands: to transform and expose (via web services and reporting) open/raw Air Quality data from <a href="http://www.rivm.nl/" target="_blank" rel="noreferrer noopener">RIVM</a>
, the Dutch National Institute for Public Health and the Environment. The main link to this project is <a href="http://sensors.geonovum.nl" target="_blank" rel="noreferrer noopener">sensors.geonovum.nl</a>
. All software is developed as FOSS <a href="https://github.com/Geonovum/sospilot" target="_blank" rel="noreferrer noopener">via a GitHub project</a>
. There are already some results there. I may post on these later.</p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/sospilot-screenshot-300x206.png"
alt="" />
</a>
</div>
<p>Within a sub-project the aim is to expose measurements from a physical weather station via standardized OGC web services like WMS, WFS and SOS. As a first step I dived into the world of weather hardware and software, in particular their vivid open source/open data communities. A whole new world expanded to me. To no surprise: Location and The Weather are part of everyday life since the beginnings of humanity. <a href="http://openweathermap.org/" target="_blank" rel="noreferrer noopener">OpenWeatherMap</a>
and <a href="http://www.wunderground.com/" target="_blank" rel="noreferrer noopener">Weather Underground</a>
are just two of the many communities around open weather data. In addition there’s an abundance of FOSS weather software. Personal weather stations are measuring not just temperature but also pressure, humidity, rainfall, wind, up to UV radiation and are built <a href="http://www.zipfelmaus.com/blog/arduino-weather-shield-schematics-layout-code-everything-you-need/" target="_blank" rel="noreferrer noopener">homebrew</a>
or <a href="http://www.weathershop.co.uk/" target="_blank" rel="noreferrer noopener">bought for as cheap as $50,-. </a>
</p>
<p><img src="https://justobjects.nl/uploads/2014/10/weather-hacking.png" alt="Weather Hacking"></p>
<p>Being a noob in weather soft/hardware technology I had to start somewhere and then go step-by-step. The overall “architecture” can be even depicted in text:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>weather station --> soft/middleware --> web services + reporting
</code></pre></div>
<div class="float_right">
<a href="http://www.davisnet.com/weather/products/vantage-pro-professional-weather-stations.asp">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/davis-vantage-pro2-300x188.jpg"
alt="Davis Vantage Pro2 Weather Station" />
</a>
</div>
<p>Being more of a software person, I decided to start with the weather soft/middleware. Also, since Geonovum already owns a <a href="http://www.davisnet.com/weather/products/vantage-pro-professional-weather-stations.asp" target="_blank" rel="noreferrer noopener">Davis Vantage Pro2 Weather Station</a>
and the <a href="http://www.raspberrypi.org/products/model-b-plus/" target="_blank" rel="noreferrer noopener">Raspberry Pi B+</a>
I plan to use is still underway…</p>
<p>From what I gathered, <a href="http://www.weewx.com" target="_blank" rel="noreferrer noopener">weewx</a>
is the most widely used engine/framework within the weather FOSS community. Also the fact that it is written in Python with a very extensible architecture immediately settled my choice. Explaining <a href="http://www.weewx.com" target="_blank" rel="noreferrer noopener">weewx</a>
is a subject by itself but <a href="http://www.weewx.com/docs.html" target="_blank" rel="noreferrer noopener">very well documented</a>
. I’ll try in a few sentences what <a href="http://www.weewx.com" target="_blank" rel="noreferrer noopener">weewx</a>
does:</p>
<ul>
<li>collect current and archive weather station measurement data (drivers)</li>
<li>storing weather data (archive and statistics) in a database (<a href="http://www.sqlite.org/" target="_blank" rel="noreferrer noopener">SQLite</a>
or MySQL)</li>
<li>submitting data to weather community services like <a href="http://www.wunderground.com/" target="_blank" rel="noreferrer noopener">Weather Underground</a>
</li>
<li>creating formatted/templated reports for your local or remote website</li>
</ul>
<p>Any of these functionalities is highly extensible through a configurable plugin architecture. The drivers support most common weather stations. Installing is a breeze, either in a local directory or via Linux package managers. Also note that weather data have quite some different local units (Fahrenheit/Celsius, knots/meters etc). <a href="http://www.weewx.com" target="_blank" rel="noreferrer noopener">weewx</a>
will all take care of this.</p>
<p>So, not yet having access to a weather station, what could I do? One of the weather station drivers is the <a href="http://www.weewx.com/docs/usersguide.htm#[Simulator]" target="_blank" rel="noreferrer noopener">Simulator</a>
which intelligently generates weather data for testing.</p>
<div class="float_right">
<a href="http://openweathermap.org/maps">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/openweathermap-300x196.png"
alt="" />
</a>
</div>
<p>Trying to have some real-world data I set out on what appeared to be a two-hour hack: create a weather station driver that obtains its data from an open weather API. There are many off course. I choose the <a href="http://openweathermap.org/api" target="_blank" rel="noreferrer noopener">OpenWeatherMap API</a>
to get data in the area of our cabin in the woods near the place of <a href="http://en.wikipedia.org/wiki/Otterlo" target="_blank" rel="noreferrer noopener">Otterlo in the Netherlands</a>
. Writing this hard-coded driver took just a few line of Python. The <a href="https://github.com/Geonovum/sospilot/blob/master/src/weewx/test/weatherapidriver.py" target="_blank" rel="noreferrer noopener">source code can be found here</a>
. To not overask the API, I’ve set the time interval to 2 minutes within the <a href="https://github.com/Geonovum/sospilot/blob/master/src/weewx/test/weewx.conf" target="_blank" rel="noreferrer noopener">weewx configuration file</a>
. Also it would not be fair to report these values to any of the weather communities. If the weewx community is interested I can donate this software, with some generalization (e.g. URLvia config).</p>
<p>But all in all my first driver is still running fine in weewx. The main challenge was converting all the values between different metric systems. weewx allows and even encourages you to store all data in US metrics. All the reporting and conversion utilities will always allow you to show your local metric units.</p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/otterlo-weewx-report-300x275.png"
alt="" />
</a>
</div>
<p>As a Linux daemon now runs fine in our test system. It is time to show some results. weewx reporting is basically a website generated via <a href="http://www.cheetahtemplate.org/" target="_blank" rel="noreferrer noopener">Cheetah templates</a>
. The default template is basic white on black. I found a nice template called <a href="http://davies-barnard.co.uk/2014/01/weewx-byteweather-template" target="_blank" rel="noreferrer noopener">Byteweather</a>
. You can find my continuous weather report here at <a href="http://sensors.geonovum.nl/weather/" target="_blank" rel="noreferrer noopener">sensors.geonovum.nl/weather</a>
. Measurements are now building up thanks to the weewx archive database. Values are mostly matching Dutch weather station data. Expect for the rainfall…Surely we have lots of rain but not that much…</p>
<p>Next posting I hope to tell more about deploying the Raspberry Pi and connecting to the Geonovum Davis Weather station. Then there will be also more “geo” in the post, I promise!</p>
-
Unlocking the amazing OpenTopo.nl Maps
https://justobjects.nl/unlocking-the-amazing-opentopo-nl-maps/
Mon, 13 Oct 2014 01:08:08 +0000
https://justobjects.nl/unlocking-the-amazing-opentopo-nl-maps/
<p>The essence of this post is to illustrate via some web-apps the great topographic mapping done by <a href="http://www.imergis.nl/asp/44.asp" target="_blank" rel="noreferrer noopener">Jan-Willem van Aalst</a>
using Dutch free geo-data. The summary/TL;DR is:</p>
<ul>
<li><a href="http://opentopo.nl/" target="_blank" rel="noreferrer noopener">great maps produced from Dutch Open Data</a>
by Jan-Willem van Aalst</li>
<li>unlocked by me via web tiling services and <a href="http://app.nlextract.nl/ot/" target="_blank" rel="noreferrer noopener">simple web apps</a>
like <a href="http://app.nlextract.nl/ot/opentopord.html" target="_blank" rel="noreferrer noopener">this one</a>
.</li>
</ul>
<p>The longer post here below…</p>
<div class="float_left">
<a href="http://www.swaen.com/mapmaker.php">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/04461.jpg"
alt="" />
</a>
</div>
<p>The Netherlands and Belgium have produced some influential mapmakers throughout history. Up to the 20th century the “Low Countries” were part of empires that ruled at the time. Mapmakers could be found in cities like Antwerp, Ghent and Amsterdam. I found <a href="http://www.swaen.com/mapmaker.php" target="_blank" rel="noreferrer noopener">an overview here</a>
, but there may be more authoritative sources. <a href="https://en.wikipedia.org/wiki/Gerardus_Mercator" target="_blank" rel="noreferrer noopener">Mercator</a>
, was one of them, unaware probably of the later <a href="http://spatialreference.org/ref/sr-org/6627/" target="_blank" rel="noreferrer noopener">Google Mercator</a>
;-)</p>
<p>Map-making has moved into the digital era. Anyone can render maps from source data, using attributed polygons, lines and points with some styling. Hmm, but there is more to the picture than meets the eye: maps have and always will be a combination of art, science and technology.</p>
<p> </p>
<div class="float_right">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/ot1-300x215.png"
alt="" />
</a>
</div>
<p>Every mapmaker, now and back then, is dependent on (accurate) map-data. Within the Netherlands we have been fortunate by having multiple free geo-data sources: <a href="http://www.openstreetmap.nl/" target="_blank" rel="noreferrer noopener">OpenStreetMap</a>
(with several imports) and since 2012 Dutch topographic and cadastral geospatial <a href="https://www.pdok.nl/nl/producten/downloaden-van-data-pdok" target="_blank" rel="noreferrer noopener">data sources via PDOK</a>
, plus many others. These data-sources can all be combined, but this endeavour still requires a true mapmaker…</p>
<p> </p>
<p> </p>
<div class="float_left">
<a href="http://opentopo.nl/">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/combi8-300x245.png"
alt="Combining data source in OpenTopo (img by JW van Aalst)" />
</a>
</div>
<!--
<div id="attachment_397" style="width: 310px" class="wp-caption alignleft">
<a href="http://opentopo.nl/"><img aria-describedby="caption-attachment-397" loading="lazy" class="wp-image-397 size-medium" src="uploads/2014/10/combi8-300x245.png" alt="Combining data source in OpenTopo (img by JW van Aalst)" width="300" height="245" srcset="https://justobjects.nl/wp-content/uploads/2014/10/combi8-300x245.png 300w, https://justobjects.nl/wp-content/uploads/2014/10/combi8-183x150.png 183w, https://justobjects.nl/wp-content/uploads/2014/10/combi8-150x122.png 150w, https://justobjects.nl/wp-content/uploads/2014/10/combi8.png 600w" sizes="(max-width: 300px) 100vw, 300px" /></a>
<p id="caption-attachment-397" class="wp-caption-text">
Combining data source in OpenTopo (img by JW van Aalst)
</p>
</div>
-->
<p>Jan-Willem van Aalst, via his website <a href="http://opentopo.nl/" target="_blank" rel="noreferrer noopener">OpenTopo</a>
, tastefully combines Dutch open geo-data sources like <a href="http://www.openstreetmap.org/" target="_blank" rel="noreferrer noopener">OpenStreetMap</a>
, Dutch public data sources like: <a href="http://nl.wikipedia.org/wiki/Basisregistratie_Topografie" target="_blank" rel="noreferrer noopener">BRT</a>
(topography), <a href="http://www.kadaster.nl/bag" target="_blank" rel="noreferrer noopener">Buildings (BAG)</a>
and <a href="http://www.ahn.nl/" target="_blank" rel="noreferrer noopener">“AHN”, Dutch height data</a>
(yes, we are not that flat!) into detailed and attractive topographic maps. His main editing tool is <a href="http://www.qgis.org/" target="_blank" rel="noreferrer noopener">QGIS</a>
. Producing several resolutions up to 800 pixels/kilometer, the output is rendered as a series of straight <a href="http://en.wikipedia.org/wiki/Tagged_Image_File_Format" target="_blank" rel="noreferrer noopener">TIFF</a>
files in various resolutions (up to 800px/km).</p>
<p> </p>
<p> </p>
<div class="float_right">
<a href="http://app.nlextract.nl/ot/opentopord.html">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/10/ot3-300x209.png"
alt="" />
</a>
</div>
<p>But how to make Jan-Willem’s results (TIFFs) available online? <a href="http://www.nlextract.nl/" target="_blank" rel="noreferrer noopener">The NLExtract project</a>
aims to convert Dutch Open Geo-Data to manageable formats by providing tools and other services. For example the raw downloaded GML data for Dutch Address and Building Data (BAG) and the BRT Top10NL Topography data were first converted to PostGIS tables using the <a href="http://docs.nlextract.nl/en/latest/" target="_blank" rel="noreferrer noopener">NLExtract ETL tools</a>
for use by Jan-Willem in QGIS.</p>
<p> </p>
<p> </p>
<p>In addition NLExtract provides <a href="http://data.nlextract.nl/" target="_blank" rel="noreferrer noopener">downloads of converted and TIFF data and some apps</a>
to demonstrate transformation results.</p>
<p>In order to make the OpenTopo maps of Jan-Willem more widely available, I have provided both a tiling (TMS/WMTS) service for the maps plus some simple apps for browsers and mobile devices (tables/smartphones/phablets). These can be found at: <a href="http://app.nlextract.nl/ot/" target="_blank" rel="noreferrer noopener">app.nlextract.nl/ot</a>
, in particular the <a href="http://app.nlextract.nl/ot/opentopord.html" target="_blank" rel="noreferrer noopener">OpenTopoMap tiled in Dutch Projection</a>
. Thanks to Bart van den Eijnden for the <a href="https://github.com/bartvde/PDOK-Leaflet" target="_blank" rel="noreferrer noopener">Leaflet-Proj4s integration</a>
.</p>
<p>The following steps and technologies have been used within NLExtract to unlock JW’s raw TIFFs into tiled web services and <a href="http://app.nlextract.nl/ot/" target="_blank" rel="noreferrer noopener">demo-apps</a>
:</p>
<ul>
<li>Generate Worldfiles from a spreadsheet (by Frank Steggink) with a <a href="https://github.com/opengeogroep/NLExtract/tree/master/opentopo/src" target="_blank" rel="noreferrer noopener">small Python program</a>
.</li>
<li>from TIFF+Worldfiles to GeoTIFF: the <a href="https://github.com/opengeogroep/NLExtract/blob/master/opentopo/bin/topotrans.sh" target="_blank" rel="noreferrer noopener">NLExtract Topotrans script</a>
(using GDAL commands)</li>
<li>GeoTIFFs directory to WMS via <a href="http://docs.geoserver.org/stable/en/user/data/raster/imagemosaic.html" target="_blank" rel="noreferrer noopener">GeoServer ImageMosaic</a>
</li>
<li>WMS to tiles in EPSG:900913 and EPSG:28992 via <a href="http://geowebcache.org/" target="_blank" rel="noreferrer noopener">GeoWebCache</a>
or <a href="http://mapproxy.org/" target="_blank" rel="noreferrer noopener">MapProxy</a>
</li>
<li><a href="http://app.nlextract.nl/ot/" target="_blank" rel="noreferrer noopener">map viewing apps</a>
built with <a href="http://leafletjs.com/" target="_blank" rel="noreferrer noopener">Leaflet</a>
+ <a href="https://github.com/bartvde/PDOK-Leaflet" target="_blank" rel="noreferrer noopener">Bart vd E proj4js</a>
</li>
</ul>
<p>The source code for the above is <a href="https://github.com/opengeogroep/NLExtract/tree/master/opentopo/src" target="_blank" rel="noreferrer noopener">mostly on GitHub</a>
. In a next blog I will dive deeper into the ETL/transformation processes, <a href="http://www.stetl.org" target="_blank" rel="noreferrer noopener">introducing Stetl, a generic ETL framework in Python</a>
.</p>
-
New website – but what is Open Geospatial?
https://justobjects.nl/new-website-but-what-is-open-geospatial/
Thu, 03 Jul 2014 00:07:20 +0000
https://justobjects.nl/new-website-but-what-is-open-geospatial/
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2014/07/def.jpg"
alt="" />
</a>
</div>
</p>
<p>This is a “sign-of-life-post” as my <a href="https://justobjects.nl/moving-postgis-tables-from-the-public-schema-to-a-new-schema/">last blog</a>
was in 2013. After too many years, I created a new website for <a href="https://justobjects.nl/">justobjects.nl</a>
which also integrates my barren blog. These days, making a website is a breeze thanks to <a href="https://wordpress.org" target="_blank" rel="noreferrer noopener">WordPress</a>
(WP) and <a href="http://en.wikipedia.org/wiki/Responsive_web_design" target="_blank" rel="noreferrer noopener">responsive design</a>
themes. WP provides an ideal dynamic platform (just update regularly and minimize plugins). I hope to regularly publish here about some of the Open Geospatial projects I am working on plus other events in the <a href="https://osgeo.org" target="_blank" rel="noreferrer noopener">OSGeo</a>
arena and my personal thoughts on “Open Geospatial”.</p>
<p>Huh, “Open Geospatial”? Ok, one of the main issues I am struggling with is how to call my/our working domain. I often try to explain this as a combination of two domains:</p>
<ol>
<li>what is known as <a href="http://en.wikipedia.org/wiki/Geographic_information_system" target="_blank" rel="noreferrer noopener">“GIS”</a>
, “Geospatial”, or better <a href="https://www.youtube.com/watch?v=01yrhqCro7I" target="_blank" rel="noreferrer noopener">“Spatial IT”</a>
(watch this Paul Ramsey Video!).</li>
<li>the world of <a href="http://www.fsf.org/" target="_blank" rel="noreferrer noopener">Free Software</a>
, <a href="http://en.wikipedia.org/wiki/Open-source_software" target="_blank" rel="noreferrer noopener">Open Source Software (OSS)</a>
and <a href="http://en.wikipedia.org/wiki/Free_and_open-source_software" target="_blank" rel="noreferrer noopener">Free and Open Source Software (FOSS)</a>
</li>
</ol>
<p>Both have definition issues. How to even name the combination of these two, so that your family, friends and even IT-folks immediately know <a href="https://www.youtube.com/watch?v=v9JSKPxQjMo" target="_blank" rel="noreferrer noopener">what you are doing</a>
(another recommended Paul Ramsey video)? A challenge. I sometimes call it the <a href="http://en.wikipedia.org/wiki/Intersection_%28set_theory%29" target="_blank" rel="noreferrer noopener">intersection</a>
of the two, but set-theory gets too far for some. The abbreviation <a href="http://foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G</a>
appeals, but gives confusion with the great conferences by that name.</p>
<p>And then I would even like to include <a href="http://en.wikipedia.org/wiki/Open_standard" target="_blank" rel="noreferrer noopener">Open Standards</a>
and <a href="http://en.wikipedia.org/wiki/Open_data" target="_blank" rel="noreferrer noopener">Open Data</a>
into the mix. So, for convenience, I have dubbed the domain here as Open Geospatial, maybe Open Spatial would be better, at least the term summarizes reference to the <a href="http://www.opengeospatial.org/" target="_blank" rel="noreferrer noopener">Open Geospatial Consortium (OGC)</a>
and the <a href="https://osgeo.org" target="_blank" rel="noreferrer noopener">Open Source Geospatial Foundation (OSGeo)</a>
.</p>
<p>Ok, probably I should not bother, but if someone comes up with a crisp definition, I am Open (sic!).</p>
-
Moving PostGIS tables from the public schema to a new schema
https://justobjects.nl/moving-postgis-tables-from-the-public-schema-to-a-new-schema/
Fri, 01 Feb 2013 16:40:38 +0000
https://justobjects.nl/moving-postgis-tables-from-the-public-schema-to-a-new-schema/
<p>Update 30 nov 2013: <a href="https://gist.github.com/justb4/7719180" target="_blank" rel="noreferrer noopener">code is now as a GIST on GitHub</a>
, also with function to move single table.</p>
<p>Update 19 feb 2014: <strong><em>not yet tested with PostGIS v2.0 and up, so beware (or let me know if that works)!</em></strong></p>
<p>Update 4 june 2015: <strong>_tested with PostgreSQL 9.3.4 and PostGIS v2.1: all ok!! _</strong></p>
<p>Using <a href="http://www.postgresql.org/docs/9.2/static/ddl-schemas.html" target="_blank" rel="noreferrer noopener">PostgreSQL Schema’s</a>
when using <a href="http://postgis.org/" target="_blank" rel="noreferrer noopener">PostGIS</a>
is very useful. Instead of the default <em>public</em> schema where PostGIS and its meta-tables (<em>geometry_columns</em> and <em>spatial_ref_sys</em>) are installed one can use an explicit schema. One main reason, at least for me, is that PostgreSQL Schema’s allow me to make data dumps of the Schema (via <a href="http://www.postgresql.org/docs/9.2/static/app-pgdump.html" target="_blank" rel="noreferrer noopener">pg_dump</a>
) and restore these dumps in another database, even on another system. When using the default _public _schema a dump would also include PostGIS functions and metatables. This is problematic to restore on another system or higher PostgreSQL/PostGIS version. So I recommend always to use Schema’s.</p>
<p>But what if your tables are already in the <em>public</em> schema? This was the case in some of my older projects like <a href="http://geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating</a>
which has tables in PostgreSQL 8.2 in the <em>public</em> schema. With some hacking and <a href="http://blog.coreycoogan.com/2010/12/22/how-to-move-postgresql-tables-to-a-different-schema/" target="_blank" rel="noreferrer noopener">surfing</a>
on the web, I constructed an SQL function that would move my tables from the _public S_chema to any other Schema and update the PostGIS metatables (and leave these in the _public _Schema). See the function <em>postgis_schema_move</em>() below.</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>-- Function to move tables to schema
<span class="ln"> 2</span>CREATE OR REPLACE FUNCTION
<span class="ln"> 3</span> public.postgis_schema_move(old_schema varchar(32),
<span class="ln"> 4</span> new_schema varchar(32)) RETURNS void AS $$
<span class="ln"> 5</span>DECLARE
<span class="ln"> 6</span> row record;
<span class="ln"> 7</span>BEGIN
<span class="ln"> 8</span> FOR row IN SELECT tablename FROM pg_tables
<span class="ln"> 9</span> WHERE schemaname = old_schema and tablename != 'spatial_ref_sys'
<span class="ln">10</span> AND tablename != 'geometry_columns'
<span class="ln">11</span> LOOP
<span class="ln">12</span> EXECUTE 'ALTER TABLE ' || quote_ident(old_schema) || '.'
<span class="ln">13</span> || quote_ident(row.tablename) ||
<span class="ln">14</span> ' SET SCHEMA ' || new_schema || ';';
<span class="ln">15</span> EXECUTE 'UPDATE public.geometry_columns
<span class="ln">16</span> SET f_table_schema = ' || quote_literal(new_schema) ||
<span class="ln">17</span> ' WHERE f_table_schema = ' || quote_literal(old_schema) ||'
<span class="ln">18</span> AND f_table_name = ' || quote_literal(row.tablename) || ';';
<span class="ln">19</span> END LOOP;
<span class="ln">20</span>END;
<span class="ln">21</span>$$ LANGUAGE plpgsql;
<span class="ln">22</span>
<span class="ln">23</span>-- Example: Move from public to schema app
<span class="ln">24</span>SELECT public.postgis_schema_move('public', 'app');</pre>
</code></pre></div><p>The SQL function <em>postgis_schema_move()</em> takes two string arguments: the <em>old_schema</em> and <em>new_schema</em>, the old (usually <em>‘public’</em>) and new schema names. The new schema needs to be created first. Also moving back from a new schema to the <em>public</em> schema works.</p>
-
OSGeo Open Source Seminar at Geospatial World Forum
https://justobjects.nl/osgeo-seminar-gwf-2012/
Tue, 17 Apr 2012 12:11:47 +0000
https://justobjects.nl/osgeo-seminar-gwf-2012/
<p>The <a href="https://osgeo.org" target="_blank" rel="noreferrer noopener">OSGeo Foundation</a>
has been invited to provide a 4-hour <a href="http://www.geospatialworldforum.org/2012/open.htm" title="Open Source Seminar at GWF Page" target="_blank" rel="noreferrer noopener">Open Source Seminar at the Geospatial World Forum</a>
. This event will take place on April 25, 2012 in the <a href="http://maps.google.com/maps/place?q=RAI+Amsterdam,+The+Netherlands&cid=11569527158754808632" target="_blank" rel="noreferrer noopener">RAI Amsterdam, The Netherlands</a>
. As Trailblazer (Dutch “Kwartiermaker”) for the newly formed <a href="https://osgeo.nl" title="OSGeo.nl The Dutch OSGeo Chapter" target="_blank" rel="noreferrer noopener">OSGeo.nl Dutch Language Chapter</a>
I was in charge of finding speakers and structuring the program with the valuable aid of <a href="http://arnulf.us/Arnulf_Christl" target="_blank" rel="noreferrer noopener">Arnulf Christl, president OSGeo</a>
. For <a href="https://osgeo.nl" target="_blank" rel="noreferrer noopener">OSGeo.nl</a>
this was an opportunity to expose our newly formed chapter. <a href="http://www.linkedin.com/in/willybakker" target="_blank" rel="noreferrer noopener">Willy Bakker</a>
of OSGeo.nl is still busy trying to arrange OSGeo marketing exposure. I feel very happy and proud that all these experts in their respective fields committed to speak in 15-min slots (the GWF-format). I think the program also stands out in Variety: Open Source for Geospatial here is not just presented from a technical/developers’ angle, but also from end-users’ perspectives in local and global governments, from within education and from the pure business angle. As we have 11 speakers (of which even two presidents!), and hearing someone mutter in the social media about a ‘Dreamteam’, I could not resist collating the undesigned poster as below.</p>
<p><img src="https://justobjects.nl/uploads/2014/06/OSGeo_OpenSource_GWF12_DreamTeam1.jpg" alt=" "></p>
<p>Below is also the detailed program as it stands now, small changes may occur. As there is little time for questions within the slots, we will devote the Forum at the end to Q&A. Questions can also be posed, even if you are not there, via Twitter, by using the <a href="https://twitter.com/#!/search/realtime/%23gwfos" title="twitter tag for questions at GWF OS seminar" target="_blank" rel="noreferrer noopener">Twitter hashtag #gwfos</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2014/06/Open-Source-GWF2012-final1-1024x842.jpg" alt=" "></p>
<p><a href="http://wiki.osgeo.org/images/b/bf/GeospatialWorldForum_2012_OpenSource_Seminar_Program.pdf" target="_blank" rel="noreferrer noopener">Click here for a PDF version of the program</a>
</p>
-
SOTM and FOSS4G Conferences 2011 Denver
https://justobjects.nl/sotm-and-foss4g-conferences-2011-denver/
Tue, 20 Sep 2011 13:07:19 +0000
https://justobjects.nl/sotm-and-foss4g-conferences-2011-denver/
<p>What a great time it was in Denver, Colorado, USA ! Attended both the <a href="http://stateofthemap.org/" title="stateofthemap.org" target="_blank" rel="noreferrer noopener">OpenStreetMap State Of The Map (OSM SOTM)</a>
and the <a href="https://osgeo.org" target="_blank" rel="noreferrer noopener">OSGeo</a>
<a href="http://2011.foss4g.org/" title="2011.foss4g.org/" target="_blank" rel="noreferrer noopener">Free and Open Source for Geo (FOSS4G)</a>
conferences between september 9 and 16, 2011. Went hiking in between in the Rocky Mountains (<a href="http://www.nps.gov/romo/index.htm" target="_blank" rel="noreferrer noopener">State Park</a>
and <a href="http://www.coloradowilderness.com/wildpages/indian.html" target="_blank" rel="noreferrer noopener">Indian Peaks Wilderness</a>
) where I saw a live bear the first time of my life.</p>
<p><img src="https://justobjects.nl/uploads/2011/09/nederland-colorado-chipmunketc.jpg" alt=" "></p>
<p>At my first SOTM I was impressed how strong and vibrant the <a href="http://openstreetmap.org" target="_blank" rel="noreferrer noopener">OSM community</a>
is, how an ecosystem of tools, data, people and businesses is continuously expanding and how far “The Map” has progressed. The FOSS4G at the Sheraton Hotel in downtown Denver was very well-organized with high technical/social value. FOSS4G is a must-go conference if you are into FOSS Geo.</p>
<p><img src="https://justobjects.nl/uploads/2011/09/denver-just-shots.jpg" alt=" "></p>
<p>Denver is a real city but with a country-like, mellow atmosphere. Socializing among the participants went smoothly, also since all major pubs were all directly near the Sheraton. The opening social night on tuesday at the <a href="http://www.wynkoop.com/" title="www.wynkoop.com/" target="_blank" rel="noreferrer noopener">“Wynkoop Brewery”</a>
was an immediate hit. Many also went there on friday to close off. At FOSS4G I gave a talk on <a href="http://2011.foss4g.org/sessions/openstreetmap-and-inspire-can-twain-meet" title="2011.foss4g.org/sessions/openstreetmap-and-inspire-can-twain-meet" target="_blank" rel="noreferrer noopener">“OpenStreetMap and INSPIRE, Can the Twain Meet?”</a>
. You can find my slides in <a href="https://files.justobjects.nl/presentation/foss4g-2011/html/img0.html" target="_blank" rel="noreferrer noopener">HTML</a>
or <a href="https://files.justobjects.nl/presentation/foss4g-2011/osm-inspire-foss4g2011-broecke.pdf" target="_blank" rel="noreferrer noopener">PDF</a>
. I prefer staying in Bed & Breakfasts these days: they are in general cheaper, cleaner and cozy compared to regular Hotels/Motels. I had a real pleasant stay at the <a href="http://www.denver-bed-breakfast.com" target="_blank" rel="noreferrer noopener">Holiday Chalet</a>
on East Colfax, run by fashion designer <a href="http://signaturecrystalsharp.com/" target="_blank" rel="noreferrer noopener">Crystal Sharp</a>
with her <a href="http://www.denver-bed-breakfast.com/images/clementine.jpg" target="_blank" rel="noreferrer noopener">Labrador Clementine</a>
. Getting around in Denver was easy and healthy using the <a href="http://denver.bcycle.com/" target="_blank" rel="noreferrer noopener">Denver B-Cycle public bicycle sharing</a>
. As I had no dataplan on my Android phone, I hacked out the B-Cycle station locations from the B-Cycle website and loaded them into my <a href="http://gpstracklog.com/2007/07/garmin-etrex-le.html" target="_blank" rel="noreferrer noopener">Garmin eTrex Legend HCx</a>
handheld GPS. The bicycle station locations are also <a href="http://geocommons.com/overlays/154716" target="_blank" rel="noreferrer noopener">available on GeoCommons</a>
another great geo-sharing site (from a.o. <a href="http://highearthorbit.com/" target="_blank" rel="noreferrer noopener">Andrew Turner</a>
) I knew about but never had used before. In the Rocky Mountains I stayed in a <a href="http://www.airbnb.com/rooms/59868" target="_blank" rel="noreferrer noopener">lovely studio treehouse</a>
found through <a href="http://www.airbnb.com" target="_blank" rel="noreferrer noopener">AirBnB</a>
in the tiny place <a href="http://nederlandco.org/" target="_blank" rel="noreferrer noopener">Nederland, Colorado</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2011/09/nederland-colorado.jpg" alt=" "></p>
<p>Stay tuned to the <a href="https://wiki.openstreetmap.org/wiki/State_Of_The_Map_2011" target="_blank" rel="noreferrer noopener">SOTM</a>
, <a href="http://2011.foss4g.org/" target="_blank" rel="noreferrer noopener">FOSS4G</a>
and <a href="http://www.fosslc.org/drupal/" target="_blank" rel="noreferrer noopener">FOSSLC</a>
websites, since all sessions were recorded and will be made available there. Below some photo’s I made. Hikes with photo’s, video’s and GPX will become available on my <a href="http://georambling.com" target="_blank" rel="noreferrer noopener">GeoRambling website</a>
. I should talk more on SOTM and FOSS4G session content but you can find also lots on the mentioned sites and tweets tagged with <a href="http://twitter.com/#!/search/%23foss4g" target="_blank" rel="noreferrer noopener">#foss4g</a>
and <a href="http://twitter.com/#!/search/%23sotm" target="_blank" rel="noreferrer noopener">#sotm</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2011/09/colorado-hiking.jpg" alt=" "></p>
<p>Next year FOSS4G will be in Beijing, China ! Will SOTM also be held there ?</p>
-
OSGeo Code Sprint Bolsena 2010
https://justobjects.nl/osgeo-code-sprint-bolsena-2010/
Fri, 23 Jul 2010 13:24:11 +0000
https://justobjects.nl/osgeo-code-sprint-bolsena-2010/
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2010/07/bolsena-2010-thumb.jpg"
alt="" />
</a>
</div>
</p>
<p>Like in <a href="http://www.justobjects.org/blog/?p=29" target="_blank" rel="noreferrer noopener">2009</a>
, I attended the <a href="http://wiki.osgeo.org/wiki/Bolsena_Code_Sprint_2010" target="_blank" rel="noreferrer noopener">OSGeo Code Sprint (and Hacking Event)</a>
in <a href="http://en.wikipedia.org/wiki/Bolsena" target="_blank" rel="noreferrer noopener">Bolsena, Italy</a>
during June 6-12, 2010. For one week developers from various Free and Open Source for Geospatial (often abbreviated as FOSS4G) projects get together in a <a href="http://www.conventobolsena.org/" target="_blank" rel="noreferrer noopener">monastery near Bolsena</a>
for code sprints, cross-project collaborations, presentations and geo-hacking in general. This all takes place in a relaxed atmosphere mostly outside. Thanks to the cook Enzo we enjoyed the Italian kitchen. This year I did some work on <a href="http://geonetwork-opensource.org" target="_blank" rel="noreferrer noopener">GeoNetwork</a>
and more in depth on <a href="http://code.google.com/p/inspire-foss/" target="_blank" rel="noreferrer noopener">INSPIRE FOSS</a>
, a new FOSS project I initiated to support the development of <a href="http://inspire.jrc.ec.europa.eu/" target="_blank" rel="noreferrer noopener">INSPIRE</a>
using FOSS. In Bolsena I worked closely together with the <a href="http://deegree.org" target="_blank" rel="noreferrer noopener">deegree</a>
lead developers Markus Schneider and Andreas Schmitz from <a href="http://www.latlon.de/" target="_blank" rel="noreferrer noopener">lat/lon</a>
. At the spot we were able to construct a <a href="http://inspire.kademo.nl/deegree-inspire-demo/" target="_blank" rel="noreferrer noopener">demo</a>
that showcased the power of the upcoming <a href="http://deegree.org" target="_blank" rel="noreferrer noopener">deegree version 3 WFS/WMS</a>
in supporting <a href="http://inspire.jrc.ec.europa.eu/" target="_blank" rel="noreferrer noopener">INSPIRE</a>
Data Themes. Meeting project leads and developers and following presentations from other projects like <a href="http://www.geoserver.org" target="_blank" rel="noreferrer noopener">GeoServer</a>
, <a href="http://deegree.org" target="_blank" rel="noreferrer noopener">deegree</a>
and <a href="http://mapbender.org" target="_blank" rel="noreferrer noopener">MapBender</a>
was really worthwhile. Like last year I have made a video impression you can see below or on <a href="http://www.youtube.com/watch?v=8Vjx06Rlols" target="_blank" rel="noreferrer noopener">YouTube</a>
.</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/8Vjx06Rlols?controls=1&rel=0" loading="lazy"></iframe>
</div>
-
OpenStreetMap Tiles for Dutch Projection EPSG:28992
https://justobjects.nl/openstreetmap-tiles-for-dutch-projection-epsg28992/
Wed, 17 Mar 2010 11:11:25 +0000
https://justobjects.nl/openstreetmap-tiles-for-dutch-projection-epsg28992/
<!--
<img loading="lazy" class="alignleft wp-image-262 size-thumbnail" src="uploads/2010/03/osm-rdtiles-150x150.jpg" alt="osm-rdtiles" width="150" height="150" />][1]
-->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2010/03/osm-rdtiles-150x150.jpg"
alt="" />
</a>
</div>
This article documents how to generate <a href="http://www.openstreetmap.org" target="_blank" rel="noreferrer noopener">OpenStreetMap (OSM)</a>
tiles for the <a href="http://www.rdnap.nl/" target="_blank" rel="noreferrer noopener">Dutch RD (“Rijksdriehoeksmeting”) projection</a>
also known as <a href="http://spatialreference.org/ref/epsg/28992" target="_blank" rel="noreferrer noopener">EPSG:28992</a>
. The steps described below can be used for other projections as well. I assume you are familiar with the OpenStreetMap (OSM) project. If not, there is ample information on the web, for example the <a href="http://wiki.openstreetmap.org" target="_blank" rel="noreferrer noopener">OSM Wiki</a>
. What makes OSM very attractive is not just <a href="http://wiki.openstreetmap.org/wiki/Map_Making_Overview" target="_blank" rel="noreferrer noopener">the shared mapmaking</a>
and an <a href="http://wiki.openstreetmap.org/wiki/OpenStreetMap_License" target="_blank" rel="noreferrer noopener">unrestrictive license</a>
on the resulting map(data), but a toolchain, that allows you to <a href="http://wiki.openstreetmap.org/wiki/Renderers" target="_blank" rel="noreferrer noopener">generate/render your own maps !</a>
.</p>
<p>In addition, <a href="http://www.openstreetmap.nl" target="_blank" rel="noreferrer noopener">OSM within The Netherlands</a>
is very detailed since <a href="http://www.and.com/" target="_blank" rel="noreferrer noopener">Automotive Navigation Data (AND)</a>
has donated a complete road dataset for The Netherlands in 2007 to the OSM project. OSM maps are usually rendered as 256×256 tiles in a <a href="http://docs.openlayers.org/library/spherical_mercator.html" target="_blank" rel="noreferrer noopener">Spherical Mercator projection</a>
with the (unofficial) code EPSG:900913, a.k.a. the “Google Projection”. Spherical Mercator has an official designation of <a href="http://spatialreference.org/ref/epsg/3785" target="_blank" rel="noreferrer noopener">EPSG:3785</a>
but you will mostly find EPSG:900913. Most countries however use local map-projections, mainly for better accuracy and calculations. Most Dutch mapping applications use the aforementioned <a href="http://www.rdnap.nl/" target="_blank" rel="noreferrer noopener">Dutch RD projection, EPSG:28992</a>
. Generating OSM tiles for EPSG:28992 requires some extra steps and has some gotchas you need to be aware of.</p>
<p>Below, I will not describe the setup of the entire toolchain needed to <a href="http://wiki.openstreetmap.org/wiki/Mapnik" target="_blank" rel="noreferrer noopener">generate OSM map tiles with Mapnik</a>
, but just the steps that are specific to our goal: generate OSM map tiles for extent of The Netherlands with the projection EPSG:28992. These steps were done on <a href="http://www.ubuntu.com/" target="_blank" rel="noreferrer noopener">Ubuntu Linux</a>
9.04 (Jaunty). So let’s take the seven steps!</p>
<h3 id="step-1-download-osm-data">Step 1: download OSM data</h3>
<p>Since we only plan to generate tiles for The Netherlands, plus the fact that the projection EPSG:28992 will not even work around the world, we need only an extract for The Netherlands. I have downloaded this extract from <code>http://hypercube.telascience.org/planet/planet-nl-latest.osm.gz</code>, but at the time of this writing this file was not present. Best is to go to <a href="http://wiki.openstreetmap.org/wiki/Planet.osm">http://wiki.openstreetmap.org/wiki/Planet.osm</a> to find a suitable download server. Unpack <code>planet-nl-latest.osm.gz</code>. The resulting XML file <code>planet-nl-latest.osm</code> is around 4.5 GB.</p>
<h3 id="step-2-import-osm-data-in-postgis">Step 2: import OSM data in PostGIS</h3>
<p>Use <a href="http://wiki.openstreetmap.org/wiki/Osm2pgsql" target="_blank" rel="noreferrer noopener">osm2pgsql</a>
to import the Planet XML file into the PostgreSQL/PostGIS database. Since the standard version from the Ubuntu repository gave errors I have built a custom version of <code>osm2pgsql</code> from SVN (rev. 20274) using these steps:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span><span class="cp">#!/bin/bash
</span><span class="ln">2</span><span class="cp"></span>
<span class="ln">3</span>sudo apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj
<span class="ln">4</span>mkdir /opt/osm/osm2pgsql
<span class="ln">5</span><span class="nb">cd</span> /opt/osm/osm2pgsql
<span class="ln">6</span>svn <span class="nb">export</span> http://svn.openstreetmap.org/applications/utils/export/osm2pgsql svn-20274
<span class="ln">7</span>sed -i <span class="s1">'s/-g -O2/-O2 -march=native -fomit-frame-pointer/'</span> Makefile
<span class="ln">8</span>make
<span class="ln">9</span>make install
</code></pre></div><p>Import the OSM file with this command line:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>osm2pgsql --slim -c -E EPSG:4326 -d georzlab -U postgres -W -H localhost S /opt/osm/osm2pgsql/svn-20274/default.style /path/to/planet-nl-latest.osm
</code></pre></div><p>Note the use of EPSG:4326 (standard lon/lat projection) to store data in the DB. Maybe I could have used the default EPSG:900913. The <code>--slim</code> option was needed to prevent errors.</p>
<h3 id="step-3-install-mapnik">Step 3: install Mapnik</h3>
<p>An install of <a href="http://mapnik.org" target="_blank" rel="noreferrer noopener">Mapnik</a>
, the map tile renderer, version 0.7.0 from <a href="http://svn.mapnik.org/tags/release-0.7.0">http://svn.mapnik.org/tags/release-0.7.0</a> was done. Installing Mapnik itself involves many steps. These are described in many places, such as <a href="http://wiki.openstreetmap.org/wiki/Mapnik/Installation" target="_blank" rel="noreferrer noopener">here</a>
and for Ubuntu at <a href="http://trac.mapnik.org/wiki/UbuntuInstallation">http://trac.mapnik.org/wiki/UbuntuInstallation</a>. Best is to have a Mapnik version as recent as possible.</p>
<h3 id="step-4-download-and-extract-world-boundary-files">Step 4: download and extract World Boundary files</h3>
<p>This is a standard step in the Mapnik rendering process for OSM. Specific in our case is that we will extract only the area of The Netherlands from the World Boundary shape files. This is not just for efficiency purposes but required, <em><strong>otherwise rendering boundaries/geonames will silently fail (see below)</strong></em>.</p>
<p>Two steps are required here:</p>
<ol>
<li>extract/clip the Netherlands’ bounding box and</li>
<li>reproject extracted data to EPSG:28992.</li>
</ol>
<p>Thanks to the wonderful geo-library <a href="http://gdal.org" target="_blank" rel="noreferrer noopener">GDAL/OGR</a>
and the command <code>ogr2ogr</code> for vector data manipulations, this can be done in a script as follows:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln"> 1</span><span class="cp">#!/bin/bash
</span><span class="ln"> 2</span><span class="cp"></span>
<span class="ln"> 3</span><span class="c1"># location of shape files</span>
<span class="ln"> 4</span><span class="nb">cd</span> /var/kademo/data/osm/world_boundaries
<span class="ln"> 5</span>
<span class="ln"> 6</span><span class="c1">#</span>
<span class="ln"> 7</span><span class="c1"># Extract NL area to Dutch RD (EPSG:28992)</span>
<span class="ln"> 8</span><span class="c1"># get extent in EPSG:900913 from PostGIS:</span>
<span class="ln"> 9</span><span class="c1"># select ST_Extent(ST_Transform(way,900913)) from planet_osm_line;</span>
<span class="ln">10</span><span class="c1">#</span>
<span class="ln">11</span><span class="nv">extent</span><span class="o">=</span><span class="s2">"311523.765594493 6555476.44574815 822461.515529216 7160903.43417988"</span>
<span class="ln">12</span><span class="nv">srs</span><span class="o">=</span><span class="m">28992</span>
<span class="ln">13</span><span class="nb">echo</span> <span class="s2">"Extract NL for EPSG:</span><span class="si">${</span><span class="nv">srs</span><span class="si">}</span><span class="s2">"</span>
<span class="ln">14</span>/bin/rm <span class="sb">`</span>/bin/ls *<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>*<span class="sb">`</span>
<span class="ln">15</span>ogr2ogr -f <span class="s2">"ESRI Shapefile"</span> -s_srs EPSG:900913 -t_srs EPSG:<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>
<span class="ln">16</span> -spat <span class="si">${</span><span class="nv">extent</span><span class="si">}</span> builtup_area_<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>.shp builtup_area.shp
<span class="ln">17</span>ogr2ogr -f <span class="s2">"ESRI Shapefile"</span> -s_srs EPSG:900913 -t_srs EPSG:<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>
<span class="ln">18</span> -spat <span class="si">${</span><span class="nv">extent</span><span class="si">}</span> processed_p_<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>.shp processed_p.shp
<span class="ln">19</span>ogr2ogr -f <span class="s2">"ESRI Shapefile"</span> -s_srs EPSG:900913 -t_srs EPSG:<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>
<span class="ln">20</span> -spat <span class="si">${</span><span class="nv">extent</span><span class="si">}</span> shoreline_300_<span class="si">${</span><span class="nv">srs</span><span class="si">}</span>.shp shoreline_300.shp
</code></pre></div><p>The extent in EPSG:900913 can be obtained from the data in PostGIS with the <a href="http://www.postgresql.org/docs/8.4/static/app-psql.html" target="_blank" rel="noreferrer noopener">psql</a>
command:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>select ST_Extent(ST_Transform(way,900913)) from planet_osm_line;
</code></pre></div><p>This extra step came about after great help from the very active <a href="http://www.mail-archive.com/[email protected]/info.html" target="_blank" rel="noreferrer noopener">Dutch OSM mailing list</a>
. You can read the relevant thread <a href="http://www.mail-archive.com/[email protected]/msg09240.html" target="_blank" rel="noreferrer noopener">here</a>
. It became clear that the clip/reproject step was necessary. The reason is most probably the Mapnik bug <a href="http://trac.mapnik.org/ticket/308">http://trac.mapnik.org/ticket/308</a>.</p>
<p>Also make sure that you have the proper settings for EPSG:28992 in PROJ’s EPSG file, usually located in <code>/usr/share/proj/epsg</code> and make sure that this setting is actually used by <code>ogr2ogr</code>. Older versions of GDAL may use their own PROJ settings in their .csv files. The <a href="http://bit.ly/9G5fAq" target="_blank" rel="noreferrer noopener">PROJ/PostGIS/GDAL issues around EPSG:28992</a>
deserve a blog-post by themselves. At this moment even <a href="http://spatialreference.org/ref/epsg/28992/" target="_blank" rel="noreferrer noopener">http://spatialreference.org/ref/epsg/28992</a>
publishes wrong PROJ values. The issue mainly deals with the <code>+towgs84</code> parameter, needed for reprojections, not being present.</p>
<h3 id="step-5-install-and-configure-osm-mapnik-tools">Step 5: install and configure OSM Mapnik tools</h3>
<p>This step involves changing the OSM-specific Python-scripts and the Mapnik XML configuration (“The Mapnik Map File”) for invoking Mapnik.</p>
<p>I installed SVN rev. 20274 with the command:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>svn export http://svn.openstreetmap.org/applications/rendering/mapnik
</code></pre></div><p>and ran:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>generate_xml.py
</code></pre></div><p>to generate a basic configuration.</p>
<p>The main step is making changes to the Mapnik map file <code>osm.xml</code> and its included files in <code>inc/*.xml.inc</code>. Below is relevant info.</p>
<p>We need to determine the extent for our tiling scheme. This is in general different from the extent of the dataset. It is the same extent that you will need in your tiling server like <a href="http://tilecache.org" target="_blank" rel="noreferrer noopener">TileCache</a>
and your web client like <a href="http://openlayers.org" target="_blank" rel="noreferrer noopener">OpenLayers</a>
. There is unfortunately no Dutch standard for this extent. I have used the following values</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>EPSG:28992 (RD) -65200.96, 242799.04 375200.96, 683200.96
<span class="ln">2</span> EPSG:4326 (WGS84) 2.307, 50.134 8.752, 54.087
</code></pre></div><p>Change extent in <code>datasource-settings.xml.inc</code>:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span>2.307,50.134,8.752,54.087
</code></pre></div><p>Since our PostGIS data is in EPSG:4326 change <code>inc/settings.xml.inc</code>:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span><!ENTITY osm2pgsql_projection "&srs4326;" >
</code></pre></div><p>Edit <code>inc/entities.xml.inc</code> and add new XML entity for the <a href="http://trac.osgeo.org/proj" target="_blank" rel="noreferrer noopener">Proj</a>
definition for EPSG:28992.</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span><!ENTITY srs28992 "+proj=sterea
<span class="ln">2</span> +lat_0=52.15616055555555 +lon_0=5.38763888888889
<span class="ln">3</span> +k=0.9999079 +x_0=155000 +y_0=463000
<span class="ln">4</span> +ellps=bessel
<span class="ln">5</span> +towgs84=565.237,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812
<span class="ln">6</span> +units=m +no_defs" >
</code></pre></div><p>See also <a href="http://spatialreference.org/ref/sr-org/6781/mapnik" target="_blank" rel="noreferrer noopener">here</a>
for the right “Proj” definition. The only change required in <code>osm.xml</code> is:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span><Map bgcolor="#b5d0d0" srs="&srs28992;" minimum_version="0.6.1">
</code></pre></div><p>There is no need to change Layer elements in <code>osm.xml</code> since they keep the projection from the entity <code>osm2pgsql_projection</code>.</p>
<p>In <code>inc/layer-shapefiles.xml.inc</code> change the names/projections to those of the extracted/reprojected shape files in Step 4. I have used XML entities as follows:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span><layer name="world" status="on" srs="&srs;">
<span class="ln">2</span> <stylename>world</stylename>
<span class="ln">3</span> <datasource>
<span class="ln">4</span> <parameter name="type">shape</parameter>
<span class="ln">5</span> <parameter name="file">&world_boundaries;/shoreline_300_&projection;</parameter>
<span class="ln">6</span> </datasource>
<span class="ln">7</span></layer>
</code></pre></div><p>With <code>&srs;</code> being EPSG:28992 and <code>&projection;</code> 28992.</p>
<h3 id="step-6-generate-test-tile">Step 6: Generate Test Tile</h3>
<p>The moment of truth ! We are going to generate a single map image to test all of our settings.<br>
I made a copy of the Python file <code>generate_image.py</code> and modifed this file as follows:</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln"> 1</span>if __name__ == "__main__":
<span class="ln"> 2</span> try:
<span class="ln"> 3</span> mapfile = os.environ['MAPNIK_MAP_FILE']
<span class="ln"> 4</span> except KeyError:
<span class="ln"> 5</span> mapfile = "osm.xml"
<span class="ln"> 6</span> map_uri = "/path/to/output/file.png"
<span class="ln"> 7</span>
<span class="ln"> 8</span> # Map image bbox
<span class="ln"> 9</span> ll = (4, 52.3, 5, 52.5)
<span class="ln">10</span>
<span class="ln">11</span> # zoomlevel
<span class="ln">12</span> z = 10
<span class="ln">13</span> imgx = 50 * z
<span class="ln">14</span> imgy = 50 * z
<span class="ln">15</span>
<span class="ln">16</span> m = mapnik.Map(imgx,imgy)
<span class="ln">17</span> mapnik.load_map(m,mapfile)
<span class="ln">18</span> prj = mapnik.Projection("
<span class="ln">19</span> +proj=sterea +lat_0=52.15616055555555
<span class="ln">20</span> +lon_0=5.38763888888889
<span class="ln">21</span> +k=0.9999079 +x_0=155000 +y_0=463000
<span class="ln">22</span> +ellps=bessel
<span class="ln">23</span> +towgs84=565.237,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812
<span class="ln">24</span> +units=m +no_defs")
<span class="ln">25</span> c0 = prj.forward(mapnik.Coord(ll[0],ll[1]))
<span class="ln">26</span> c1 = prj.forward(mapnik.Coord(ll[2],ll[3]))
<span class="ln">27</span> if hasattr(mapnik,'mapnik_version') and mapnik.mapnik_version() &gt;= 800:
<span class="ln">28</span> bbox = mapnik.Box2d(c0.x,c0.y,c1.x,c1.y)
<span class="ln">29</span> else:
<span class="ln">30</span> bbox = mapnik.Envelope(c0.x,c0.y,c1.x,c1.y)
<span class="ln">31</span> m.zoom_to_box(bbox)
<span class="ln">32</span> im = mapnik.Image(imgx,imgy)
<span class="ln">33</span> mapnik.render(m, im)
<span class="ln">34</span> view = im.view(0,0,imgx,imgy) # x,y,width,height
<span class="ln">35</span> view.save(map_uri,'png')
</code></pre></div><p>It was here that many of the issues solved above emerged. Below is the image of the first attempt with a silent failure resulting in the World boundary shapefiles being ignored.</p>
<p><img src="http://www.justobjects.org/assets/media/osm-28992-no-shapes.png" alt=""></p>
<p>After using extract/clip (Step 4) the resulting image became as follows.</p>
<p><img src="http://www.justobjects.org/assets/media/osm-28992-ok.png" alt=""></p>
<p>This looked much better. Now the final step is generating all tiles for The Netherlands. Normally this can be done with the OSM script <code>generate_tiles.py</code>, but this script is specific for the Google projection and should be rewritten for EPSG:28992 and the extent used above. For the time being I have used <a href="http://tilecache.org" target="_blank" rel="noreferrer noopener">TileCache</a>
to render and serve the tiles. This is the final step.</p>
<h3 id="step-7-render-tiles-with-tilecache">Step 7: render tiles with TileCache</h3>
<p>Here I used a standard <a href="http://tilecache.org" target="_blank" rel="noreferrer noopener">TileCache</a>
installation with the following configuration.</p>
<pre>[osm_28992]
type=Mapnik
mapfile=/path/to/osm.xml
spherical_mercator=false
resolutions=860.160,430.080,215.040,107.520,53.760,26.880,13.440,6.720,3.360,
1.680,0.840,0.420,0.210,0.105,0.0525
metatile=yes
bbox=-65200.96, 242799.04, 375200.96, 683200.96
srs=EPSG:28992
</pre>
<p>Note that the bbox is the same as the extent in the Mapnik mapfile. Together with these specific resolutions the resulting zoom-levels will approach natural map scales used in The Netherlands like 1:25000. Tiles will be generated during requests. One can also explicitly generate tiles using the standard TileCache script <code>tilecache_seed.py</code>. I used:</p>
<pre>su -s /bin/bash -c "tilecache_seed.py osm_28992 0 12" www-data
</pre>
<p>This will take quite some time also dependent on your TileCache installation (CGI/FastCGI). IMO it will be better to rewrite OSM <code>generate_tiles.py</code>. Below is a resulting excerpt from generated tiles.</p>
<p><img src="http://www.justobjects.org/assets/media/osm-28992-detail.png" alt=""></p>
<p>Somehow the map looks somewhat more busy than the standard OSM “Slippy Map”. This may be due to settings in <code>osm.xml</code> with respect to scales and showing/hiding layers.</p>
<h3 id="finally">Finally</h3>
<p>I hope the above info is useful not just for those that need to generate tiles in Dutch projection but also for other projections. For example for an <a href="http://inspire.jrc.ec.europa.eu/" target="_blank" rel="noreferrer noopener">INSPIRE</a>
project I have generated tiles in ETRS89 (EPSG:4258) with some slight modifications to the Mapnik config and TileCache config. Some further work could include more automation within the OSM Mapnik scripts/config in particular <code>generate_tiles.py</code>. Also, being able to use these tiles in <a href="http://geowebcache.org" target="_blank" rel="noreferrer noopener">GeoWebCache</a>
would be very useful.</p>
-
Guilty landscapes, an Omaha Beach/Band of Brothers geo-journey
https://justobjects.nl/guilty-landscapes-an-omaha-beachband-of-brothers-geo-journey/
Fri, 22 Jan 2010 12:43:07 +0000
https://justobjects.nl/guilty-landscapes-an-omaha-beachband-of-brothers-geo-journey/
<!--
<img loading="lazy" class="alignleft wp-image-264 size-full" src="uploads/2010/01/normandy-1944.jpg" alt="normandy-1944" width="300" height="209" srcset="https://justobjects.nl/wp-content/uploads/2010/01/normandy-1944.jpg 300w, https://justobjects.nl/wp-content/uploads/2010/01/normandy-1944-215x150.jpg 215w, https://justobjects.nl/wp-content/uploads/2010/01/normandy-1944-150x104.jpg 150w" sizes="(max-width: 300px) 100vw, 300px" />
-->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2010/01/normandy-1944.jpg"
alt="" />
</a>
</div>
Last summer I participated in a full-day guided tour along the second world war (WWII) invasion beaches in Normandy, France. Specific subjects of the tour were Omaha Beach and places where events of “Easy Company” (<a href="http://en.wikipedia.org/wiki/E_Company,_506th_Infantry_Regiment_%28United_States%29" target="_blank" rel="noreferrer noopener">E Company of the 2nd Battalion, 506th IR</a>
, 101st Airborne Division, US Army) a.k.a. the <a href="http://en.wikipedia.org/wiki/Band_of_Brothers_%28TV_miniseries%29" target="_blank" rel="noreferrer noopener">Band of Brothers</a>
took place. These true events are described in a <a href="http://tinyurl.com/yfwcd8d" target="_blank" rel="noreferrer noopener">book</a>
by <a href="http://en.wikipedia.org/wiki/Stephen_Ambrose" target="_blank" rel="noreferrer noopener">Stephen Ambrose</a>
from which the famous <a href="http://en.wikipedia.org/wiki/Band_of_Brothers_%28TV_miniseries%29" target="_blank" rel="noreferrer noopener">TV-series</a>
was made in 2001 (the book is also worthwhile!). The tour was organized by <a href="http://www.overlordtour.com" target="_blank" rel="noreferrer noopener">Overlord Tour</a>
. This tour, in a company with mainly American WWII-buffs, was very in-depth and worldwhile, in particular in combination with the landscape. The Dutch painter/writer <a href="http://en.wikipedia.org/wiki/Armando_%28artist%29" target="_blank" rel="noreferrer noopener">Armando</a>
once coined the phrase “guilty landscape” (“schuldig landschap”).</p>
<p>Since I have an interest in WWII history, geography, geospatial, GPS and multimedia, I made a trace with my <a href="https://buy.garmin.com/shop/shop.do?cID=145&pID=8701" target="_blank" rel="noreferrer noopener">Garmin eTrex Legend® HCx</a>
, shot photo’s/video and uploaded all to <a href="http://georambling.com" target="_blank" rel="noreferrer noopener">georambling.com</a>
, one of my pet projects. You can <a href="http://georambling.com/gr/index.html?cmd=showtrace&id=53964&user=just" target="_blank" rel="noreferrer noopener">view the trace with geotagged media</a>
there as well. I’ve also created a video with iMovie from all materials including the GPS traces (see below).</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/S3MCBRWabXY?controls=1&rel=0" loading="lazy"></iframe>
</div>
<p>Subsequently I got a message from Google that my video was banned in Germany ! First I thought because of some swastika’s appearing (carried btw by American soldiers as a trophy after conquering Marmion’s farm), but the reason was more modern: it has to do with a <a href="http://tinyurl.com/ygyzlt8" target="_blank" rel="noreferrer noopener">long-standing dispute between Google and the German collecting society, GEMA</a>
. This makes more sense and may also explain the ads popping up to buy the music (roll over Beethoven). So it is not a <a href="http://en.wikipedia.org/wiki/The_Germans" target="_blank" rel="noreferrer noopener">“Don’t mention the war”</a>
thing. BTW I am a pacifist.</p>
-
OSGeo Hacking Event Bolsena 2009
https://justobjects.nl/osgeo-hacking-event-bolsena/
Mon, 29 Jun 2009 15:42:08 +0000
https://justobjects.nl/osgeo-hacking-event-bolsena/
<!-- <img loading="lazy" class="alignleft wp-image-267 size-medium" src="uploads/2009/06/bolsena-2009-thumb-300x225.jpg" alt="bolsena-2009-thumb" width="300" height="225" srcset="https://justobjects.nl/wp-content/uploads/2009/06/bolsena-2009-thumb-300x225.jpg 300w, https://justobjects.nl/wp-content/uploads/2009/06/bolsena-2009-thumb-200x150.jpg 200w, https://justobjects.nl/wp-content/uploads/2009/06/bolsena-2009-thumb-150x112.jpg 150w, https://justobjects.nl/wp-content/uploads/2009/06/bolsena-2009-thumb.jpg 400w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2009/06/bolsena-2009-thumb-300x225.jpg"
alt="" />
</a>
</div>
Last week I attended the <a href="http://wiki.osgeo.org/wiki/OSGeo_Hacking_Event_2009" target="_blank" rel="noreferrer noopener">OSGeo Hacking event in Bolsena, Italy</a>
. For a week developers from various Free/Open Source Geo-projects (mostly <a href="http://www.osgeo.org" target="_blank" rel="noreferrer noopener">OSGeo</a>
projects) get together in a monastry near Bolsena for code sprints, cross-project discussions, presentations, enjoying italian food and more fun. I have been working on the <a href="http://geonetwork-opensource.org" target="_blank" rel="noreferrer noopener">GeoNetwork</a>
project some time, so it was good to meet and work with the GN team. Meeting project leads and developers and following presentations from other projects like <a href="http://geoserver.org" target="_blank" rel="noreferrer noopener">GeoServer</a>
, <a href="http://deegree.org" target="_blank" rel="noreferrer noopener">Deegree</a>
and <a href="http://mapbender.org" target="_blank" rel="noreferrer noopener">MapBender</a>
was really worthwhile. To get an impression you can watch the media mix I made from various photo’s/videos made by me and other participants.</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/Mxox3AsjH6Q?controls=1&rel=0" loading="lazy"></iframe>
</div>
-
Tracking Frisian Solar Challenge 2008
https://justobjects.nl/tracking-frisian-solar-challenge/
Sat, 28 Jun 2008 11:25:14 +0000
https://justobjects.nl/tracking-frisian-solar-challenge/
<!-- <img loading="lazy" class="alignleft wp-image-269 size-medium" src="uploads/2008/06/frisian-solar-challenge-2009-300x108.jpg" alt="frisian-solar-challenge-2009" width="300" height="108" srcset="https://justobjects.nl/wp-content/uploads/2008/06/frisian-solar-challenge-2009-300x108.jpg 300w, https://justobjects.nl/wp-content/uploads/2008/06/frisian-solar-challenge-2009-250x90.jpg 250w, https://justobjects.nl/wp-content/uploads/2008/06/frisian-solar-challenge-2009-150x54.jpg 150w, https://justobjects.nl/wp-content/uploads/2008/06/frisian-solar-challenge-2009.jpg 515w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2008/06/frisian-solar-challenge-2009-300x108.jpg"
alt="" />
</a>
</div>
</p>
<p>Tracking the 2008-edition of the <a href="http://www.frisiansolarchallenge.nl/" target="_blank" rel="noreferrer noopener">Frisian Solar Challenge</a>
, a ‘World Cup for Solar Powered Boats’, with <a href="http://www.geosailing.com" target="_blank" rel="noreferrer noopener">GeoSailing</a>
. See <a href="http://www.geosailing.com/fsc08" target="_blank" rel="noreferrer noopener">www.geosailing.com/fsc08</a>
.</p>
<p> </p>
<p>Below random images from the race.</p>
<!--
<table border="0" cellspacing="0" cellpadding="8">
<tr>
<td>
<a href="http://www.geosailing.com/fsc08" target="_new"><img src="http://www.geosailing.com/fsc08/media.srv?id=*&resize=240x180" alt="random image" border="0" /></a>
</td>
<td>
<a href="http://www.geosailing.com/fsc08" target="_new"><img src="http://www.geosailing.com/fsc08/media.srv?id=*&resize=241x181" alt="random image" border="0" /></a>
</td>
</tr>
</table>
-->
<table>
<thead>
<tr>
<th style="text-align:center"><img src="http://www.geosailing.com/fsc08/media.srv?id=*&resize=240x180" alt=" "></th>
<th style="text-align:center"><img src="http://www.geosailing.com/fsc08/media.srv?id=*&resize=241x181" alt=" "></th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center"><a href="http://www.geosailing.com/fsc08" target="_blank" rel="noreferrer noopener">random image</a>
</td>
<td style="text-align:center"><a href="http://www.geosailing.com/fsc08" target="_blank" rel="noreferrer noopener">random image</a>
</td>
</tr>
</tbody>
</table>
-
Waag/7Scenes Launches GamesAtelier
https://justobjects.nl/gamestudio-launched/
Mon, 17 Mar 2008 14:18:12 +0000
https://justobjects.nl/gamestudio-launched/
<p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2008/03/gamesatelier-just-300x217.jpg"
alt="" />
</a>
</div>
</p>
<div class="float_right">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2008/03/gamesatelier-sara-300x225.jpg"
alt="" />
</a>
</div>
<!-- <img loading="lazy" class="alignleft wp-image-275 size-medium" src="uploads/2008/03/gamesatelier-just-300x217.jpg" alt="gamesatelier-just" width="300" height="217" srcset="https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-just-300x217.jpg 300w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-just-206x150.jpg 206w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-just-150x108.jpg 150w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-just.jpg 640w" sizes="(max-width: 300px) 100vw, 300px" />
<img loading="lazy" class="alignright wp-image-274 size-medium" src="uploads/2008/03/gamesatelier-sara-300x225.jpg" alt="gamesatelier-sara" width="300" height="225" srcset="https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-sara-300x225.jpg 300w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-sara-199x150.jpg 199w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-sara-150x112.jpg 150w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-sara.jpg 306w" sizes="(max-width: 300px) 100vw, 300px" />] -->
<p>On friday march 14, 2008 Mayor <a href="http://nl.wikipedia.org/wiki/Job_Cohen" target="_blank" rel="noreferrer noopener">Job Cohen</a>
of Amsterdam was the first player of the <a href="http://7scenes.com/games_atelier" target="_blank" rel="noreferrer noopener">GamesAtelier</a>
product that <a href="http://www.waag.org" target="_blank" rel="noreferrer noopener">Waag Society</a>
launched. We at <a href="http://7scenes.com" target="_blank" rel="noreferrer noopener">7Scenes</a>
have worked hard to get the software done.</p>
<p> </p>
<!-- <img loading="lazy" class="alignleft wp-image-273 size-medium" src="uploads/2008/03/gamesatelier-cohen-300x215.jpg" alt="gamesatelier-cohen" width="300" height="215" srcset="https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-cohen-300x215.jpg 300w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-cohen-208x150.jpg 208w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-cohen-150x107.jpg 150w, https://justobjects.nl/wp-content/uploads/2008/03/gamesatelier-cohen.jpg 319w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2008/03/gamesatelier-cohen.jpg"
alt="" />
</a>
</div>
<p> </p>
<p>Links<br>
<a href="http://www.gamesatelier.nl" target="_blank" rel="noreferrer noopener">www.gamesatelier.nl</a>
<br>
<a href="http://www.amsterdam.nl/nieuws?ActItmIdt=117909" target="_blank" rel="noreferrer noopener">Games Atelier officieel van start</a>
<br>
<a href="http://archief.parool.nl/artikel?text=frenk&SORT=date&ED=ola&PRD=2y&SEC=%2A&FDOC=0" target="_blank" rel="noreferrer noopener">parool.nl</a>
</p>
<p> </p>
<p>More photos here: <a href="http://gallery.waag.org/gamesatelier">http://gallery.waag.org/gamesatelier</a></p>
-
Making a Mobile Game at Picnic 2007
https://justobjects.nl/making-a-mobile-game-at-picnic-07/
Wed, 03 Oct 2007 14:15:08 +0000
https://justobjects.nl/making-a-mobile-game-at-picnic-07/
<!--
<img loading="lazy" class="alignleft wp-image-279 size-medium" src="uploads/2007/10/walkandplay-thumb-300x242.jpg" alt="walkandplay-thumb" width="300" height="242" srcset="https://justobjects.nl/wp-content/uploads/2007/10/walkandplay-thumb-300x242.jpg 300w, https://justobjects.nl/wp-content/uploads/2007/10/walkandplay-thumb-185x150.jpg 185w, https://justobjects.nl/wp-content/uploads/2007/10/walkandplay-thumb-150x121.jpg 150w, https://justobjects.nl/wp-content/uploads/2007/10/walkandplay-thumb.jpg 428w" sizes="(max-width: 300px) 100vw, 300px" />
-->
<p> </p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2007/10/walkandplay-thumb-300x242.jpg"
alt="" />
</a>
</div>
<p> </p>
<p>At <a href="http://www.picnicnetwork.org" target="_blank" rel="noreferrer noopener">Picnic 07</a>
we held 2 workshops using our <a href="http://walkandplay.com" target="_blank" rel="noreferrer noopener">WalkAndPlay mobile gamekit</a>
(based on <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
). Here participants can Create, Play and View a GPS-based locative mobile game.</p>
<p>Below is one of the results from the <a href="http://www.picnicnetwork.org/article-7287-en.html" target="_blank" rel="noreferrer noopener">Picnic Junior Workshop for highschools</a>
. You may also want to view the <a href="http://extern.waag.org/just/7scenes/walknplay-picnicjr.mov" target="_blank" rel="noreferrer noopener">high resolution movie</a>
.</p>
<p> </p>
<div class="video">
<iframe src="https://www.youtube.com/embed/4-_HOk_83s4?controls=1&rel=0" loading="lazy"></iframe>
</div>
<p>This work will be continued at <a href="http://7scenes.com" target="_blank" rel="noreferrer noopener">7scenes</a>
(more info soon).</p>
-
Tracking Schuttevaer Sailing Race 2007
https://justobjects.nl/geosailing-live-schuttevaer/
Fri, 08 Jun 2007 19:51:17 +0000
https://justobjects.nl/geosailing-live-schuttevaer/
<!-- <img loading="lazy" class="alignleft wp-image-290 size-medium" src="uploads/2007/06/schuttevaer2-300x121.jpg" alt="schuttevaer2" width="300" height="121" srcset="https://justobjects.nl/wp-content/uploads/2007/06/schuttevaer2-300x121.jpg 300w, https://justobjects.nl/wp-content/uploads/2007/06/schuttevaer2-250x101.jpg 250w, https://justobjects.nl/wp-content/uploads/2007/06/schuttevaer2-150x60.jpg 150w, https://justobjects.nl/wp-content/uploads/2007/06/schuttevaer2.jpg 527w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2007/06/schuttevaer2-300x121.jpg"
alt="" />
</a>
</div>
</p>
<p>During June 7-9, 2007 the <a href="http://www.schuttevaerrace.nl" target="_blank" rel="noreferrer noopener">Dutch Schuttevaerrace</a>
was <a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">covered live</a>
using Just Objects’ <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
platform in combination with <a href="http://www.falcom.de/index.php?id=801" target="_blank" rel="noreferrer noopener">Falcom Mambo GPS/GPRS tracking devices</a>
attached onboard with 15 boats. I did this project jointly with the <a href="http://www.kenniscentrumjachtbouw.nl" target="_blank" rel="noreferrer noopener">Kenniscentrum Jachtbouw</a>
(Dennis Carton, project initiator), <a href="http://www.grrr.nl" target="_blank" rel="noreferrer noopener">Grrr</a>
(Jelmer Boomsma and Rolf Coppens) and the Internet Acedemy of the <a href="http://www.nhl.nl" target="_blank" rel="noreferrer noopener">Noordelijke Hogeschool Leeuwarden</a>
(Frederik van der Meulen, Ronald Klooster).</p>
<p>Except with some GPS-fixing and GPRS coverage issues and this event was a huge success.</p>
<p>(Random images taken by sailors from <a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">www.geosailing.com/svr</a>
)</p>
<!-- <table border="0" cellspacing="0" cellpadding="8">
<tr>
<td>
<a href="http://www.geosailing.com/svr" target="_new"><img src="http://www.geosailing.com/svr/srv/randomimage.jsp?resize=240x180" alt="random image" border="0" /></a>
</td>
<td>
<a href="http://www.geosailing.com/svr" target="_new"><img src="http://www.geosailing.com/svr/srv/randomimage.jsp?n=1&resize=240x180" alt="random image" border="0" /></a>
</td>
</tr>
</table> -->
<table>
<thead>
<tr>
<th style="text-align:center"><img src="http://www.geosailing.com/svr/srv/randomimage.jsp?resize=240x180" alt=" "></th>
<th style="text-align:center"><img src="http://www.geosailing.com/svr/srv/randomimage.jsp?n=1&resize=240x180" alt=" "></th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center"><a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">random image</a>
</td>
<td style="text-align:center"><a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">random image</a>
</td>
</tr>
</tbody>
</table>
<p>Some screenshots.</p>
<!-- <table border="0" cellspacing="0" cellpadding="8">
<tr>
<td>
<a href="http://www.geosailing.com/svr" target="_new"><img src="http://www.geosailing.com/svr/media.srv?id=1441&resize=240x180!" alt="screenshot image" border="0" /></a><br /> The Start
</td>
<td>
<a href="http://www.geosailing.com/svr" target="_new"><img src="http://www.geosailing.com/svr/media.srv?id=1446&resize=240x180!" alt="screenshot image" border="0" /></a><br /> Routes and Photo’s
</td>
</tr>
</table> -->
<table>
<thead>
<tr>
<th style="text-align:center"><img src="http://www.geosailing.com/svr/media.srv?id=1441&resize=240x180!" alt=" "></th>
<th style="text-align:center"><img src="http://www.geosailing.com/svr/media.srv?id=1446&resize=240x180!" alt=" "></th>
</tr>
</thead>
<tbody>
<tr>
<td style="text-align:center"><a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">The Start</a>
</td>
<td style="text-align:center"><a href="http://www.geosailing.com/svr" target="_blank" rel="noreferrer noopener">Routes and Photo’s</a>
</td>
</tr>
</tbody>
</table>
<p>Read more on <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">www.geosailing.com</a>
.</p>
-
Waag Society Sarai Workshop with GeoTracing
https://justobjects.nl/waag-society-sarai-workshop-with-geotracing/
Mon, 02 Apr 2007 07:05:26 +0000
https://justobjects.nl/waag-society-sarai-workshop-with-geotracing/
<!-- <img loading="lazy" class="alignleft wp-image-293 size-medium" src="uploads/2007/04/waag-sarai1-map-300x205.jpg" alt="waag-sarai1-map" width="300" height="205" srcset="https://justobjects.nl/wp-content/uploads/2007/04/waag-sarai1-map-300x205.jpg 300w, https://justobjects.nl/wp-content/uploads/2007/04/waag-sarai1-map-218x150.jpg 218w, https://justobjects.nl/wp-content/uploads/2007/04/waag-sarai1-map-150x102.jpg 150w, https://justobjects.nl/wp-content/uploads/2007/04/waag-sarai1-map.jpg 587w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2007/04/waag-sarai1-map-300x205.jpg"
alt="" />
</a>
</div>
</p>
<p> </p>
<p><a href="http://www.waag.org" target="_blank" rel="noreferrer noopener">Waag Society</a>
held a Locative Media workshop @ Sarai Media lab, Delhi, India using <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
. View the video for an impression.</p>
<p> <br>
</p>
<div class="video">
<iframe src="https://www.youtube.com/embed/KSReeo4fhm4?controls=1&rel=0" loading="lazy"></iframe>
</div>
-
Bliin
https://justobjects.nl/bliin/
Mon, 20 Nov 2006 09:07:15 +0000
https://justobjects.nl/bliin/
<p><img src="https://justobjects.nl/uploads/2006/11/blinn-full.jpg" alt=" "></p>
<p>Users create ‘bliins’ to navigate and monitor their interests in a location or area. bliins can be saved and shared amongst users. Another <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
project:</p>
<p><em>“<a href="http://bliin.com" target="_blank" rel="noreferrer noopener">bliin</a>
is a social networking service where users can spot, trace and share experiences — pictures, videos, audio and text — with one another in real-time on a Google Map.</em></p>
<p><em><a href="http://bliin.com" target="_blank" rel="noreferrer noopener">bliin</a>
is available for Desktop & Pocket internet.</em></p>
<p><em><a href="http://bliin.com" target="_blank" rel="noreferrer noopener">bliin</a>
is currently testing in closed beta. Users you see are live and media is uploaded in real-time.”</em></p>
<p>Read all on <a href="http://bliin.com" target="_blank" rel="noreferrer noopener">bliin.com</a>
. While still in alpha-stage <a href="http://bliin.com" target="_blank" rel="noreferrer noopener">bliin</a>
already attracted press and 1000-s of visitors. Below some links to various press articles and blogs.</p>
<p><a href="http://www.rtl.nl/%28/actueel/editienl/%29/components/actueel/editienl/2006/44/bliin.xml" target="_blank" rel="noreferrer noopener">“Altijd te vinden via GPS”</a>
, Editie NL, RTL 4, David Behrens, 10-30-2006<br>
<a href="http://www.bnr.nl/Weblog.asp?WeblogId=16" target="_blank" rel="noreferrer noopener">“Digitale pioniers”</a>
, Elektronische Eeuw, BNR Nieuwsradio, Herbert Blankesteijn, 10-26-2006<br>
<a href="http://cgi.omroep.nl/cgi-bin/streams?/radio1/tros/radioonline/20061024.wma?start=48:15" target="_blank" rel="noreferrer noopener">“Op bliin kan je jezelf volgbaar maken…”</a>
Radio Online, Tros, Francisco van Jole, 10-24-2006<br>
<a href="http://www.yme.nl/ymerce/2006/10/digitale-pioniers.html" target="_blank" rel="noreferrer noopener">“Digitale pioniers?”</a>
, Ymerce, Yme Bosma, 10-23-2006<br>
<a href="http://www.digitalepioniers.nl/Academy" target="_blank" rel="noreferrer noopener">“Woophy, bliin en Bugpool winnaars…”</a>
, Digitale Pioniers Academy, Syb Groeneveld, 10-23-2006<br>
<a href="http://www.bright.nl/drie-digitale-pioniers-krijgen-25-duizend-euro" target="_blank" rel="noreferrer noopener">“Drie digitale pioniers krijgen 25 duizend euro”</a>
, Bright, Tonie van Ringelestijn, 10-23-2006<br>
<a href="http://gnispen.blogspot.com/2006/10/final-part-of-digitale-pioniers.html" target="_blank" rel="noreferrer noopener">“Digitale pioniers academie, de uitreiking”</a>
, SUbWAY, 10-21-2006<br>
<a href="http://gnispen.blogspot.com/2006/10/final-part-of-digitale-pioniers.html" target="_blank" rel="noreferrer noopener">“Final Part of Digital Pioneers Academy”</a>
, gnispen World in Pictures, Guido van Nispen, 10-21-2006<br>
<a href="http://radar.oreilly.com/brady/" target="_blank" rel="noreferrer noopener">“GeoTracing”</a>
, O’Reilly Radar, Brady Forrest, 9-14-2006</p>
-
Sense of Brainport
https://justobjects.nl/sense-of-brainport/
Mon, 20 Nov 2006 08:29:45 +0000
https://justobjects.nl/sense-of-brainport/
<!-- <img loading="lazy" class="alignleft wp-image-298 size-large" src="uploads/2006/11/senseofbrainport-1024x612.jpg" alt="senseofbrainport" width="820" height="490" srcset="https://justobjects.nl/wp-content/uploads/2006/11/senseofbrainport-1024x612.jpg 1024w, https://justobjects.nl/wp-content/uploads/2006/11/senseofbrainport-300x179.jpg 300w, https://justobjects.nl/wp-content/uploads/2006/11/senseofbrainport-250x150.jpg 250w, https://justobjects.nl/wp-content/uploads/2006/11/senseofbrainport-150x89.jpg 150w, https://justobjects.nl/wp-content/uploads/2006/11/senseofbrainport.jpg 1075w" sizes="(max-width: 820px) 100vw, 820px" /> -->
<p><img src="https://justobjects.nl/uploads/2006/11/senseofbrainport.jpg" alt=" "></p>
<p><a href="http://www.senseofbrainport.nl" target="_blank" rel="noreferrer noopener">Sense of Brainport</a>
again a <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing project</a>
was launched on September 21, 2006. During one week <a href="http://www.senseofthecity.nl/sobp/deelnemers.jsp" target="_blank" rel="noreferrer noopener">participants</a>
from <a href="http://www.brainport.nl" target="_blank" rel="noreferrer noopener">Brainport</a>
were “geotracing” the region of Eindhoven/”Zuid Oost Brabant” with a mobile phone and a GPS.</p>
<p>During the event they could be traced <a href="http://www.senseofthecity.nl/sobp/map.jsp?cmd=live" target="_blank" rel="noreferrer noopener">in real-time on the map</a>
. While tracing the tracers are posting photos of their impressions. These photo’s are geotagged at the location taken. All their routes are stored in an archive for <a href="http://www.senseofthecity.nl/sobp/map.jsp" target="_blank" rel="noreferrer noopener">viewing and playback</a>
.</p>
-
Sense of the City
https://justobjects.nl/sense-of-the-city/
Tue, 25 Apr 2006 08:11:15 +0000
https://justobjects.nl/sense-of-the-city/
<!-- <img loading="lazy" class="alignleft wp-image-300 size-medium" src="uploads/2006/04/sotce-300x236.jpg" alt="sotce" width="300" height="236" srcset="https://justobjects.nl/wp-content/uploads/2006/04/sotce-300x236.jpg 300w, https://justobjects.nl/wp-content/uploads/2006/04/sotce-190x150.jpg 190w, https://justobjects.nl/wp-content/uploads/2006/04/sotce-150x118.jpg 150w, https://justobjects.nl/wp-content/uploads/2006/04/sotce.jpg 381w" sizes="(max-width: 300px) 100vw, 300px" /> -->
<p> <br>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2006/04/sotce-300x236.jpg"
alt="" />
</a>
</div>
</p>
<p><a href="http://www.senseofthecity.nl" target="_blank" rel="noreferrer noopener">Sense of the City</a>
the latest <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing project</a>
was launched on april 21, 2006. During the last week of april 2006, ten civilians are “geotracing” the Dutch city of Eindhoven with a mobile phone and a GPS. During the event they can be traced <a href="http://www.senseofthecity.nl/sotce/sotc.jsp?cmd=live" target="_blank" rel="noreferrer noopener">in real-time on the map</a>
.</p>
<p>While tracing the tracers are posting photos of their impressions. These photo’s are geotagged at the location taken. All their routes are stored in an archive for <a href="http://www.senseofthecity.nl/sotce/sotc.jsp?cmd=staalkaart" target="_blank" rel="noreferrer noopener">viewing and playback.</a>
</p>
-
GeoDrawing in the Night
https://justobjects.nl/geodrawing-in-the-night/
Tue, 08 Nov 2005 09:53:20 +0000
https://justobjects.nl/geodrawing-in-the-night/
<p><img src="https://justobjects.nl/uploads/2005/11/n8spel1.jpg" alt=" "></p>
<p>Again I had the opportunity to do a fun and technically challenging geo-project using my <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
platform: developing a <a href="http://www.n8spel.nl" target="_blank" rel="noreferrer noopener">GPS-based mobile drawing game</a>
for the <a href="http://www.n8.nl" target="_blank" rel="noreferrer noopener">Amsterdam Museum Night</a>
. Teams would go into the city where they compete on who would (geo)draw the most beautiful “8” by walking with a GPS and a mobile phone. They could embellish their drawings with photo’s and video’s taken and submitted on the spot. The competitive element was creativity with both the drawing and the media. All submitted media were tagged to the geographic locations where they were taken. The player’s movements, tracks and media could be followed in real-time through a webbrowser. You can view <a href="http://www.bright.nl/node/257" target="_blank" rel="noreferrer noopener">a report with video made by Bright magazine</a>
.</p>
<p><img src="https://justobjects.nl/uploads/2005/11/n8spel2.jpg" alt=" "></p>
<p>The N8-game application was developed in about one month by two developers. It consisted of three main components: (1) the server (2) mobile clients and (3) a web-browser front end.</p>
<p>The <strong><em>mobile client</em></strong> is a Java J2ME application (Midlet) running on a Nokia 6600 communicating with a <a href="http://www.holux.com.tw/Temp%20web/GR-236.htm" target="_blank" rel="noreferrer noopener">Holux GPSlim Bluetooth Sirf III GPS module</a>
. The main function of this app is to sample GPS data and transmit it to the server. Players could indicate with a button push to start (“pen-down”) and stop drawing (“pen-up”). Media were captured using the standard camera application on a Sony Z1010 phone. Media was submitted by email. The phone’s email adress is coupled to each team.</p>
<p>The <strong><em>server</em></strong> utilized the GeoTracing server without any modification. Basically a GeoTracing server functions as a remote GPS track-logger coupled to a Content Management System. GeoTracing is based on <a href="http://www.keyworx.org" target="_blank" rel="noreferrer noopener">KeyWorx</a>
. Incoming media are tied to a player (tracer) and a tracklog and a geographic location using the date in the medium (e.g. EXIF date) or the email submit time. The GeoTracing server provides a <a href="http://en.wikipedia.org/wiki/Representational_State_Transfer" target="_blank" rel="noreferrer noopener">REST</a>
service for clients to obtain tracklog meta information, (converted) media and tracklog data in an extended <a href="http://www.topografix.com/gpx.asp" target="_blank" rel="noreferrer noopener">GPX format</a>
. This also facilitates coupling with <a href="http://en.wikipedia.org/wiki/AJAX" target="_blank" rel="noreferrer noopener">AJAX</a>
browser-technology (see below). The server also pushes live events like user movements and other tracklog events through <a href="http://www.pushlets.com" target="_blank" rel="noreferrer noopener">Pushlets</a>
.</p>
<p>The <strong><em>web-browser front-end</em></strong> was written using pure DHTML with <a href="http://www.google.com/apis/maps" target="_blank" rel="noreferrer noopener">Google Maps</a>
, <a href="http://en.wikipedia.org/wiki/AJAX" target="_blank" rel="noreferrer noopener">AJAX</a>
and <a href="http://www.pushlets.com" target="_blank" rel="noreferrer noopener">a Pushlet client</a>
.<br>
Through Pushlets the browser receives real-time events like player movements and incoming media. Using the server REST service with AJAX player and tracklog info is obtained. Conceptually the browser-server interaction follows a distributed <a href="http://en.wikipedia.org/wiki/Model_view_controller" target="_blank" rel="noreferrer noopener">Model-View-Controller pattern</a>
with the Model on the server, the events to the View (browser) transmitted with Pushlets and the Controller function using AJAX.</p>
-
Google Maps Hacking is Fun
https://justobjects.nl/?p=15/
Fri, 16 Sep 2005 15:59:58 +0000
https://justobjects.nl/?p=15/
<p>(The quick link for my experiments is <a href="http://www.geoskating.com/gmap" target="_blank" rel="noreferrer noopener">www.geoskating.com/gmap</a>
.)</p>
<p>Just a week ago I learned about the <a href="http://www.google.com/apis/maps" target="_blank" rel="noreferrer noopener">Google Maps JavaScript API</a>
. Surprising how easy it was to use and build upon. Especially for my <a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating project</a>
I needed a more flexible way to display routes and media on a map. So I started experimenting with the Google Maps API. In less then 5 minutes I was able to create a basic map. But I needed more. Based on a <a href="http://www.topografix.com/gpx.asp" target="_blank" rel="noreferrer noopener">GPX (GPS track format)</a>
player from <a href="http://jibbering.com" target="_blank" rel="noreferrer noopener">Jim Ley</a>
I built a <a href="http://www.geoskating.com/gs/player/trackplayer.jsp" target="_blank" rel="noreferrer noopener">TrackPlayer to play back skate routes</a>
. In addition the <a href="http://gmaps.tommangan.us/tlabel.html" target="_blank" rel="noreferrer noopener">TLabel lib</a>
allows you to overlay any HTML on a Google Map.</p>
<p><strong>Adding layers from any WMS server</strong></p>
<p>Many map servers use a standard URL-pattern based on the <a href="http://www.opengeospatial.org" target="_blank" rel="noreferrer noopener">Web Map Server (WMS)</a>
standard.</p>
<p><a href="http://www.geoskating.com/gmap/gmap-wms-1.html" target="_blank" rel="noreferrer noopener"><img src="/assets/media/gmap-overlay.jpg" border="0" /></a>
</p>
<p>So I wanted more: adding my own layers integrated in the map preferably with transparency. Well, this is possible thanks to work by <a href="http://www.spatialdatalogic.com/cs/blogs/brian_flood/archive/2005/07/11/39.aspx" target="_blank" rel="noreferrer noopener">Brian Flood</a>
and<br>
<a href="http://blog.kylemulka.com/?p=287" target="_blank" rel="noreferrer noopener">Kyle Mulka</a>
. I have created a <a href="http://www.geoskating.com/gmap/gmap-wms.js" target="_blank" rel="noreferrer noopener">simple JavaScript<br>
library, gmap-wms.js</a>
through which you can add your own WMS layers to a Google Map. The example below<br>
is trivial using a single transparent GIF image<img src="http://www.geoskating.com/gmap/route-wms.jsp" border="0" /><br>
by faking a WMS server.</p>
<p>See all experiments at <a href="http://www.geoskating.com/gmap" target="_blank" rel="noreferrer noopener">www.geoskating.com/gmap</a>
</p>
-
Google Maps Hacking is Fun
https://justobjects.nl/google-maps-hacking-is-fun-2/
Fri, 16 Sep 2005 15:54:37 +0000
https://justobjects.nl/google-maps-hacking-is-fun-2/
<p>(The quick link for my experiments is <a href="http://www.geoskating.com/gmap" target="_blank" rel="noreferrer noopener">www.geoskating.com/gmap</a>
)</p>
<p>Just a week ago I learned about the <a href="http://www.google.com/apis/maps" target="_blank" rel="noreferrer noopener">Google Maps JavaScript API</a>
. Surprising how easy it was to use and build upon. Especially for my <a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating project</a>
I needed a more flexible way to display routes and media on a map. So I started experimenting with the Google Maps API. In less then 5 minutes I was able to <a href="http://www.geoskating.com/gmap/gmap.html" target="_blank" rel="noreferrer noopener">create a basic map</a>
. But I needed more. Based on a <a href="http://www.topografix.com/gpx.asp" target="_blank" rel="noreferrer noopener">GPX (GPS track format)</a>
player from <a href="http://jibbering.com" target="_blank" rel="noreferrer noopener">Jim Ley</a>
I built a <a href="http://www.geoskating.com/gs/player/trackplayer.jsp" target="_blank" rel="noreferrer noopener">TrackPlayer to play back skate routes</a>
. In addition the <a href="http://gmaps.tommangan.us/tlabel.html" target="_blank" rel="noreferrer noopener">TLabel lib</a>
allows you to overlay any HTML on a Google Map. Note: also check out <a href="http://ka-map.maptools.org/" target="_blank" rel="noreferrer noopener">ka-Map</a>
. With ka-Map you can do similar things plus it is open source.</p>
<p><strong>Adding layers from any WMS server</strong></p>
<p>Many map servers use a standard URL-pattern based on the <a href="http://www.opengeospatial.org" target="_blank" rel="noreferrer noopener">Web Map Server (WMS)</a>
standard.</p>
<p><img src="https://justobjects.nl/uploads/2005/09/gmap-overlay.jpg" alt=" "></p>
<p>So I wanted more: adding my own layers integrated in the map preferably with transparency. Well, this is possible thanks to work by <a href="http://www.spatialdatalogic.com/cs/blogs/brian_flood/archive/2005/07/11/39.aspx" target="_blank" rel="noreferrer noopener">Brian Flood</a>
and <a href="http://blog.kylemulka.com/?p=287" target="_blank" rel="noreferrer noopener">Kyle Mulka</a>
. I have created a <a href="http://www.geoskating.com/gmap/gmap-wms.js" target="_blank" rel="noreferrer noopener">simple JavaScript library, gmap-wms.js</a>
through which you can add your own WMS layers to a Google Map. The example above is trivial using a single transparent GIF image by faking a WMS server. All Google Maps does is requesting tiles from your WMS server (a lot of them!). In reality you will be running your own WMS server like <a href="http://mapserver.gis.umn.edu/" target="_blank" rel="noreferrer noopener">MapServer</a>
.</p>
<p>See all experiments at <a href="http://www.geoskating.com/gmap" target="_blank" rel="noreferrer noopener">www.geoskating.com/gmap</a>
</p>
<p><img src="http://www.geoskating.com/gmap/route-wms.jsp" alt=" "></p>
-
From GeoSkating and GeoSailing to GeoTracing
https://justobjects.nl/from-geoskating-and-geosailing-to-geotracing/
Sun, 21 Aug 2005 11:00:50 +0000
https://justobjects.nl/from-geoskating-and-geosailing-to-geotracing/
<!-- <img loading="lazy" class="alignleft wp-image-311 size-full" src="uploads/2005/08/geosailing-logo.jpg" alt="geosailing-logo" width="240" height="50" srcset="https://justobjects.nl/wp-content/uploads/2005/08/geosailing-logo.jpg 240w, https://justobjects.nl/wp-content/uploads/2005/08/geosailing-logo-150x31.jpg 150w" sizes="(max-width: 240px) 100vw, 240px" /> -->
<p> <br>
<div class="float_left">
<a href="http://www.geosailing.com">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/08/geosailing-logo.jpg"
alt="" />
</a>
</div>
Opened an experimental site for <a href="http://www.geosailing.com" target="_blank" rel="noreferrer noopener">GeoSailing</a>
to cover the <a href="http://www.kustzeilers.nl/24uurs/index.html" target="_blank" rel="noreferrer noopener">24 Uurs Zeilrace</a>
, the largest Dutch annual sailing event. Some 800 ships are competing on the IJsselmeer, Markermeer and Waddenzee to cover as many miles as possible within a 24 hour period. This event is held on August 26 and 27, 2005. One of the competitors will be the Semper, sailed by Dennis,Peter, Rolf and Jelmer. If it all works out the Semper can be followed live, complete with media made and submitted on the spot.</p>
<p>Based on the concepts and software for <a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating</a>
and <a href="http://www.geosailing.com" target="_blank" rel="noreferrer noopener">GeoSailing</a>
I am working on a more generic framework for <a href="http://www.geotracing.com" target="_blank" rel="noreferrer noopener">GeoTracing</a>
. The key idea is to provide a customizable application for live tracing, annotated mapping, GPS-based digital storytelling and location-based media for any outdoors activity.</p>
-
GeoSkating in the News
https://justobjects.nl/geoskating-in-the-news/
Sat, 18 Jun 2005 11:45:42 +0000
https://justobjects.nl/geoskating-in-the-news/
<p>News travels fast on the Net. My website for the mobile GPS project <a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating</a>
went from 10s to 1000s of visitors a day when it entered the international blogsphere and the Dutch national newspaper <a href="https://files.justobjects.nl/doc/geoskating-volkskrant-2005.pdf" target="_blank" rel="noreferrer noopener">De Volkskrant</a>
. On Google “geoskating” <a href="http://www.google.com/search?q=geoskating" target="_blank" rel="noreferrer noopener">went from 10 hits to 16900 in a week.</a>
. I want to thank everyone for their interest, also for the many kind emails I received. It encourages me to proceed with the project.</p>
<div class="float_left w30 h230">
<a href="https://files.justobjects.nl/doc/geoskating-volkskrant-2005.pdf">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/artikel-vk-magazine-thumb-109x150.jpg"
alt="Volkskrant article (by Carel Helder)" />
</a>
</div>
<div class="float_left w30 h230">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/gs-stats-small-150x73.jpg"
alt="" />
</a>
</div>
<div class="float_left w30 h230">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/mobile-bulgaria-small-150x115.jpg"
alt="" />
</a>
</div>
<div class="float_left w30 h230">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/sbn-cover-tb-110x150.jpg"
alt="Skeeler Bond Nederland – Magazine – on the cover" />
</a>
</div>
<div class="float_left w30 h230">
<a href="http://we-make-money-not-art.com/archives/2005/06/geoskating-aims.php">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/we-make-money-not-art-tb-150x122.jpg"
alt="we make money not art" />
</a>
</div>
<div class="float_left w30 h230">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/www.100shiki-tb-150x120.jpg"
alt="" />
</a>
</div>
<div class="float_left w30 h230">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/06/www.engadget.com-tb-146x150.jpg"
alt="" />
</a>
</div>
<!--
<table border="0" cellspacing="4" cellpadding="4">
<tr>
<td>
<a href="https://files.justobjects.nl/doc/geoskating-volkskrant-2005.pdf"><br /> <img loading="lazy" class="alignnone size-full wp-image-315" src="uploads/2005/06/artikel-vk-magazine-thumb.jpg" alt="artikel-vk-magazine-thumb" width="180" height="247" srcset="https://justobjects.nl/wp-content/uploads/2005/06/artikel-vk-magazine-thumb.jpg 180w, https://justobjects.nl/wp-content/uploads/2005/06/artikel-vk-magazine-thumb-109x150.jpg 109w" sizes="(max-width: 180px) 100vw, 180px" /></a><br /> Volkskrant article (by Carel Helder).
</td>
<td>
<img loading="lazy" class="alignnone size-full wp-image-316" src="uploads/2005/06/gs-stats-small.jpg" alt="gs-stats-small" width="240" height="118" srcset="https://justobjects.nl/wp-content/uploads/2005/06/gs-stats-small.jpg 240w, https://justobjects.nl/wp-content/uploads/2005/06/gs-stats-small-150x73.jpg 150w" sizes="(max-width: 240px) 100vw, 240px" />
</td>
<td>
<img loading="lazy" class="alignnone size-medium wp-image-317" src="uploads/2005/06/mobile-bulgaria-small.jpg" alt="mobile-bulgaria-small" width="199" height="153" srcset="https://justobjects.nl/wp-content/uploads/2005/06/mobile-bulgaria-small.jpg 199w, https://justobjects.nl/wp-content/uploads/2005/06/mobile-bulgaria-small-195x150.jpg 195w, https://justobjects.nl/wp-content/uploads/2005/06/mobile-bulgaria-small-150x115.jpg 150w" sizes="(max-width: 199px) 100vw, 199px" />
</td>
</tr>
<tr>
<td>
<img loading="lazy" class="alignnone size-medium wp-image-318" src="uploads/2005/06/sbn-cover-tb.jpg" alt="sbn-cover-tb" width="199" height="271" srcset="https://justobjects.nl/wp-content/uploads/2005/06/sbn-cover-tb.jpg 199w, https://justobjects.nl/wp-content/uploads/2005/06/sbn-cover-tb-110x150.jpg 110w" sizes="(max-width: 199px) 100vw, 199px" /><br /> Skeeler Bond Nederland – Magazine – on the cover
</td>
<td>
<a href="http://we-make-money-not-art.com/archives/2005/06/geoskating-aims.php"><img loading="lazy" class="alignnone size-medium wp-image-319" src="uploads/2005/06/we-make-money-not-art-tb.jpg" alt="we-make-money-not-art-tb" width="199" height="163" srcset="https://justobjects.nl/wp-content/uploads/2005/06/we-make-money-not-art-tb.jpg 199w, https://justobjects.nl/wp-content/uploads/2005/06/we-make-money-not-art-tb-183x150.jpg 183w, https://justobjects.nl/wp-content/uploads/2005/06/we-make-money-not-art-tb-150x122.jpg 150w" sizes="(max-width: 199px) 100vw, 199px" /><br /> </a>we make money not art
</td>
<td>
<img loading="lazy" class="alignnone size-medium wp-image-320" src="uploads/2005/06/www.100shiki-tb.jpg" alt="www.100shiki-tb" width="200" height="161" srcset="https://justobjects.nl/wp-content/uploads/2005/06/www.100shiki-tb.jpg 200w, https://justobjects.nl/wp-content/uploads/2005/06/www.100shiki-tb-186x150.jpg 186w, https://justobjects.nl/wp-content/uploads/2005/06/www.100shiki-tb-150x120.jpg 150w" sizes="(max-width: 200px) 100vw, 200px" />
</td>
</tr>
<tr>
<td colspan="3">
<img loading="lazy" class="alignnone size-medium wp-image-321" src="uploads/2005/06/www.engadget.com-tb.jpg" alt="www.engadget.com-tb" width="200" height="205" srcset="https://justobjects.nl/wp-content/uploads/2005/06/www.engadget.com-tb.jpg 200w, https://justobjects.nl/wp-content/uploads/2005/06/www.engadget.com-tb-146x150.jpg 146w" sizes="(max-width: 200px) 100vw, 200px" />
</td>
</tr>
</table>
-->
-
GeoSkating – Draw maps while skating
https://justobjects.nl/geoskating-new-project/
Sun, 29 May 2005 15:08:13 +0000
https://justobjects.nl/geoskating-new-project/
<p><a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating</a>
is my latest project started in february 2005.</p>
<p><img src="https://justobjects.nl/uploads/2005/05/geoskating.jpg" alt=" "></p>
<p><a href="http://www.geoskating.com" target="_blank" rel="noreferrer noopener">GeoSkating</a>
aims to automate the generation of interactive annotated skate-maps by using the Global Positioning System (GPS), Mobile Phones and the Internet. The key idea is that while skating, GPS position data is being assembled and published to a server through a mobile phone. At the same time the skater can enrich the GPS data with road surface ratings and by submitting media items like pictures. The server will draw geographic maps showing road quality through colouring plus the submitted media on the GPS locations where they were captured. In addition, skaters can also be seen moving in real-time on the map while skating!</p>
<p>The technical setup is globally as follows. GPS data is sampled using a standard <a href="http://www.delorme.com/bluelogger" target="_blank" rel="noreferrer noopener">Bluetooth GPS module</a>
. This module communicates with a mobile phone, a <a href="http://www.nokia.com/nokia/0,4879,33210,00.html" target="_blank" rel="noreferrer noopener">Nokia 6600</a>
. On the mobile phone runs a small <a href="http://java.sun.com/j2me" target="_blank" rel="noreferrer noopener">Java (J2ME)</a>
program that reads the GPS data from the GPS module and sends it through the mobile data network (GPRS) to the geoskating.com server. The skater can enter the road quality as a number (1-5) on the phone keypad. The current quality is always added to each GPS sample sent to the server.</p>
-
MIDP 2 on Mac OS X is here !
https://justobjects.nl/midp-2-on-mac-os-x-is-here/
Fri, 11 Mar 2005 21:21:08 +0000
https://justobjects.nl/midp-2-on-mac-os-x-is-here/
<p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/03/bluetooth.gif"
alt="" />
</a>
</div>
Since Mac OS X is already my preferred platform for Java development, I was very pleased to experience that J2ME development for MIDP 2.0 has finally become reality. I can now develop, compile, verify, package, run, debug and deploy MIDP 2.0 MIDlets from within my Java IDE (<a href="http://www.jetbrains.com" target="_blank" rel="noreferrer noopener">IntelliJ IDEA</a>
). All thanks to <a href="http://www.mpowerplayer.com" target="_blank" rel="noreferrer noopener">Michael Powers mpowerplayer</a>
. Best way to start is to go to <a href="http://developer.mpowerplayer.com" target="_blank" rel="noreferrer noopener">developer.mpowerplayer.com</a>
and download the SDK. But there is more.</p>
<p>The <a href="http://www.mpowerplayer.com" target="_blank" rel="noreferrer noopener">Mpowerplayer</a>
offers the tools familiar to J2ME developers: MIDP2.0 jars, the preverify tool and a MIDP2.0 emulator. Additionally, if your Mac has <a href="http://www.bluetooth.org" target="_blank" rel="noreferrer noopener">Bluetooth</a>
support, you can quickly deploy your MIDLet using the OS X <a href="http://www.apple.com/bluetooth/" target="_blank" rel="noreferrer noopener">Bluetooth File Exchange</a>
. To automate deployment I wrote a one-line script <code>btsend.sh</code> that is called directly from within <a href="http://ant.apache.org" target="_blank" rel="noreferrer noopener">Ant</a>
:</p>
<div class="highlight"><pre class="chroma"><code class="language-bash" data-lang="bash"><span class="ln">1</span><span class="cp">#!/bin/sh
</span><span class="ln">2</span><span class="cp"></span>
<span class="ln">3</span>/usr/bin/open -a <span class="s2">"/Applications/Utilities/Bluetooth File Exchange.app"</span> <span class="nv">$1</span>
</code></pre></div><p>and then from within Ant</p>
<div class="highlight"><pre class="chroma"><code class="language-fallback" data-lang="fallback"><span class="ln">1</span><target name="deploy">
<span class="ln">2</span> <exec executable="${basedir}/btsend.sh">
<span class="ln">3</span> <arg line="${myproject.jar}"/>
<span class="ln">4</span> </exec>
<span class="ln">5</span></target>
</code></pre></div><p>The only manual action is to select your mobile device from the Bluetooth File Exchange list of devices. On my Nokia 6600 phone I receive an incoming message and the installer is run.</p>
<p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/03/gps-earthmate.png"
alt="" />
</a>
</div>
If you are developing Bluetooth-based MIDlets using <a href="http://www.jcp.org/en/jsr/detail?id=82" target="_blank" rel="noreferrer noopener">the JSR-82 Bluetooth API</a>
you can additionally download, evaluate and acquire the <a href="http://www.avetana-gmbh.de/avetana-gmbh/produkte/jsr82.eng.xml" target="_blank" rel="noreferrer noopener">Avetana JSR-82 implementation for OS X</a>
. This allows you to fully test Bluetooth-based J2ME apps from within your IDE on Mac OS X. For example, I was able to connect and test to the <a href="http://www.delorme.com/bluelogger" target="_blank" rel="noreferrer noopener">Delorme Blue Logger GPS</a>
</p>
-
Frequency 1550 – cross-media gaming into medieval times
https://justobjects.nl/frequency-1550-cross-media-gaming-into-medieval-times/
Wed, 09 Feb 2005 00:39:32 +0000
https://justobjects.nl/frequency-1550-cross-media-gaming-into-medieval-times/
<p>
<div class="float_left">
<a href="http://www.computable.nl/artikels/archief5/d07hb5nh.htm">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/02/f1150-hq-small-200x150.jpg"
alt="" />
</a>
</div>
<a href="http://freq1550.waag.org" target="_blank" rel="noreferrer noopener">Frequency 1550</a>
is a multi-user city game using mobile phones and GPS-technology. The game provides a real-time, location-based experience, transporting players into medieval Amsterdam of 1550 via today’s most advanced personal medium: a UMTS mobile phone. It is the one of the most exciting projects I ever worked on!F ind more info in <a href="http://www.computable.nl/artikel/achtergrond/onderwijs/1415901/1277214/umtspilot-in-middeleeuws-amsterdam.html">an article in the Computable magazine</a> (Dutch).</p>
<p> </p>
<p>
<div class="float_right">
<a href="https://files.justobjects.nl/presentation/f1550tech/">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/02/f1150-infra-thumb-199x150.jpg"
alt="" />
</a>
</div>
<a href="http://connectmedia.waag.org/media/freq1550killerclub.mov" target="_blank" rel="noreferrer noopener">There is a project presentation in streaming video on connect.waag.org</a>
. There are also <a href="https://files.justobjects.nl/doc/jspring-2005.pdf" target="_blank" rel="noreferrer noopener">slides of presentations</a>
. I gave at the <a href="http://www.nljug.org/en/pages/events/content/jspring_2005/sessions/00001" target="_blank" rel="noreferrer noopener">Dutch Java User Group (NLJUG)</a>
and at the <a href="http://extern.waag.org/just/freq1550" target="_blank" rel="noreferrer noopener">Society for Old and New Media (KillerClub)</a>
.</p>
<p>The project even hit the national news ! Click <a href="https://files.justobjects.nl/doc/mg-journaal.html" target="_blank" rel="noreferrer noopener">on anchor-man Gijs</a>
to watch the Flash video.</p>
<p>
<div class="float_left">
<a href="https://files.justobjects.nl/doc/mg-journaal.html">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/02/f1150-journaal-thumb-201x150.jpg"
alt="" />
</a>
</div>
The game evolves around players being personages in an adventure where they help the “schout” (medieval police) who will contact them by video when entering zones or reaching (GPS) locations. Players will try to solve assignments given by the “schout” on the street or through one of their<br /> team members on the home base, HQ, (behind a Flash-based interface). Assignments are solved by making media (photo’s, videos) at the spot<br /> and submitting these. The HQ may consult internet for additional media and textual answers. Thus together, teams provide multi-media<br /> content, resulting in a media-collage at the end. Additional game elements include, GPS-based boobytrapping, confrontations and cloaking.</p>
<p>My task in this project was development of the server-side gaming engine, media management and location tracking. Yes, the server was done in Java using the <a href="http://www.keyworx.org" target="_blank" rel="noreferrer noopener">KeyWorx</a>
platform. Client software was done using the Java 2 Mobile Edition (J2ME) for the phones and Macromedia Flash for the HQ.</p>
<p>I am grateful to have been part of the team at <a href="http://www.waag.org" target="_blank" rel="noreferrer noopener">Waag Society</a>
who has initiated and developed this mobile learning game together with IVKO, part of the Montessori comprehensive school in Amsterdam. The project was supported by <a href="http://www.kpn-corporate.com/nl/pers/?id=2.02&taal=nl&pers_id=227" target="_blank" rel="noreferrer noopener">KPN</a>
who supplied mobile phones and access to their UMTS network. A pilot has just been conducted in 2005 from 7 to 9 February.</p>
-
Vertical centering with CSS
https://justobjects.nl/vertical-centering-with-css/
Wed, 19 Jan 2005 09:37:56 +0000
https://justobjects.nl/vertical-centering-with-css/
<p>After years of tweaking with HTML tables I finally found how to <a href="http://www.wpdfd.com/editorial/wpd0103.htm#toptip" target="_blank" rel="noreferrer noopener">vertical Centring with CSS</a>
. Found this through <a href="http://www.hicksdesign.co.uk/journal/30/vertical-centering-with-css" target="_blank" rel="noreferrer noopener">www.hicksdesign.co.uk</a>
.</p>
-
Steve Jobs @ MacExpo
https://justobjects.nl/steve-jobs-macexpo/
Wed, 12 Jan 2005 14:21:09 +0000
https://justobjects.nl/steve-jobs-macexpo/
<p>Just watched the <a href="http://stream.apple.akadns.net/" target="_blank" rel="noreferrer noopener">video stream</a>
of <a href="http://www.macworld.com/news/2005/01/11/keynote/index.php" target="_blank" rel="noreferrer noopener">Steve Jobs’ keynote address for Macworld Conference</a>
. What I found the most exciting announcement is the introduction of the <a href="http://www.apple.com/macmini/" target="_blank" rel="noreferrer noopener">Mac Mini</a>
, a barebone G4 for around EUR 500.</p>
-
New Linux Server
https://justobjects.nl/new-linux-server/
Tue, 11 Jan 2005 10:53:48 +0000
https://justobjects.nl/new-linux-server/
<!-- ![pundit 1][1] ![pundit 2][2] ![pundit 3][3] -->
<p><code>.pundit 1</code> <code>.pundit 2</code> <code>.pundit 3</code></p>
<p>Today finally switched over to my new Linux server, an <a href="http://usa.asus.com/products/desktop/pundit-r/overview.htm" target="_blank" rel="noreferrer noopener">Asus Pundit-R</a>
, running <a href="http://www.debian.org" target="_blank" rel="noreferrer noopener">Debian Sarge</a>
</p>
-
Modded XBOX becomes MediaBox
https://justobjects.nl/modded-xbox-becomes-mediabox/
Sun, 09 Jan 2005 20:23:58 +0000
https://justobjects.nl/modded-xbox-becomes-mediabox/
<p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/01/xbox.jpg"
alt="" />
</a>
</div>
I acquired an <a href="http://www.xbox.com" target="_blank" rel="noreferrer noopener">Xbox</a>
at <a href="http://www.dixons.nl" target="_blank" rel="noreferrer noopener">Dixons</a>
for EUR 199,99 (ok, leave the 1 ct to Bill I said at the counter). Sometime later I had an almost complete home entertainment center/multimedia jukebox playing all my audio files (mp3, wav, ogg,..) , shoutcast streams, video files (mpg, avi, divx, and even quicktime) and photo slideshows (with background music) with a remote control on the TV-set in the living room from my Linux server in the attic. Plus offcourse I can play regular audio/mp3/video CDs and <strong>region-free</strong> DVDs.</p>
<p>My original topic <a href="index.php?p=1">LinuxMMBox</a>
is now sort of obsolete, except maybe for the PVR option, but the <a href="http://dreamix.sourceforge.net" target="_blank" rel="noreferrer noopener">Dreamix Project</a>
is even working on Xbox PVR.</p>
<p>The Xbox out of the box is boring: you can play games, audio CDs and DVDs (<a href="http://www.xbox.com/system/DVD+Movie+Playback+Kit.htm" target="_blank" rel="noreferrer noopener">with remote control</a>
) bound to your region. I acquired the <a href="http://www.xbox.com/system/DVD+Movie+Playback+Kit.htm" target="_blank" rel="noreferrer noopener">remote control</a>
as well. Now comes the fun part: <a href="http://www.xbox-scene.com/articles/beginnersguide.php" target="_blank" rel="noreferrer noopener">the Xbox can be modified</a>
into basically a regular PC (not bad for EUR 200!) with multimedia capabilities and networking (network card is built-in).</p>
<h3 id="the-xbox-is-a-pc">The Xbox is a PC</h3>
<p>Although the XBox is a games console (and Microsoft have tried hard to hide the fact) it is essentially a standard computer. It has all the same major hardware components but costs less than a similar specification PC – the clearest differences lie only in the packaging and the software. The most important piece of software is a small bit of code that runs as soon as the machine turns on called the BIOS. This BIOS program is held in a memory chip soldered directly onto the board.</p>
<p>On startup the Xbox runs the Bios which tells it to look for a disc in the DVD drive. If one is found, the BIOS ensures it is either a Microsoft certified game or an audio CD. Assuming the disc is ok, the CD plays and all is well… (if you buy a <a href="http://www.xbox.com/system/DVD+Movie+Playback+Kit.htm" target="_blank" rel="noreferrer noopener">DVD dongle</a>
you can also play DVDs of your own region).</p>
<p>However, if the drive is empty or any other type of disc is found, it runs its own mini-OS known as the dashboard which lets you set the clock and look at save games but very little else. Now, imagine for a moment you can alter the XBox’s BIOS to let you play backups of your XBox games, DivX movies, MP3s, any region DVDs (without the dongle), play any arcade/SNES/megadrive/atari/etc games, play online without XBox Live and much MUCH more.</p>
<p>Put simply, you can! You need to replace the BIOS with something that is less picky about what it can run. Fortunately these BIOSes have been written for you, you just need to get them into your XBOX…</p>
<h3 id="my-setup">My Setup</h3>
<p>With my Xbox having been modded, my software setup is simple: as a “dashboard” (the program that starts when you power up the Xbox) I use the <a href="http://www.xboxmediaplayer.de" target="_blank" rel="noreferrer noopener">XBox Media Player (XBMP)</a>
,<br>
a simple and very easy-to-use media player. XBMP can play everything: audio (mp3, wav..), video (DivX, mpeg, QuickTime etc), images (JPEG,..) in slideshows (also with audio playing in the background!). Even Shoutcast channels (internet radio) can be played.</p>
<p>I keep all my media files on my Linux server. I use Samba shares and <a href="http://www.apachemp3.com/" target="_blank" rel="noreferrer noopener">Apache::MP3</a>
to play my MP3s anywhere. Now as not having to copy all my files to the Xbox, I run the <a href="http://www.xboxmediaplayer.de/" target="_blank" rel="noreferrer noopener">ccxstream server</a>
on my Linux server, such that these files can be accessed from the Xbox. The concept of ccxtream is similar<br>
to that of Samba. A regular cronjob extracts all top-Shoutcast channels.</p>
<h3 id="adding-pvr">Adding PVR</h3>
<p>The XBox itself cannot do PVR (Personal Video Recording) but I am planning a setup using <a href="http://www.elgato.com" target="_blank" rel="noreferrer noopener">EyeTV from ElGato</a>
on my Mac and saving the recordings on my Linux server, who in turn can serve them out to the Xbox as it does already. This setup is similar to <a href="http://www.jonsthoughtsoneverything.com/archives/000009.php" target="_blank" rel="noreferrer noopener">an article written by Jon</a>
.</p>
<p>Another option is to use an open-source digital video jukebox (PVR, DVR) based on Linux like <a href="http://freevo.sourceforge.net" target="_blank" rel="noreferrer noopener">Freevo</a>
, or <a href="http://www.mythtv.org" target="_blank" rel="noreferrer noopener">MythTV</a>
</p>
<h3 id="links">Links</h3>
<p>Try out these links. They give you ample info on how to get things done. Tip: try the Tutorials first on <a href="http://www.xbox-scene.com">http://www.xbox-scene.com</a></p>
<p><a href="http://www.xbox-scene.com">http://www.xbox-scene.com</a> – the main Xbox site to consult<br>
<a href="http://www.xboxmediaplayer.de">http://www.xboxmediaplayer.de</a> – the Xbox Media Player (XBMP) and streaming servers (ccxstream)<br>
<a href="http://dreamix.sourceforge.net">http://dreamix.sourceforge.net</a> – Dreamix Project<br>
<a href="http://freevo.sourceforge.net">http://freevo.sourceforge.net</a> – Freevo Project<br>
<a href="http://www.delgul.demon.nl/xbox/">http://www.delgul.demon.nl/xbox/</a> – HOWTO on getting Xebian, Freevo and the xbox remote running</p>
-
SMIL
https://justobjects.nl/h/
Thu, 06 Jan 2005 23:17:00 +0000
https://justobjects.nl/h/
<p>Investigated the <a href="https://www.w3.org/AudioVideo" target="_blank" rel="noreferrer noopener">W3C SMIL (Synchronized Multimedia Interchange Language)</a>
. Goal was to see what SMIL is and how it can be used in the <a href="https://www.keyworx.org" target="_blank" rel="noreferrer noopener">KeyWorX</a>
multi-user environment.</p>
<p>(from <a href="http://www.w3.org/AudioVideo">http://www.w3.org/AudioVideo</a>)<br>
The Synchronized Multimedia Integration Language (SMIL, pronounced "smile") enables simple authoring of interactive audiovisual presentations. SMIL is typically used for "rich media"/multimedia presentations which integrate streaming audio and video with images, text or any other media type. SMIL is an easy-to-learn HTML-like language, and many SMIL presentations are written using a simple text-editor.</p>
<p>Tutorials</p>
<ul>
<li><a href="http://www.xml.com/pub/a/2002/05/29/smil.html">http://www.xml.com/pub/a/2002/05/29/smil.html</a> – good starting point, create your first SMIL presentation in 21 minutes!</li>
<li><a href="http://www.helio.org/products/smil/tutorial/toc.html">http://www.helio.org/products/smil/tutorial/toc.html</a> – good</li>
<li><a href="http://www.cwi.nl/~media/SMIL/Tutorial/SMILTut.html">http://www.cwi.nl/~media/SMIL/Tutorial/SMILTut.html</a> – slides</li>
<li>do a <a href="https://www.google.com/search?q=SMIL+Tutorial" target="_blank" rel="noreferrer noopener">Google search on SMIL tutorial</a>
</li>
</ul>
-
Linux MediaBox
https://justobjects.nl/hello-world-2/
Thu, 06 Jan 2005 22:58:50 +0000
https://justobjects.nl/hello-world-2/
<p> </p>
<div class="float_left">
<a href="#ZgotmplZ">
<img class="lazy"
src="https://justobjects.nl/uploads/2005/01/xbox.jpg"
alt="" />
</a>
</div>
<p> </p>
<p>The stuff below was written in 2003. Much has happened since on what may be called the “Home Media Center”. For example Linux <a href="http://www.mythtv.org" target="_blank" rel="noreferrer noopener">Myth TV</a>
on a barebones PC like the Asus Pundit. The stuff below is for reference.</p>
<p> </p>
<h3 id="the-challenge">The Challenge</h3>
<p>Around the house I have numerous devices that each serve multimedia functions:</p>
<p><a href="http://www6.tomshardware.com/video/00q2/000418/index.html" target="_blank" rel="noreferrer noopener">Tom’s Hardware – good intro to TV cards<br>
</a>
<a href="http://www.tweakers.net/reviews/cat/66" target="_blank" rel="noreferrer noopener">Tweakers.net</a>
– Hauppauge WinTV Radio versus Miro PCTV Pro<br>
<a href="http://www.linuxgazette.com/issue62/silva.html" target="_blank" rel="noreferrer noopener">Linuxgazette</a>
– shows how to install a TV card + realproducer</p>
<h3 id="hauppauge">Hauppauge</h3>
<p>So far the <a href="http://www.hauppauge.com/html/wintvpvr_datasheet.htm" target="_blank" rel="noreferrer noopener">Hauppage WinTV PVR 350</a>
seems to match<br>
this feature set only the reviews are differing. But most importantly the<br>
<a href="http://sourceforge.net/projects/pvr" target="_blank" rel="noreferrer noopener">SourceForge project that develops PVR Linux drivers</a>
for its MPEG-2 capabilities seems to have been abandoned. UPDATE: <a href="http://ivtv.sourceforge.net/" target="_blank" rel="noreferrer noopener">A new? sourceforge project</a>
is going strongly. Capturing a video stream from the MPEG encoder seems to work.</p>
<h3 id="linux-media-labs">Linux Media Labs</h3>
<p><a href="http://www.linuxmedialabs.com/" target="_blank" rel="noreferrer noopener">Linux Media Labs</a>
has produced a couple of linux video capture cards.The LML33 is a hardware Motion JPEG capture card, and the LMLM4 does hardware MPEG-4 compression. They are a bit expensive and I’m not sure either of those cards has a bulid-in tuner.</p>
<h3 id="ati">ATI</h3>
<p>Need more research on ATI cards like the <a href="http://www.ati.com/products/pc/aiwradeon8500" target="_blank" rel="noreferrer noopener">ALL-IN-WONDER® RADEON 8500</a>
. It seems a bid odd to have an integrated video/TV card…<br>
<a href="http://gatos.sourceforge.net">http://gatos.sourceforge.net</a> – Linux drivers for TV</p>
<h3 id="the-search-for-a-silent-box">The Search for a Silent Box</h3>
<p><a href="http://www.tweakers.net/reviews/344">http://www.tweakers.net/reviews/344</a> – but still not really silent<br>
<a href="http://www.silentpcreview.com">http://www.silentpcreview.com</a> – Silent PC Review<br>
<a href="http://www.solarpc.com/20.htm">http://www.solarpc.com/20.htm</a> – Solar PC takes Notebook motherboards and puts them in 1U cases with fanless power supplies</p>
<h3 id="links">Links</h3>
<p><a href="http://www2.warehouse.com/product.asp?pf%5Fid=EX10893&cat=pc" target="_blank" rel="noreferrer noopener">AverMedia</a>
– USB TV card<br>
<a href="http://www.paranoyaxc.de/bungee/bungee.html" target="_blank" rel="noreferrer noopener">Bungee Command-Line Recorder/Player for Linux</a>
<br>
<a href="http://pvrusb.sourceforge.net">http://pvrusb.sourceforge.net</a> – an experimental Linux 2.4 driver for the Hauppauge WinTV PVR USB (NTSC only)<br>
<a href="http://www.mplayerhq.hu/homepage">http://www.mplayerhq.hu/homepage</a> – Movie Player for Linux, plays any format<br>
<a href="http://yolinux.com/TUTORIALS/LinuxTutorialVideo.html">http://yolinux.com/TUTORIALS/LinuxTutorialVideo.html</a> – YoLinux Tutorial: Video, DVD, TV and Multimedia<br>
<a href="http://bytesex.org">http://bytesex.org</a> – home of bttv, the device driver for bt848 / bt878 based grabber boards<br>
<a href="http://cserv.sourceforge.net">http://cserv.sourceforge.net</a> – Camserv, free program to do streaming video through the web</p>