<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://philenius.github.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://philenius.github.io/" rel="alternate" type="text/html" /><updated>2026-03-13T22:03:25+01:00</updated><id>https://philenius.github.io/feed.xml</id><title type="html">philenius</title><subtitle>Blog about scalable cloud setups, ML life-cycle tooling, and modern app stacks.</subtitle><entry><title type="html">How to Update Nextcloud Docker Installation to the Latest Version</title><link href="https://philenius.github.io/cloud/2022/11/13/how-to-updade-nextcloud-docker-installation-to-latest-version.html" rel="alternate" type="text/html" title="How to Update Nextcloud Docker Installation to the Latest Version" /><published>2022-11-13T13:00:00+01:00</published><updated>2022-11-13T13:00:00+01:00</updated><id>https://philenius.github.io/cloud/2022/11/13/how-to-updade-nextcloud-docker-installation-to-latest-version</id><content type="html" xml:base="https://philenius.github.io/cloud/2022/11/13/how-to-updade-nextcloud-docker-installation-to-latest-version.html"><![CDATA[<p>The developers of Nextcloud typically release <a href="https://docs.nextcloud.com/server/latest/admin_manual/release_schedule.html">three major releases per year</a>. With this release frequency, your current Nextcloud installation will quickly become obsolete. Luckily, updating Nextcloud is super easy. In this blog post, I will briefly describe how to update a Docker or Docker Compose based Nextcloud installation.</p>

<p>To learn how to set up Nextcloud with Docker and Docker Compose, see my previous blog post <a href="/cloud/2022/11/13/run-nextcloud-as-docker-container-with-docker-compose.html"><em>Run Nextcloud as Docker container with Docker Compose (updated in Nov 2022)</em></a></p>

<hr />

<h2 id="updating-nextcloud-to-the-latest-version">Updating Nextcloud to the latest version</h2>

<h4 id="1-things-to-consider">1. Things to consider:</h4>

<p>:warning: <strong>In case of data loss or corruption, I’d recommend to always maintain regular backups and make a fresh backup before every upgrade.</strong></p>

<p><a href="https://docs.nextcloud.com/server/latest/admin_manual/maintenance/upgrade.html">The Nextcloud documentation</a> lists two important rules when upgrading Nextcloud:</p>

<ol>
  <li>
    <p><strong>Before you can upgrade to the next major release, Nextcloud upgrades to the latest minor and patch release.</strong></p>
  </li>
  <li>
    <p><strong>You cannot skip major releases. Always increase the major version only one step at a time.</strong></p>
  </li>
</ol>

<h4 id="2-example-upgrade-procedure-in-theory">2. Example upgrade procedure in theory:</h4>

<p>Given these two rules, let me briefly describe the upgrade path if we were to start with a Nextcloud <code class="language-plaintext highlighter-rouge">23.0.9</code> installation with Docker Compose:</p>

<ol>
  <li>
    <p>Current installation <code class="language-plaintext highlighter-rouge">23.0.9</code></p>
  </li>
  <li>
    <p>The first step is to always upgrade to the latest minor and patch of your current major version. You can check for the latest available minor/patch version of Nextcloud on <a href="https://hub.docker.com/_/nextcloud/tags">Docker Hub</a>.<br />
:arrow_right: upgrade to latest minor/patch <code class="language-plaintext highlighter-rouge">23.0.11</code></p>
  </li>
  <li>
    <p>Now that we have upgrade to the latest minor and patch, we can bump our version to the <strong>next</strong> major version. Keep in mind that major releases must not be skipped. We must increase the major version only one step at a time. However, you can jump straight to the latest minor and patch version.<br />
:arrow_right: upgrade to the <strong>next</strong> major <code class="language-plaintext highlighter-rouge">24.0.7</code></p>
  </li>
  <li>
    <p>You can now repeatedly increase the major version until you reach the latest release of Nextcloud:<br />
:arrow_right: upgrade to the <strong>next</strong> major <code class="language-plaintext highlighter-rouge">25.0.1</code></p>
  </li>
</ol>

<h4 id="3-example-upgrade-procedure-in-practice">3. Example upgrade procedure in practice:</h4>

<ol>
  <li>
    <p>Let’s assume were starting with Nextcloud version <code class="language-plaintext highlighter-rouge">23.0.9</code> and the <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> looks similar to the following snippet:</p>

    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:23.0.9-apache"</span>

    <span class="s">...</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>First, shut down Nextcloud:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down
</code></pre></div>    </div>
  </li>
  <li>
    <p>Update the Nextcloud Docker image to <code class="language-plaintext highlighter-rouge">23.0.11</code>, the latest minor and patch version of the current major version. :warning: <strong>Do not increase the major version yet!</strong></p>

    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:23.0.11-apache"</span>

    <span class="s">...</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>You can now start Nextcloud. Nextcloud will automatically apply all required updates. During the upgrade, the Nextcloud web UI is automatically disabled. You can check if the upgrade completed successfully when you log back in to the web Nextcloud web UI at <a href="http://localhost:8080/">localhost:8080</a>. Alternatively, you can also watch the log statements to track the progress of the upgrade:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up <span class="nt">-d</span>

docker-compose logs <span class="nt">-f</span>
<span class="c"># nextcloud_1  | Initializing nextcloud 23.0.11.1 ...</span>
<span class="c"># nextcloud_1  | Upgrading nextcloud from 23.0.9.1 ...</span>
<span class="c"># ...</span>
<span class="c"># #### And a few moments later ####</span>
<span class="c"># ...</span>
<span class="c"># #### The following line indicates that the Nextcloud web server has started ####</span>
<span class="c"># nextcloud_1  | [Sun Nov 13 17:36:38.725816 2022] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>Once the upgrade is completed successfully, you can now shut down Nextcloud again:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down
</code></pre></div>    </div>
  </li>
  <li>
    <p>We can now bump the Nextcloud version to the <strong>next</strong> major version <code class="language-plaintext highlighter-rouge">24.0.7</code>. Luckily, we can instantly update to the latest minor and patch version. :warning: <strong>Never skip major a release. Increase the major version only one step at a time.</strong></p>

    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:24.0.7-apache"</span>

    <span class="s">...</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>Start Nextcloud. Again, the logs should indicate Nextcloud detecting an old version and applying an upgrade:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up <span class="nt">-d</span>

docker-compose logs <span class="nt">-f</span>
<span class="c"># nextcloud_1  | Initializing nextcloud 24.0.7.1 ...</span>
<span class="c"># nextcloud_1  | Upgrading nextcloud from 23.0.11.1 ...</span>
<span class="c"># ...</span>
<span class="c"># #### And a few moments later ####</span>
<span class="c"># ...</span>
<span class="c"># #### The following line indicates that the Nextcloud web server has started ####</span>
<span class="c"># nextcloud_1  | [Sun Nov 13 17:44:05.397079 2022] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>Shut down once again, change the Docker image to the latest major version, and finally restart:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down
</code></pre></div>    </div>

    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:25.0.1-apache"</span>

    <span class="s">...</span>
</code></pre></div>    </div>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up <span class="nt">-d</span>
</code></pre></div>    </div>
  </li>
</ol>

<p>Congrats, you should now have the latest version of Nextcloud running. :tada:</p>

<hr />

<h2 id="updating-mariadb-to-the-latest-minor-or-patch-version">Updating MariaDB to the latest minor or patch version</h2>

<p>:warning: <strong>Make sure to have backups before updating MariaDB!!!</strong></p>

<p>Unless Nextcloud is incompatible with your current version of MariaDB, it is not necessary to upgrade both Docker images simultaneously. Instead, you can first upgrade Nextcloud to the latest major, minor, and patch version, and afterwards do an upgrade of MariaDB.</p>

<p>Upgrading to the latest minor or patch version is trivial. Simply replace the tag of the Docker image in your <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> with the latest available version. Example <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> with older MariaDB version <code class="language-plaintext highlighter-rouge">10.4.8-bionic</code>:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">mariadb</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">mariadb:10.4.8-bionic"</span>

    <span class="s">...</span>
</code></pre></div></div>

<p>Updated <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> with MariaDB’s latest version <code class="language-plaintext highlighter-rouge">10.9.4-jammy</code> as of November 2022 (<a href="https://hub.docker.com/_/mariadb/tags">check on Docker Hub</a> for the latest available version):</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">mariadb</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">mariadb:10.9.4-jammy"</span>

    <span class="s">...</span>
</code></pre></div></div>

<p>Then simply restart your Nextcloud and MariaDB using <code class="language-plaintext highlighter-rouge">docker-compose down</code> followed by <code class="language-plaintext highlighter-rouge">docker-compose up -d</code>.</p>

<p>After the update, the log statements might include erros from MariaDB due to incorrect database schemas or compatibility issues. Example logs:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mariadb_1    | 2022-11-13 17:53:06 13 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'histogram' at position 10 to have type longblob, found type varbinary(255).
mariadb_1    | 2022-11-13 17:53:06 13 [ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9 to have type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB','JSON_HB'), found type enum('SINGLE_PREC_HB','DOUBLE_PREC_HB').
</code></pre></div></div>

<p>Luckily, MariaDB ships with the utility program <code class="language-plaintext highlighter-rouge">mariadb-upgrade</code> to identify and fix these compatibility issues:</p>

<ol>
  <li>
    <p>Prerequisite: the MariaDB Docker container must be running.</p>
  </li>
  <li>
    <p>Establish an interactive shell session for the MariaDB container:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose <span class="nb">exec </span>mariadb bash
</code></pre></div>    </div>
  </li>
  <li>
    <p>Inside the MariaDB Docker container, run the following command. It should prompt for your MariaDB’s root password:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mariadb-upgrade <span class="nt">--user</span><span class="o">=</span>root <span class="nt">--password</span>
<span class="c"># Enter password:</span>
<span class="c"># Phase 1/7: Checking and upgrading mysql database</span>
<span class="c"># Processing databases</span>
<span class="c"># mysql</span>
<span class="c"># mysql.column_stats                                 OK</span>
<span class="c">#</span>
<span class="c"># ...</span>
<span class="c">#</span>
<span class="c"># nextcloud.oc_webauthn                              OK</span>
<span class="c"># nextcloud.oc_whats_new                             OK</span>
<span class="c"># sys</span>
<span class="c"># sys.sys_config                                     OK</span>
<span class="c"># Phase 7/7: Running 'FLUSH PRIVILEGES'</span>
<span class="c"># OK</span>
</code></pre></div>    </div>
  </li>
</ol>

<p>Congrats, you’ve successfully updated MariaDB. Your logs should contain no more error statements. For a more in-depth documentation on how to upgrade MariaDB please refer to the <a href="https://mariadb.com/docs//all/service-management/upgrades/community-server/release-series-cs10-5">official MariaDB documentation</a>.</p>]]></content><author><name></name></author><category term="cloud" /><category term="nextcloud" /><category term="mariadb" /><category term="maria" /><category term="docker" /><category term="docker compose" /><category term="update" /><category term="upgrade" /><summary type="html"><![CDATA[The developers of Nextcloud typically release three major releases per year. With this release frequency, your current Nextcloud installation will quickly become obsolete. Luckily, updating Nextcloud is super easy. In this blog post, I will briefly describe how to update a Docker or Docker Compose based Nextcloud installation.]]></summary></entry><entry><title type="html">Run Nextcloud as Docker container with Docker Compose (updated in Nov 2022)</title><link href="https://philenius.github.io/cloud/2022/11/13/run-nextcloud-as-docker-container-with-docker-compose.html" rel="alternate" type="text/html" title="Run Nextcloud as Docker container with Docker Compose (updated in Nov 2022)" /><published>2022-11-13T12:00:00+01:00</published><updated>2022-11-13T12:00:00+01:00</updated><id>https://philenius.github.io/cloud/2022/11/13/run-nextcloud-as-docker-container-with-docker-compose</id><content type="html" xml:base="https://philenius.github.io/cloud/2022/11/13/run-nextcloud-as-docker-container-with-docker-compose.html"><![CDATA[<p>I’ve set up my private cloud using <a href="https://nextcloud.com/">Nextcloud</a>. Because I’m a huge fan of
Docker, I decided to run Nextcloud as a Docker container. Luckily, there’s an official
<a href="https://hub.docker.com/_/nextcloud">Docker image</a> and they also provide examples on how
to run Nextcloud with a standalone database using Docker Compose.</p>

<p><strong><em>Updated in November 2022 to include the latest Docker images</em></strong></p>

<hr />

<h2 id="a-basic-setup">a) Basic setup</h2>

<p>The basic setup without subdomain, without TLS and without a reverse proxy is super easy. The corresponding <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> looks like this:</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:25.0.1-apache"</span>
    <span class="na">ports</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s2">"</span><span class="s">8080:80"</span>
    <span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
    <span class="na">volumes</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">nextcloud:/var/www/html</span>
    <span class="na">environment</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">MYSQL_DATABASE=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_USER=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_PASSWORD=&lt;MYSQL_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_HOST=mariadb</span>
      <span class="pi">-</span> <span class="s">NEXTCLOUD_ADMIN_USER=&lt;NEXTCLOUD_ADMIN_USER&gt;</span>
      <span class="pi">-</span> <span class="s">NEXTCLOUD_ADMIN_PASSWORD=&lt;NEXTCLOUD_ADMIN_PASSWORD&gt;</span>
  <span class="na">mariadb</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">mariadb:10.9.4-jammy"</span>
    <span class="na">command</span><span class="pi">:</span> <span class="s2">"</span><span class="s">--transaction-isolation=READ-COMMITTED</span><span class="nv"> </span><span class="s">--binlog-format=ROW"</span>
    <span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
    <span class="na">volumes</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">db:/var/lib/mysql</span>
    <span class="na">environment</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">MYSQL_ROOT_PASSWORD=&lt;MYSQL_ROOT_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_PASSWORD=&lt;MYSQL_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_DATABASE=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_USER=nextcloud</span>
<span class="na">volumes</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
  <span class="na">db</span><span class="pi">:</span>
</code></pre></div></div>

<p>You can then start your Nextcloud setup using the following shell command:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up <span class="nt">-d</span>
</code></pre></div></div>

<p>Use the following command to see a continuous log stream of your Nextcloud and MariaDB Docker contains on stdout. You can press <code class="language-plaintext highlighter-rouge">Ctrl + C</code> at any time to stop the log output (this will not shutdown your Nextcloud container):</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose logs <span class="nt">-f</span>
</code></pre></div></div>

<hr />

<h2 id="b-advanced-setup">b) Advanced setup</h2>

<p>Now, suppose we want to add a reverse proxy like NGINX in front of Nextcloud because we might want to run several applications behind different subdomains on our server. Furthermore, we want to secure our connection to Nextcloud through TLS because we don’t want that all our private data is transfered unencrypted over the internet:</p>

<p><a class="img" href="/assets/2019-10-23/plantumlScenario.png">
  <img src="/assets/2019-10-23/plantumlScenario.png" alt="PlantUML diagram denoting the scenario with Docker" />
</a></p>

<!--
```plantuml
@startuml

together {
    actor Client
    node Server {
        node "Docker Daemon" {
            together {
                node NGINX
            }
            together {
                node "App1"
                node "App2"
                node Nextcloud
            }
        }
    }
}

NGINX -[hidden]-> Nextcloud
NGINX -[hidden]-> App1
NGINX -[hidden]-> App2

Client-> NGINX : cloud.example.com
NGINX -> App1 : app1.example.com
NGINX -> App2 : app2.example.com
NGINX -> Nextcloud : cloud.example.com

@enduml
```
-->

<p>To realize this scenario, we need to make these changes:</p>

<ul>
  <li>
    <p>The Docker containers of NGINX and Nextcloud (+ MariaDB) need to run on the same Docker network so that NGINX can proxy traffic to Nextcloud. I assume that you already have a running NGINX with TLS setup (<a href="https://medium.com/@pentacent/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71">read this for a how-to</a>). <strong>Make sure to use the appropriate network name!</strong> When using Docker Compose, the network name is a concatenation of the directory name (where the <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> is stored) + the name of the NGINX container + <code class="language-plaintext highlighter-rouge">network</code> (each part separated by hyphens). In my case the NGINX is running inside a Docker network called <code class="language-plaintext highlighter-rouge">apps_nginx_network</code>.</p>
  </li>
  <li>
    <p>Since the Docker containers of NGINX and Nextcloud are now running on the same network, it is not necessary to expose the port of Nextcloud. For this reason, compared to the basic setup described above, the port exposure (<code class="language-plaintext highlighter-rouge">ports: - "8080:80"</code>) can be omitted.</p>
  </li>
  <li>
    <p>For security reasons, Nextcloud prints an error message when running on a subdomain. To fix this error, set the environment variable <code class="language-plaintext highlighter-rouge">NEXTCLOUD_TRUSTED_DOMAINS</code> to the fully-qualified domain name (FQDN) under which your Nextcloud instance will be served, in this case <code class="language-plaintext highlighter-rouge">cloud.example.com</code>.</p>
  </li>
</ul>

<h4 id="resulting-configuration-docker-composeyml"><strong>Resulting configuration <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>:</strong></h4>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s2">"</span><span class="s">3"</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:25.0.1-apache"</span>
    <span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
    <span class="na">volumes</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">nextcloud:/var/www/html</span>
    <span class="na">environment</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">MYSQL_DATABASE=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_USER=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_PASSWORD=&lt;MYSQL_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_HOST=mariadb</span>
      <span class="pi">-</span> <span class="s">NEXTCLOUD_ADMIN_USER=&lt;NEXTCLOUD_ADMIN_USER&gt;</span>
      <span class="pi">-</span> <span class="s">NEXTCLOUD_ADMIN_PASSWORD=&lt;NEXTCLOUD_ADMIN_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">NEXTCLOUD_TRUSTED_DOMAINS=cloud.example.com</span>
    <span class="na">networks</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">&lt;NAME_OF_NGINX_DOCKER_NETWORK&gt;</span>
  <span class="na">mariadb</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">mariadb:10.9.4-jammy"</span>
    <span class="na">command</span><span class="pi">:</span> <span class="s2">"</span><span class="s">--transaction-isolation=READ-COMMITTED</span><span class="nv"> </span><span class="s">--binlog-format=ROW"</span>
    <span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>
    <span class="na">volumes</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">db:/var/lib/mysql</span>
    <span class="na">environment</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">MYSQL_ROOT_PASSWORD=&lt;MYSQL_ROOT_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_PASSWORD=&lt;MYSQL_PASSWORD&gt;</span>
      <span class="pi">-</span> <span class="s">MYSQL_DATABASE=nextcloud</span>
      <span class="pi">-</span> <span class="s">MYSQL_USER=nextcloud</span>
    <span class="na">networks</span><span class="pi">:</span>
      <span class="pi">-</span> <span class="s">&lt;NAME_OF_NGINX_DOCKER_NETWORK&gt;</span>
<span class="na">volumes</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
  <span class="na">db</span><span class="pi">:</span>
<span class="na">networks</span><span class="pi">:</span>
  <span class="na">&lt;NAME_OF_NGINX_DOCKER_NETWORK&gt;</span><span class="pi">:</span>
    <span class="na">external</span><span class="pi">:</span> <span class="no">true</span>  
</code></pre></div></div>

<p>After updating your <code class="language-plaintext highlighter-rouge">docker-compose.yml</code>, you can restart Nextcloud so that the changes take effect:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose down
docker-compose up <span class="nt">-d</span>
</code></pre></div></div>

<h4 id="forward-traffic-from-nginx-to-nextcloud"><strong>Forward traffic from NGINX to Nextcloud:</strong></h4>

<p>Because NGINX and Nextcloud run inside the same Docker network, they can ping / reach eachother through their container names (= their hostnames). That means, in this example the NGINX container can forward traffic to Nextcloud through <code class="language-plaintext highlighter-rouge">http://nextcloud:80</code>.</p>

<p>Your NGINX configuration could look similar to this:</p>

<div class="language-conf highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...

<span class="n">server</span> {
    <span class="n">listen</span> <span class="m">443</span> <span class="n">ssl</span>;
    <span class="n">server_name</span> <span class="n">cloud</span>.<span class="n">example</span>.<span class="n">com</span>;
    <span class="n">server_tokens</span> <span class="n">off</span>;

    <span class="n">ssl_certificate</span> /<span class="n">etc</span>/<span class="n">letsencrypt</span>/<span class="n">live</span>/<span class="n">cloud</span>.<span class="n">example</span>.<span class="n">com</span>/<span class="n">fullchain</span>.<span class="n">pem</span>;
    <span class="n">ssl_certificate_key</span> /<span class="n">etc</span>/<span class="n">letsencrypt</span>/<span class="n">live</span>/<span class="n">cloud</span>.<span class="n">example</span>.<span class="n">com</span>/<span class="n">privkey</span>.<span class="n">pem</span>;

    <span class="n">include</span> /<span class="n">etc</span>/<span class="n">letsencrypt</span>/<span class="n">options</span>-<span class="n">ssl</span>-<span class="n">nginx</span>.<span class="n">conf</span>;
    <span class="n">ssl_dhparam</span> /<span class="n">etc</span>/<span class="n">letsencrypt</span>/<span class="n">ssl</span>-<span class="n">dhparams</span>.<span class="n">pem</span>;

    <span class="n">location</span> / {
        <span class="n">proxy_pass</span> <span class="n">http</span>://<span class="n">nextcloud</span>:<span class="m">80</span>/;
        <span class="n">proxy_set_header</span>    <span class="n">Host</span>                $<span class="n">http_host</span>;
        <span class="n">proxy_set_header</span>    <span class="n">X</span>-<span class="n">Real</span>-<span class="n">IP</span>           $<span class="n">remote_addr</span>;
        <span class="n">proxy_set_header</span>    <span class="n">X</span>-<span class="n">Forwarded</span>-<span class="n">For</span>     $<span class="n">proxy_add_x_forwarded_for</span>;
    }

    <span class="n">client_max_body_size</span> <span class="m">10</span><span class="n">G</span>;
}

...
</code></pre></div></div>]]></content><author><name></name></author><category term="cloud" /><category term="nextcloud" /><category term="docker" /><category term="docker compose" /><category term="nginx" /><summary type="html"><![CDATA[I’ve set up my private cloud using Nextcloud. Because I’m a huge fan of Docker, I decided to run Nextcloud as a Docker container. Luckily, there’s an official Docker image and they also provide examples on how to run Nextcloud with a standalone database using Docker Compose.]]></summary></entry><entry><title type="html">How to track Detectron2 experiments in MLflow</title><link href="https://philenius.github.io/machine-learning/2022/01/09/how-to-log-artifacts-metrics-and-parameters-of-your-detectron2-model-training-to-mlflow.html" rel="alternate" type="text/html" title="How to track Detectron2 experiments in MLflow" /><published>2022-01-09T13:00:00+01:00</published><updated>2022-01-09T13:00:00+01:00</updated><id>https://philenius.github.io/machine-learning/2022/01/09/how-to-log-artifacts-metrics-and-parameters-of-your-detectron2-model-training-to-mlflow</id><content type="html" xml:base="https://philenius.github.io/machine-learning/2022/01/09/how-to-log-artifacts-metrics-and-parameters-of-your-detectron2-model-training-to-mlflow.html"><![CDATA[<p>In this blog post, I’ll show you how to integrate MLflow into your ML lifecycle so that you can log artifacts, metrics, and parameters of your model trainings/experiments with Detectron2. As a result, you’ll be able to log training parameters (<code class="language-plaintext highlighter-rouge">MODEL.WEIGHTS</code>, <code class="language-plaintext highlighter-rouge">OUTPUT_DIR</code>, <code class="language-plaintext highlighter-rouge">SOLVER.MAX_ITER</code>, etc), training metrics (AP, duration, loss, etc.), and training artifacts (such as the model itself, training log file, sample images with inference, evaluation results, etc.) to MLflow.</p>

<div style="height: 2rem"></div>

<p><a class="img" href="/assets/2022-01-09/mlflow-ui.gif">
    <img src="/assets/2022-01-09/mlflow-ui.gif" alt="" />
</a></p>

<div style="height: 2rem"></div>

<p>From my understanding, there are two different ways how you could integrate MLflow into Detectron2:</p>

<ol>
  <li>
    <p>Write a custom training loop by creating a subclass of <code class="language-plaintext highlighter-rouge">detectron2.engine.defaults.DefaultTrainer</code>. In this subclass, you could overwrite <code class="language-plaintext highlighter-rouge">DefaultTrainer</code>’s methods to log artifacts, metrics, parameters, etc. to MLflow. Example:</p>

    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">detectron2.engine</span> <span class="kn">import</span> <span class="n">DefaultTrainer</span>
<span class="kn">import</span> <span class="nn">mlflow</span>
   
<span class="k">class</span> <span class="nc">CustomTrainerWithMLflow</span><span class="p">(</span><span class="n">DefaultTrainer</span><span class="p">):</span>
   
    <span class="o">@</span><span class="nb">staticmethod</span>
    <span class="k">def</span> <span class="nf">run_step</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">().</span><span class="n">run_step</span><span class="p">()</span>
        <span class="n">mlflow</span><span class="p">.</span><span class="n">log_metric</span><span class="p">(</span><span class="s">"key"</span><span class="p">,</span> <span class="s">"value"</span><span class="p">)</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p><strong>My preferred solution:</strong> leverage Detectron2’s hook system and implement a custom hook that takes over the recording to MLflow throughout the model training. A custom hook has the advantage over a custom trainer that it can be reused for future model trainings. You can easily hook it into any trainer class even, if you were to switch from object detection to segmentation.</p>
  </li>
</ol>

<div style="height: 2rem"></div>

<h2 id="short-intro-to-experiment-tracking-with-mlflow">Short intro to experiment tracking with MLflow</h2>

<p>To line out the the next steps of this blog post, let’s start with a blazing fast intro to experiment tracking with MLflow and the concepts of <a href="https://www.mlflow.org/docs/latest/tracking.html"><em>MLflow Tracking</em></a>:</p>

<ol>
  <li>
    <p>Install MLflow from PyPI via <code class="language-plaintext highlighter-rouge">pip install mlflow</code>.</p>
  </li>
  <li>
    <p>Start an MLflow tracking server via <code class="language-plaintext highlighter-rouge">mlflow ui</code> and open <a href="http://localhost:5000">http://localhost:5000</a> in your browser to see the MLflow UI.</p>
  </li>
  <li>
    <p>Import MLflow in Python via <code class="language-plaintext highlighter-rouge">import mlflow</code>.</p>
  </li>
  <li>Set the <strong>tracking server URI</strong>:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">set_tracking_uri</span><span class="p">(</span><span class="s">"http://localhost:5000"</span><span class="p">)</span>
</code></pre></div>    </div>
  </li>
  <li><strong>Experiment:</strong> MLflow allows you to group runs under experiments, which can be useful for comparing runs intended to tackle a particular task. This is how the experiment name is set:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">set_experiment</span><span class="p">(</span><span class="s">"Balloon Object Detection"</span><span class="p">)</span>
</code></pre></div>    </div>

    <p><a class="img" href="/assets/2022-01-09/experiment-overview.png">
    <img src="/assets/2022-01-09/experiment-overview.png" alt="" />
</a></p>
  </li>
  <li><strong>Run:</strong> Each model training should be recorded as a separate <strong>run</strong>. Runs have a name, description, duration, status, and so on. This is how to start a run:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">start_run</span><span class="p">(</span><span class="n">run_name</span><span class="o">=</span><span class="s">"#0 first training"</span><span class="p">)</span>
</code></pre></div>    </div>

    <p><a class="img" href="/assets/2022-01-09/run-overview.png">
    <img src="/assets/2022-01-09/run-overview.png" alt="" />
</a></p>
  </li>
  <li><strong>Parameters:</strong> After a run has been started, you can log training parameters (key-value parameters) in MLflow like this:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">log_param</span><span class="p">(</span><span class="s">"iterations"</span><span class="p">,</span> <span class="mi">1000</span><span class="p">)</span>
</code></pre></div>    </div>

    <p><a class="img" href="/assets/2022-01-09/run-parameters.png">
    <img src="/assets/2022-01-09/run-parameters.png" alt="Parameters of an MLflow run are displayed as a simple list." />
</a></p>
  </li>
  <li><strong>Metrics:</strong> metrics (e.g. loss or AP) are handled differently by MLflow in that they can be updated throughout the course of the run. MLflow records the time-specific values of a metric and lets you visualize the metrics’s full history:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">log_metric</span><span class="p">(</span><span class="s">"loss"</span><span class="p">,</span> <span class="mf">3.456248</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
</code></pre></div>    </div>

    <p><a class="img" href="/assets/2022-01-09/run-metrics.png">
    <img src="/assets/2022-01-09/run-metrics.png" alt="The metrics view of an MLflow run shows time-dependent values in a graph." />
</a></p>
  </li>
  <li><strong>Artifacts:</strong> you can log output files of your model training in any format as so-called <em>artifacts</em>, e.g. images, data files, models, log files, etc., like this:
    <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">log_artifacts</span><span class="p">(</span><span class="s">"output/"</span><span class="p">)</span>
</code></pre></div>    </div>

    <p><a class="img" href="/assets/2022-01-09/run-artifacts.png">
    <img src="/assets/2022-01-09/run-artifacts.png" alt="The artifacts of an MLflow run are listed on the bottom of the run details. Image files and text files are previewed automatically." />
</a></p>
  </li>
  <li><strong>Tags:</strong> lastly, you can annotate runs with tags (key-value parameters). There are <em>System Tags</em> which are reserved for internal use. For example, you can use the tag <code class="language-plaintext highlighter-rouge">mlflow.note.content</code> to set the run description:</li>
</ol>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>   <span class="n">mlflow</span><span class="p">.</span><span class="n">set_tag</span><span class="p">(</span><span class="s">"mlflow.note.content"</span><span class="p">,</span> <span class="s">"First training with 1000 iterations, 210 images in the train set... blabla"</span><span class="p">)</span>
</code></pre></div></div>

<div style="height: 4rem"></div>

<h2 id="step-1-extend-detectron2s-configuration">Step 1: extend Detectron2’s configuration</h2>

<p>First, let’s extend the Detectron2 configuration so that we can make the hook , which we’ll implement in step 2, configurable and reusable. We’ll add four configurations under the new configuration node <code class="language-plaintext highlighter-rouge">MLFLOW</code> to make experiment name, run name, run description, and tracking server URI configurable:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">detectron2.config</span> <span class="kn">import</span> <span class="n">get_cfg</span><span class="p">,</span> <span class="n">CfgNode</span>

<span class="n">cfg</span> <span class="o">=</span> <span class="n">get_cfg</span><span class="p">()</span>

<span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span> <span class="o">=</span> <span class="n">CfgNode</span><span class="p">()</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">EXPERIMENT_NAME</span> <span class="o">=</span> <span class="s">"Balloon Object Detection"</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">RUN_DESCRIPTION</span> <span class="o">=</span> <span class="s">"First training with 1000 iterations, 210 images in the train set... blabla"</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">RUN_NAME</span> <span class="o">=</span> <span class="s">"#0 first training"</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">TRACKING_URI</span> <span class="o">=</span> <span class="s">"http://localhost:5000"</span>
</code></pre></div></div>

<div style="height: 2rem"></div>

<h2 id="step-2-implement-a-hook-for-mlflow">Step 2: implement a hook for MLflow</h2>

<p>Now that we extended the Detectron2 configuration, we can implement a custom hook which uses the MLflow Python package to log all experiment artifacts, metrics, and parameters to an MLflow tracking server.</p>

<p>Hooks in Detectron2 must be subclasses of <code class="language-plaintext highlighter-rouge">detectron2.engine.HookBase</code>. Each hook can implement 4 methods (<a href="https://detectron2.readthedocs.io/en/latest/modules/engine.html#detectron2.engine.HookBase">see the docs</a>):</p>

<ul>
  <li><code class="language-plaintext highlighter-rouge">before_train()</code> is called before the first training iteration</li>
  <li><code class="language-plaintext highlighter-rouge">after_train()</code> is called after the last training iteration</li>
  <li><code class="language-plaintext highlighter-rouge">before_step()</code> is called before each training iteration</li>
  <li><code class="language-plaintext highlighter-rouge">after_step()</code> is called after each training iteration</li>
</ul>

<p>Here’s my code snippet for the MLflow hook which implements three of these four methods:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">detectron2.engine</span> <span class="kn">import</span> <span class="n">HookBase</span>
<span class="kn">import</span> <span class="nn">mlflow</span>

<span class="k">class</span> <span class="nc">MLflowHook</span><span class="p">(</span><span class="n">HookBase</span><span class="p">):</span>
    <span class="s">"""
    A custom hook class that logs artifacts, metrics, and parameters to MLflow.
    """</span>

    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">cfg</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">().</span><span class="n">__init__</span><span class="p">()</span>
        <span class="bp">self</span><span class="p">.</span><span class="n">cfg</span> <span class="o">=</span> <span class="n">cfg</span><span class="p">.</span><span class="n">clone</span><span class="p">()</span>

    <span class="k">def</span> <span class="nf">before_train</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">with</span> <span class="n">torch</span><span class="p">.</span><span class="n">no_grad</span><span class="p">():</span>
            <span class="n">mlflow</span><span class="p">.</span><span class="n">set_tracking_uri</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">TRACKING_URI</span><span class="p">)</span>
            <span class="n">mlflow</span><span class="p">.</span><span class="n">set_experiment</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">EXPERIMENT_NAME</span><span class="p">)</span>
            <span class="n">mlflow</span><span class="p">.</span><span class="n">start_run</span><span class="p">(</span><span class="n">run_name</span><span class="o">=</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">RUN_NAME</span><span class="p">)</span>
            <span class="n">mlflow</span><span class="p">.</span><span class="n">set_tag</span><span class="p">(</span><span class="s">"mlflow.note.content"</span><span class="p">,</span>
                           <span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">MLFLOW</span><span class="p">.</span><span class="n">RUN_DESCRIPTION</span><span class="p">)</span>
            <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">items</span><span class="p">():</span>
                <span class="n">mlflow</span><span class="p">.</span><span class="n">log_param</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">after_step</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">with</span> <span class="n">torch</span><span class="p">.</span><span class="n">no_grad</span><span class="p">():</span>
            <span class="n">latest_metrics</span> <span class="o">=</span> <span class="bp">self</span><span class="p">.</span><span class="n">trainer</span><span class="p">.</span><span class="n">storage</span><span class="p">.</span><span class="n">latest</span><span class="p">()</span>
            <span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">latest_metrics</span><span class="p">.</span><span class="n">items</span><span class="p">():</span>
                <span class="n">mlflow</span><span class="p">.</span><span class="n">log_metric</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="n">k</span><span class="p">,</span> <span class="n">value</span><span class="o">=</span><span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">step</span><span class="o">=</span><span class="n">v</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>

    <span class="k">def</span> <span class="nf">after_train</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">with</span> <span class="n">torch</span><span class="p">.</span><span class="n">no_grad</span><span class="p">():</span>
            <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="s">"model-config.yaml"</span><span class="p">),</span> <span class="s">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
                <span class="n">f</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">dump</span><span class="p">())</span>
            <span class="n">mlflow</span><span class="p">.</span><span class="n">log_artifacts</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">)</span>
</code></pre></div></div>

<p>Explanations:</p>

<ul>
  <li>
    <p>In <code class="language-plaintext highlighter-rouge">before_train()</code> we set the tracking server URI, experiment name, and run name and start the run. Optionally, we can also set the description of the run. Run name and run description can be changed later at any time.</p>
  </li>
  <li>
    <p>In <code class="language-plaintext highlighter-rouge">after_step()</code> we request the latest training metrics from Detectron2’s <a href="https://detectron2.readthedocs.io/en/latest/modules/utils.html#detectron2.utils.events.EventStorage">EventStorage</a> at each training iteration. The class <code class="language-plaintext highlighter-rouge">EventStorage</code> stores all training metrics (accuracies, losses, bbox APs, learning rate, etc.). We simply iterate over the most recent values of all metrics and log them to MLflow. This way all metrics which appear in TensorBoard will also be logged to MLflow.</p>
  </li>
  <li>
    <p>In <code class="language-plaintext highlighter-rouge">after_train()</code> we dump the Detectron2 configuration to a YAML file and finally log all output files (inluding the configuration YAML) to MLflow. This allows us to track and reconstruct every training session afterwards, as all configurations and the model itself are stored in MLflow.</p>
  </li>
</ul>

<div style="height: 2rem"></div>

<h2 id="step-3-add-a-custom-trainer-class-for-coco-metrics">Step 3: add a custom trainer class for COCO metrics</h2>

<p>We could start training our model right away but per default the <code class="language-plaintext highlighter-rouge">DefaultTrainer</code> of Detectron2 doesn’t evaluate the model on the validation/dev set. If we want to evaluate AR for object proposals or AP for instance detection/segmentation during training, then we’ll have write a custom trainer class which extends <code class="language-plaintext highlighter-rouge">DefaultTrainer</code> like this:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CocoTrainer</span><span class="p">(</span><span class="n">DefaultTrainer</span><span class="p">):</span>
    <span class="s">"""
    A custom trainer class that evaluates the model on the validation set every `_C.TEST.EVAL_PERIOD` iterations.
    """</span>

    <span class="o">@</span><span class="nb">classmethod</span>
    <span class="k">def</span> <span class="nf">build_evaluator</span><span class="p">(</span><span class="n">cls</span><span class="p">,</span> <span class="n">cfg</span><span class="p">,</span> <span class="n">dataset_name</span><span class="p">,</span> <span class="n">output_folder</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
        <span class="k">if</span> <span class="n">output_folder</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
            <span class="n">os</span><span class="p">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_VALIDATION_SET_EVALUATION</span><span class="p">,</span>
                        <span class="n">exist_ok</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>

        <span class="k">return</span> <span class="n">COCOEvaluator</span><span class="p">(</span><span class="n">dataset_name</span><span class="p">,</span> <span class="n">distributed</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span> <span class="n">output_dir</span><span class="o">=</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_VALIDATION_SET_EVALUATION</span><span class="p">)</span>
</code></pre></div></div>

<div style="height: 2rem"></div>

<h2 id="step-4-train-the-model">Step 4: train the model</h2>

<p>Now we’re ready to start training our model with Detectron2. In line 17 we configure the logger of Detectron2 to write logs to a log file inside the output directory. This way the training log will be recorded in MLflow because all  files in the output directory are send as artifacts to MLflow by our hook class. In line 19 we instantiate our hook class <code class="language-plaintext highlighter-rouge">MLflowHook</code> and register it with the <code class="language-plaintext highlighter-rouge">DefaultTrainer</code> in line 22.</p>

<p><strong>Please note:</strong> The following code snippet assumes that you already went through the steps of registering your dataset (train set, val/dev set, and test set) in Detectron2’s <code class="language-plaintext highlighter-rouge">DatasetCatalog</code> and <code class="language-plaintext highlighter-rouge">MetadataCatalog</code>. I skipped these steps section because they are specific to your use case / dataset.</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">os</span>

<span class="kn">from</span> <span class="nn">detectron2.engine</span> <span class="kn">import</span> <span class="n">DefaultTrainer</span>

<span class="n">cfg</span><span class="p">.</span><span class="n">merge_from_file</span><span class="p">(</span><span class="n">model_zoo</span><span class="p">.</span><span class="n">get_config_file</span><span class="p">(</span><span class="s">"COCO-Detection/faster_rcnn_R_50_FPN_1x.yaml"</span><span class="p">))</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">DATASETS</span><span class="p">.</span><span class="n">TRAIN</span> <span class="o">=</span> <span class="p">(</span><span class="s">"balloon_train"</span><span class="p">,)</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">DATASETS</span><span class="p">.</span><span class="n">TEST</span> <span class="o">=</span> <span class="p">(</span><span class="s">"balloon_val"</span><span class="p">,)</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">SOLVER</span><span class="p">.</span><span class="n">MAX_ITER</span> <span class="o">=</span> <span class="mi">1000</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span> <span class="o">=</span> <span class="n">datetime</span><span class="p">.</span><span class="n">now</span><span class="p">().</span><span class="n">strftime</span><span class="p">(</span><span class="s">"%Y-%m-%d_%H-%M-%S-output"</span><span class="p">)</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_VALIDATION_SET_EVALUATION</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span>
        <span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="s">"validation-set-evaluation"</span><span class="p">)</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_TEST_SET_EVALUATION</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span>
        <span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="s">"test-set-evaluation"</span><span class="p">)</span>
<span class="n">cfg</span><span class="p">.</span><span class="n">TEST</span><span class="p">.</span><span class="n">EVAL_PERIOD</span> <span class="o">=</span> <span class="mi">100</span>

<span class="n">os</span><span class="p">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">os</span><span class="p">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_VALIDATION_SET_EVALUATION</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">os</span><span class="p">.</span><span class="n">makedirs</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_TEST_SET_EVALUATION</span><span class="p">,</span> <span class="n">exist_ok</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>

<span class="n">setup_logger</span><span class="p">(</span><span class="n">output</span><span class="o">=</span><span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="s">"training-log.txt"</span><span class="p">))</span>

<span class="n">mlflow_hook</span> <span class="o">=</span> <span class="n">MLflowHook</span><span class="p">(</span><span class="n">cfg</span><span class="p">)</span>

<span class="n">trainer</span> <span class="o">=</span> <span class="n">CocoTrainer</span><span class="p">(</span><span class="n">cfg</span><span class="p">)</span>
<span class="n">trainer</span><span class="p">.</span><span class="n">register_hooks</span><span class="p">(</span><span class="n">hooks</span><span class="o">=</span><span class="p">[</span><span class="n">mlflow_hook</span><span class="p">])</span>
<span class="n">trainer</span><span class="p">.</span><span class="n">resume_or_load</span><span class="p">(</span><span class="n">resume</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">trainer</span><span class="p">.</span><span class="n">train</span><span class="p">()</span>
</code></pre></div></div>

<div style="height: 2rem"></div>

<h2 id="step-5-evaluate-the-model-on-the-test-set">Step 5: evaluate the model on the test set</h2>

<p>Assuming that you have a train set, a val/dev set, <strong>and a separate test set</strong>, you might want to evaluate the trained model on your test and log the results to MLflow, too. Let’s do this!</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">import</span> <span class="nn">logging</span>

<span class="kn">from</span> <span class="nn">detectron2.data</span> <span class="kn">import</span> <span class="n">build_detection_test_loader</span>
<span class="kn">from</span> <span class="nn">detectron2.engine</span> <span class="kn">import</span> <span class="n">DefaultPredictor</span>
<span class="kn">from</span> <span class="nn">detectron2.evaluation</span> <span class="kn">import</span> <span class="n">COCOEvaluator</span><span class="p">,</span> <span class="n">inference_on_dataset</span>

<span class="n">setup_logger</span><span class="p">(</span><span class="n">output</span><span class="o">=</span><span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_TEST_SET_EVALUATION</span><span class="p">,</span> <span class="s">"evaluation-log.txt"</span><span class="p">))</span>

<span class="n">cfg</span><span class="p">.</span><span class="n">MODEL</span><span class="p">.</span><span class="n">WEIGHTS</span> <span class="o">=</span> <span class="n">os</span><span class="p">.</span><span class="n">path</span><span class="p">.</span><span class="n">join</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR</span><span class="p">,</span> <span class="s">"model_final.pth"</span><span class="p">)</span>

<span class="n">predictor</span> <span class="o">=</span> <span class="n">DefaultPredictor</span><span class="p">(</span><span class="n">cfg</span><span class="p">)</span>

<span class="n">evaluator</span> <span class="o">=</span> <span class="n">COCOEvaluator</span><span class="p">(</span><span class="s">"balloon_test"</span><span class="p">,</span> <span class="n">output_dir</span><span class="o">=</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_TEST_SET_EVALUATION</span><span class="p">)</span>
<span class="n">test_set_loader</span> <span class="o">=</span> <span class="n">build_detection_test_loader</span><span class="p">(</span><span class="n">cfg</span><span class="p">,</span> <span class="s">"balloon_test"</span><span class="p">)</span>

<span class="n">evaluation_results</span> <span class="o">=</span> <span class="n">inference_on_dataset</span><span class="p">(</span><span class="n">predictor</span><span class="p">.</span><span class="n">model</span><span class="p">,</span> <span class="n">test_set_loader</span><span class="p">,</span> <span class="n">evaluator</span><span class="p">)</span>
<span class="n">logging</span><span class="p">.</span><span class="n">info</span><span class="p">(</span><span class="s">"Evaluation results on test set: %s"</span><span class="p">,</span> <span class="n">evaluation_results</span><span class="p">)</span>

<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">evaluation_results</span><span class="p">[</span><span class="s">"bbox"</span><span class="p">].</span><span class="n">items</span><span class="p">():</span>
    <span class="n">mlflow</span><span class="p">.</span><span class="n">log_metric</span><span class="p">(</span><span class="sa">f</span><span class="s">"Test Set </span><span class="si">{</span><span class="n">k</span><span class="si">}</span><span class="s">"</span><span class="p">,</span> <span class="n">v</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>

<span class="n">mlflow</span><span class="p">.</span><span class="n">log_artifacts</span><span class="p">(</span><span class="n">cfg</span><span class="p">.</span><span class="n">OUTPUT_DIR_TEST_SET_EVALUATION</span><span class="p">,</span> <span class="s">"test-set-evaluation"</span><span class="p">)</span>
<span class="n">mlflow</span><span class="p">.</span><span class="n">log_text</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">evaluation_results</span><span class="p">),</span> <span class="s">"test-set-evaluation/coco-metrics.txt"</span><span class="p">)</span>
</code></pre></div></div>

<p>Explanations:</p>

<ul>
  <li>In line 5 we set up the logger to log to a different file now that the model training is done.</li>
  <li>In line 7 we update the configuration so that the weights of the previously trained model are used (instead of pre-trained COCO/ImageNet weights).</li>
  <li>In line 11 we instantiate the <code class="language-plaintext highlighter-rouge">COCOEvaluator</code> for our test set (registered in <code class="language-plaintext highlighter-rouge">DatasetCatalog</code> under the name <code class="language-plaintext highlighter-rouge">balloon_test</code>) and set the output directory.</li>
  <li>
    <p>In line 17 and 18 we iterate over all bbox COCO metrics (e.g. <code class="language-plaintext highlighter-rouge">AP</code>, <code class="language-plaintext highlighter-rouge">AP50</code>, <code class="language-plaintext highlighter-rouge">AP75</code>, <code class="language-plaintext highlighter-rouge">APl</code>, <code class="language-plaintext highlighter-rouge">APm</code>, <code class="language-plaintext highlighter-rouge">APs</code>) and log these as metrics to MLflow. This should give you the following result in MLflow:</p>

    <p><a class="img" href="/assets/2022-01-09/test-set-metrics.png">
     <img src="/assets/2022-01-09/test-set-metrics.png" alt="" />
 </a></p>
  </li>
  <li>In line 20 we record the artifacts that were produced by <code class="language-plaintext highlighter-rouge">COCOEvaluator</code> in its output directory.</li>
  <li>
    <p>Finally, we log the COCO metrics again (this time as artifact instead of a metric) to a virtual text file named <code class="language-plaintext highlighter-rouge">coco-metrics.txt</code> which should appear under MLflow’s artifact section:</p>

    <p><a class="img" href="/assets/2022-01-09/coco-metrics-artifact.png">
     <img src="/assets/2022-01-09/coco-metrics-artifact.png" alt="" />
 </a></p>
  </li>
</ul>

<p>Additionally, you could run your model in inference mode on a few random images of your test set, visualize the predictions, and log the resulting images in MLflow (not part of this blog post). MLflow integrates a simple image viewer with zoom and pan functionality:</p>

<p><a class="img" href="/assets/2022-01-09/image-artifacts.jpg">
    <img src="/assets/2022-01-09/image-artifacts.jpg" alt="" />
</a></p>

<div style="height: 2rem"></div>

<h2 id="step-6-end-the-mlflow-run">Step 6: end the MLflow run</h2>

<p>If you want to see a green check mark in MLflow and the run status set to <code class="language-plaintext highlighter-rouge">FINISHED</code>, then the last step is to end your MLflow run like this:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">mlflow</span><span class="p">.</span><span class="n">end_run</span><span class="p">()</span>
</code></pre></div></div>

<div style="height: 2rem"></div>
<hr />
<div style="height: 2rem"></div>

<h2 id="results">Results</h2>

<p>As a result of the previous steps the MLflow UI should contain one experiment with one run which contains the parameters, metrics, and artifacts of your Detectron2 model training.</p>

<p><a class="img" href="/assets/2022-01-09/mlflow-ui.gif">
    <img src="/assets/2022-01-09/mlflow-ui.gif" alt="" />
</a></p>

<p>Thank you for reading this blog post and good luck with your deep learning project! Let me know in the comments down below if you have any feedback or improvements.</p>]]></content><author><name></name></author><category term="machine-learning" /><category term="Detectron2" /><category term="MLflow" /><category term="Machine Learning" /><category term="Deep Learning" /><category term="ML lifecycle" /><category term="record experiments" /><category term="log artifacts metrics parameters" /><summary type="html"><![CDATA[In this blog post, I’ll show you how to integrate MLflow into your ML lifecycle so that you can log artifacts, metrics, and parameters of your model trainings/experiments with Detectron2. As a result, you’ll be able to log training parameters (MODEL.WEIGHTS, OUTPUT_DIR, SOLVER.MAX_ITER, etc), training metrics (AP, duration, loss, etc.), and training artifacts (such as the model itself, training log file, sample images with inference, evaluation results, etc.) to MLflow.]]></summary></entry><entry><title type="html">Usage of Passport JWT Strategy for Authentication in Socket.IO</title><link href="https://philenius.github.io/web-development/2021/03/31/use-passportjs-for-authentication-in-socket-io.html" rel="alternate" type="text/html" title="Usage of Passport JWT Strategy for Authentication in Socket.IO" /><published>2021-03-31T14:00:00+02:00</published><updated>2021-03-31T14:00:00+02:00</updated><id>https://philenius.github.io/web-development/2021/03/31/use-passportjs-for-authentication-in-socket-io</id><content type="html" xml:base="https://philenius.github.io/web-development/2021/03/31/use-passportjs-for-authentication-in-socket-io.html"><![CDATA[<p><a href="http://www.passportjs.org/">Passport</a> is an authentication middleware for <a href="https://nodejs.org/">Node.js</a> that provides dozens of pluggable authentication mechanisms. You might already have implemented a Node.js backend with <a href="https://expressjs.com/">Express</a> (and Passport). And now you’re wondering how you can protect the WebSocket communication between backend and frontend from unauthenticated access? Perhaps you don’t want to implement authentication twice, but want to reuse your existing Passport authentication via JSON Web Tokens (or any other strategy e.g. basic auth)? Then, this blog post is meant for you!</p>

<div style="height: 2rem"></div>

<p><a class="img" href="/assets/2021-03-31/cover.png">
    <img src="/assets/2021-03-31/cover-thumbnail.png" alt="" />
</a></p>

<div style="height: 2rem"></div>

<p>In this blog post, I’ll show you how to implement a Node.js application with Socket.IO and then how to integrate Passport to add authentication to the WebSocket-based API. In specific, I’ll demonstrate authentication using the example of the <a href="http://www.passportjs.org/packages/passport-jwt/">passport-jwt strategy</a>, a Passport authentication strategy that uses JSON Web Tokens (JWT).</p>

<div style="height: 2rem"></div>

<h2 id="1-npm-project">1. NPM project</h2>

<p>Let’s get started by initializing an NPM project and installing all required NPM packages:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm init <span class="nt">-y</span>
npm <span class="nb">install </span>express passport passport-jwt socket.io
</code></pre></div></div>

<h2 id="2-express-app">2. Express app</h2>

<p>Create a file named <em>server.js</em> and paste the following code to bootstrap your Node.js backend:</p>

<p class="file-name">server.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">use strict</span><span class="dl">'</span><span class="p">;</span>

<span class="kd">const</span> <span class="nx">port</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">PORT</span> <span class="o">||</span> <span class="mi">3000</span><span class="p">;</span>

<span class="kd">const</span> <span class="nx">app</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">express</span><span class="dl">'</span><span class="p">)();</span>
<span class="kd">const</span> <span class="nx">httpServer</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">http</span><span class="dl">'</span><span class="p">).</span><span class="nx">createServer</span><span class="p">(</span><span class="nx">app</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">path</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">path</span><span class="dl">'</span><span class="p">);</span>

<span class="kd">function</span> <span class="nx">log</span><span class="p">(...</span><span class="nx">args</span><span class="p">)</span> <span class="p">{</span>
    <span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="k">new</span> <span class="nb">Date</span><span class="p">(),</span> <span class="p">...</span><span class="nx">args</span><span class="p">);</span>
<span class="p">}</span>

<span class="nx">app</span><span class="p">.</span><span class="kd">get</span><span class="p">(</span><span class="dl">'</span><span class="s1">/</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="nx">res</span><span class="p">.</span><span class="nx">sendFile</span><span class="p">(</span><span class="nx">path</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="dl">'</span><span class="s1">index.html</span><span class="dl">'</span><span class="p">));</span>
<span class="p">});</span>

<span class="nx">httpServer</span><span class="p">.</span><span class="nx">listen</span><span class="p">(</span><span class="nx">port</span><span class="p">,</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="nx">log</span><span class="p">(</span><span class="s2">`Listening on http://localhost:</span><span class="p">${</span><span class="nx">port</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>

<p>So far, your application contains only one HTTP endpoint <code class="language-plaintext highlighter-rouge">GET /</code> that responds with an HTML file. Let’s create the HTML file <code class="language-plaintext highlighter-rouge">index.html</code>:</p>

<p class="file-name">index.html</p>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;!DOCTYPE html&gt;</span>
<span class="nt">&lt;html&gt;</span>
<span class="nt">&lt;head&gt;</span>
    <span class="nt">&lt;meta</span> <span class="na">charset=</span><span class="s">"utf-8"</span><span class="nt">&gt;</span>
    <span class="nt">&lt;title&gt;</span>Socket.IO with JWT Authentication<span class="nt">&lt;/title&gt;</span>
<span class="nt">&lt;/head&gt;</span>
<span class="nt">&lt;body&gt;</span>
    <span class="nt">&lt;h1&gt;</span>Socket.IO with JWT Authentication<span class="nt">&lt;/h1&gt;</span>
<span class="nt">&lt;/body&gt;</span>
<span class="nt">&lt;/html&gt;</span>
</code></pre></div></div>

<p>You can start the Node.js backend with this command:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>node server.js
</code></pre></div></div>

<p>Node.js should print <code class="language-plaintext highlighter-rouge">Listening on http://localhost:3000</code> and you should see the heading <em>Socket.IO with JWT Authentication</em> when you navigate to <a href="http://localhost:3000">http://localhost:3000</a> in your browser.</p>

<h2 id="3-add-socketio-in-frontend-and-backend">3. Add Socket.IO in frontend and backend</h2>

<p>On the server side, import the NPM package <code class="language-plaintext highlighter-rouge">socket.io</code> and define event handlers for the events <code class="language-plaintext highlighter-rouge">connection</code> and <code class="language-plaintext highlighter-rouge">message</code>:</p>

<p class="file-name">server.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">io</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">socket.io</span><span class="dl">'</span><span class="p">)(</span><span class="nx">httpServer</span><span class="p">);</span>

<span class="nx">io</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">connection</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">socket</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">new socket connection</span><span class="dl">'</span><span class="p">);</span>

    <span class="c1">// On receiving the event 'message', we'll respond with the same event back</span>
    <span class="c1">// to the client's socket.</span>
    <span class="nx">socket</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">message</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">message</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
        <span class="nx">log</span><span class="p">(</span><span class="s2">`Socket.IO event 'message' from client with payload: </span><span class="p">${</span><span class="nx">message</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
        <span class="nx">socket</span><span class="p">.</span><span class="nx">emit</span><span class="p">(</span><span class="dl">'</span><span class="s1">message</span><span class="dl">'</span><span class="p">,</span> <span class="s2">`server response: </span><span class="p">${</span><span class="nx">message</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
    <span class="p">});</span>
<span class="p">});</span>
</code></pre></div></div>

<p>On the client side, we can use a CDN to import Socket.IO’s client library. Similar to the backend’s code, we initiate the socket connection and then define three event handlers for the <em>connect</em>, <em>disconnect</em>, and <em>message</em> event. Furthermore, we add a textarea for logging purposes and a button to manually send a message via the WebSocket connection to the backend:</p>

<p class="file-name">index.html</p>

<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;!DOCTYPE html&gt;</span>
<span class="nt">&lt;html&gt;</span>
<span class="nt">&lt;head&gt;</span>
    <span class="nt">&lt;meta</span> <span class="na">charset=</span><span class="s">"utf-8"</span><span class="nt">&gt;</span>
    <span class="nt">&lt;title&gt;</span>Socket.IO with JWT Authentication<span class="nt">&lt;/title&gt;</span>
<span class="nt">&lt;/head&gt;</span>
<span class="nt">&lt;body&gt;</span>
    <span class="nt">&lt;h1&gt;</span>Socket.IO with JWT Authentication<span class="nt">&lt;/h1&gt;</span>
    <span class="nt">&lt;textarea</span> <span class="na">style=</span><span class="s">"width: 300px; height: 300px"</span><span class="nt">&gt;&lt;/textarea&gt;</span>
    <span class="nt">&lt;br&gt;</span>
    <span class="nt">&lt;button</span> <span class="na">onclick=</span><span class="s">"sendMessage()"</span><span class="nt">&gt;</span>Send message<span class="nt">&lt;/button&gt;</span>
    
    <span class="c">&lt;!-- Imports Socket.IO client library from CDN. Pay attention, the
        version of the client lib must match the version of the NPM package
        `socket.io` that is used in the Node.js backend. --&gt;</span>
    <span class="nt">&lt;script </span><span class="na">src=</span><span class="s">"https://cdnjs.cloudflare.com/ajax/libs/socket.io/4.0.0/socket.io.min.js"</span><span class="nt">&gt;&lt;/script&gt;</span>
    <span class="nt">&lt;script&gt;</span>
        <span class="c1">// Initiates the Socket.IO connection to the Node.js backend</span>
        <span class="kd">const</span> <span class="nx">socket</span> <span class="o">=</span> <span class="nx">io</span><span class="p">.</span><span class="nx">connect</span><span class="p">();</span>

        <span class="nx">socket</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">connect</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
            <span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Connected to server</span><span class="dl">'</span><span class="p">);</span>
        <span class="p">});</span>
        <span class="nx">socket</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">message</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">message</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
            <span class="nx">log</span><span class="p">(</span><span class="nx">message</span><span class="p">);</span>
        <span class="p">})</span>
        <span class="nx">socket</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">diconnect</span><span class="dl">'</span><span class="p">,</span> <span class="p">()</span> <span class="o">=&gt;</span> <span class="p">{</span>
            <span class="nx">log</span><span class="p">(</span><span class="dl">'</span><span class="s1">Disconnected from server</span><span class="dl">'</span><span class="p">);</span>
        <span class="p">});</span>

        <span class="kd">function</span> <span class="nx">log</span><span class="p">(</span><span class="nx">message</span><span class="p">)</span> <span class="p">{</span>
            <span class="nb">document</span><span class="p">.</span><span class="nx">getElementsByTagName</span><span class="p">(</span><span class="dl">'</span><span class="s1">textarea</span><span class="dl">'</span><span class="p">)[</span><span class="mi">0</span><span class="p">].</span><span class="nx">value</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementsByTagName</span><span class="p">(</span><span class="dl">'</span><span class="s1">textarea</span><span class="dl">'</span><span class="p">)[</span><span class="mi">0</span><span class="p">].</span><span class="nx">value</span> <span class="o">+</span> <span class="dl">'</span><span class="se">\n</span><span class="dl">'</span> <span class="o">+</span> <span class="nx">message</span><span class="p">;</span>
        <span class="p">}</span>

        <span class="cm">/**
         * Emits a `message` event with the payload `Hello, world` which
         * is send via the WebSocket connection to the Node.js backend.
         */</span>
        <span class="kd">function</span> <span class="nx">sendMessage</span><span class="p">()</span> <span class="p">{</span>
            <span class="nx">socket</span><span class="p">.</span><span class="nx">emit</span><span class="p">(</span><span class="dl">'</span><span class="s1">message</span><span class="dl">'</span><span class="p">,</span> <span class="dl">'</span><span class="s1">Hello, world</span><span class="dl">'</span><span class="p">);</span>
        <span class="p">}</span>
    <span class="nt">&lt;/script&gt;</span>
<span class="nt">&lt;/body&gt;</span>
<span class="nt">&lt;/html&gt;</span>
</code></pre></div></div>

<p>You can now restart the Node.js backend to see the changes in action as shown in the following GIF:</p>

<p><a class="img" href="/assets/2021-03-31/screenRecordingBrowser.gif">
    <img src="/assets/2021-03-31/screenRecordingBrowser.gif" alt="" />
</a></p>

<h2 id="4-jwt-generation">4. JWT generation</h2>

<p>For authentication, we’ll chose the Passport strategy <a href="http://www.passportjs.org/packages/passport-jwt/">passport-jwt</a>. This strategy is based on JSON Web Tokens (JWT).</p>

<p>JWT is an open standard that is based on signed JSON objects. In case of a successful login, the backend or an authorization server generates a JWT. A JWT consists of three parts: a header, the payload, and a signature. Header and payload are signed with a secret and then included into the JWT as the third part. The resulting JWT is returned to the client / browser. In the following, the browser includes the JWT with each HTTP request. That way, the backend can validate each HTTP request by comparing the signature with the header and payload. To learn more about JWT read this <a href="https://jwt.io/introduction">Introduction to JSON Web Tokens</a>.</p>

<p>To simplify this tutorial, we’ll skip the implementation of a login and the JWT generation. Instead, let’s mimic the authorization server by manually creating a valid JWT on <a href="https://jwt.io/#debugger-io">https://jwt.io/</a>:</p>

<ol>
  <li>Set the algorithm to <em>HS256</em> which is short for <em>HMAC-SHA256</em> and represents a symmetric key encryption.</li>
  <li>Change the payload so that it includes subject (<code class="language-plaintext highlighter-rouge">sub</code>) <em>1001</em> and name <em>admin</em>:
    <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
  </span><span class="nl">"sub"</span><span class="p">:</span><span class="w"> </span><span class="s2">"1001"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"admin"</span><span class="p">,</span><span class="w">
  </span><span class="nl">"iat"</span><span class="p">:</span><span class="w"> </span><span class="mi">1516044444</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div>    </div>
  </li>
  <li>For the sake of this tutorial, we’ll use <code class="language-plaintext highlighter-rouge">secret</code> as our secret to generate the JWT signature. <strong>Please note that you should use more complex secrets in production use cases and never hard-code your secret into your code.</strong></li>
</ol>

<p><a class="img" href="/assets/2021-03-31/jwtDebugger.png">
    <img src="/assets/2021-03-31/jwtDebugger.png" alt="" />
</a></p>

<p>This configuration yields the following JWT:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMDAxIiwibmFtZSI6ImFkbWluIiwiaWF0IjoxNTE2MDQ0NDQ0fQ.aWj5grFWJwcsbgxNJ7HdfL5PUfD8fMh9GwXutuR86GE
</code></pre></div></div>

<h2 id="5-integration-of-passport">5. Integration of Passport</h2>

<p>Add these imports in <code class="language-plaintext highlighter-rouge">server.js</code> to integrate Passport:</p>

<p class="file-name">server.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">passport</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">passport</span><span class="dl">'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">JWTStrategy</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">passport-jwt</span><span class="dl">'</span><span class="p">).</span><span class="nx">Strategy</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">ExtractJWT</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">passport-jwt</span><span class="dl">'</span><span class="p">).</span><span class="nx">ExtractJwt</span><span class="p">;</span>

<span class="kd">const</span> <span class="nx">User</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">./user-service</span><span class="dl">'</span><span class="p">);</span>
</code></pre></div></div>

<p>Register the <code class="language-plaintext highlighter-rouge">JWTStrategy</code>, specify the same secret as we previously did while generating the JWT, and set the algorithm to <em>HS256</em>:</p>

<p class="file-name">server.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">secret</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">SECRET</span> <span class="o">||</span> <span class="dl">'</span><span class="s1">secret</span><span class="dl">'</span><span class="p">;</span>

<span class="nx">passport</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="k">new</span> <span class="nx">JWTStrategy</span><span class="p">({</span>
    <span class="na">jwtFromRequest</span><span class="p">:</span> <span class="nx">ExtractJWT</span><span class="p">.</span><span class="nx">fromAuthHeaderAsBearerToken</span><span class="p">(),</span>
    <span class="na">jsonWebTokenOptions</span><span class="p">:</span> <span class="p">{</span>
        <span class="na">ignoreExpiration</span><span class="p">:</span> <span class="kc">false</span><span class="p">,</span>
    <span class="p">},</span>
    <span class="na">secretOrKey</span><span class="p">:</span> <span class="nx">secret</span><span class="p">,</span>
    <span class="na">algorithms</span><span class="p">:</span> <span class="p">[</span><span class="dl">'</span><span class="s1">HS256</span><span class="dl">'</span><span class="p">],</span>
<span class="p">},</span> <span class="p">(</span><span class="nx">jwtPayload</span><span class="p">,</span> <span class="nx">done</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="k">try</span> <span class="p">{</span>
        <span class="kd">const</span> <span class="nx">user</span> <span class="o">=</span> <span class="nx">User</span><span class="p">.</span><span class="nx">findById</span><span class="p">(</span><span class="nx">jwtPayload</span><span class="p">.</span><span class="nx">sub</span><span class="p">);</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">user</span><span class="p">)</span> <span class="p">{</span>
            <span class="k">return</span> <span class="nx">done</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="kc">false</span><span class="p">);</span>
        <span class="p">}</span>
        <span class="k">return</span> <span class="nx">done</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">user</span><span class="p">);</span>
    <span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">error</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">return</span> <span class="p">(</span><span class="nx">error</span><span class="p">,</span> <span class="kc">false</span><span class="p">);</span>
    <span class="p">}</span>
<span class="p">}));</span>

<span class="nx">passport</span><span class="p">.</span><span class="nx">serializeUser</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">user</span><span class="p">,</span> <span class="nx">done</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">if</span> <span class="p">(</span><span class="nx">user</span><span class="p">)</span> <span class="nx">done</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">user</span><span class="p">);</span>
<span class="p">});</span>

<span class="nx">passport</span><span class="p">.</span><span class="nx">deserializeUser</span><span class="p">(</span><span class="kd">function</span> <span class="p">(</span><span class="nx">id</span><span class="p">,</span> <span class="nx">done</span><span class="p">)</span> <span class="p">{</span>
    <span class="nx">done</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">id</span><span class="p">);</span>
<span class="p">});</span>
</code></pre></div></div>

<p>This Passport plugin automatically validates the JWT signature given our secret which we previously used to sign our JWT. If the JWT has not been tampered with, then Passport injects the JWT’s payload into the callback function. Passport refers to this callback as the <em>verify callback</em> (<a href="http://www.passportjs.org/docs/">see the docs to understand its purpose</a>). In this callback, we have to validate / search for a user that matches the credentials which have been specified in the JWT. In our case, we can search for a user that possesses the specified sub/subject ID. Passport then attaches the found user object to the Socket.IO request object which can be used in the following to apply user-specific actions on the server side (e.g. retrieve a user’s personal list of orders / recipes / etc.).</p>

<p>For this purpose, let’s define a dummy user service in <code class="language-plaintext highlighter-rouge">user-service.js</code> which contains only one valid user with name <code class="language-plaintext highlighter-rouge">admin</code> and id <code class="language-plaintext highlighter-rouge">1001</code>. This user corresponds to the values which we previously included in the payload of our generated JWT:</p>

<p class="file-name">user-service.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="dl">'</span><span class="s1">use strict</span><span class="dl">'</span><span class="p">;</span>

<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
    <span class="cm">/**
     * @param {string} userId
     * @returns {{id: string, name: string}}
     */</span>
    <span class="na">findById</span><span class="p">:</span> <span class="kd">function</span> <span class="p">(</span><span class="nx">userId</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">return</span> <span class="nx">users</span><span class="p">.</span><span class="nx">find</span><span class="p">(</span><span class="nx">u</span> <span class="o">=&gt;</span> <span class="nx">u</span><span class="p">.</span><span class="nx">id</span> <span class="o">===</span> <span class="nx">userId</span><span class="p">);</span>
    <span class="p">},</span>
<span class="p">};</span>

<span class="kd">const</span> <span class="nx">users</span> <span class="o">=</span> <span class="p">[</span>
    <span class="p">{</span>
        <span class="na">id</span><span class="p">:</span> <span class="dl">'</span><span class="s1">1001</span><span class="dl">'</span><span class="p">,</span>
        <span class="na">name</span><span class="p">:</span> <span class="dl">'</span><span class="s1">admin</span><span class="dl">'</span><span class="p">,</span>
    <span class="p">},</span>
<span class="p">];</span>
</code></pre></div></div>

<p>Finally, register Passport as a middleware for Socket.IO. Because middlewares in Socket.IO have a different function signature than those in Express, we have to define a wrapper function:</p>

<p class="file-name">server.js</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">wrapMiddlewareForSocketIo</span> <span class="o">=</span> <span class="nx">middleware</span> <span class="o">=&gt;</span> <span class="p">(</span><span class="nx">socket</span><span class="p">,</span> <span class="nx">next</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="nx">middleware</span><span class="p">(</span><span class="nx">socket</span><span class="p">.</span><span class="nx">request</span><span class="p">,</span> <span class="p">{},</span> <span class="nx">next</span><span class="p">);</span>
<span class="nx">io</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="nx">wrapMiddlewareForSocketIo</span><span class="p">(</span><span class="nx">passport</span><span class="p">.</span><span class="nx">initialize</span><span class="p">()));</span>
<span class="nx">io</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="nx">wrapMiddlewareForSocketIo</span><span class="p">(</span><span class="nx">passport</span><span class="p">.</span><span class="nx">session</span><span class="p">()));</span>
<span class="nx">io</span><span class="p">.</span><span class="nx">use</span><span class="p">(</span><span class="nx">wrapMiddlewareForSocketIo</span><span class="p">(</span><span class="nx">passport</span><span class="p">.</span><span class="nx">authenticate</span><span class="p">([</span><span class="dl">'</span><span class="s1">jwt</span><span class="dl">'</span><span class="p">])));</span>

<span class="nx">io</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="dl">'</span><span class="s1">connection</span><span class="dl">'</span><span class="p">,</span> <span class="p">(</span><span class="nx">socket</span><span class="p">)</span> <span class="o">=&gt;</span> <span class="p">{</span>
    <span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>

<p>If you restart the Node.js backend and then reload your browser tab at <a href="http://localhost:3000">http://localhost:3000</a>, then your browser should not be able to establish a WebSocket connection to the backend due to authentication failure. We’ll resolve this issue in the last step of this blog post.</p>

<p><a class="img" href="/assets/2021-03-31/connectionFailure.png">
    <img src="/assets/2021-03-31/connectionFailure.png" alt="" />
</a></p>

<h2 id="6-include-jwt-in-socketio-connection">6. Include JWT in Socket.IO Connection</h2>

<p>Per default, JSON Web Tokens are transmitted via the <code class="language-plaintext highlighter-rouge">Authorization</code> header. This HTTP header typically includes the keyword <code class="language-plaintext highlighter-rouge">Bearer</code> followed by a blank and the actual JWT. You can change this behavior by changing the line</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>jwtFromRequest: ExtractJWT.fromAuthHeaderAsBearerToken(),
</code></pre></div></div>
<p>in <code class="language-plaintext highlighter-rouge">server.js</code> which configures the Passport JWT strategy.</p>

<p>The last remaining step is to include the <code class="language-plaintext highlighter-rouge">Authorization</code> header in your website’s code which initiates the Socket.IO connection. Therefore, replace the existing connection setup in <code class="language-plaintext highlighter-rouge">index.html</code> with the following code:</p>

<p class="file-name">index.html</p>

<div class="language-javascript highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">socket</span> <span class="o">=</span> <span class="nx">io</span><span class="p">.</span><span class="nx">connect</span><span class="p">(</span><span class="dl">''</span><span class="p">,</span> <span class="p">{</span>
    <span class="na">extraHeaders</span><span class="p">:</span> <span class="p">{</span>
        <span class="na">Authorization</span><span class="p">:</span> <span class="dl">"</span><span class="s2">Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMDAxIiwibmFtZSI6ImFkbWluIiwiaWF0IjoxNTE2MDQ0NDQ0fQ.aWj5grFWJwcsbgxNJ7HdfL5PUfD8fMh9GwXutuR86GE</span><span class="dl">"</span><span class="p">,</span>
    <span class="p">},</span>
<span class="p">});</span>
</code></pre></div></div>

<p>After reloading your browser tab at <a href="http://localhost:3000">http://localhost:3000</a>, you should be able reestablish a Socket.IO connection.</p>

<h2 id="conclusion">Conclusion</h2>

<p>Thank you for reading this blog post and good luck with your project! You can find the complete source code of this demo application down below or in <a href="https://gist.github.com/philenius/641aebd1ba56769829e1fc7771326bf8">this gist on GitHub</a>. I hope that this blog post helped you to implement your own Node.js app with Socket.IO and integrate JWT authentication via Passport.</p>

<div style="display: table; margin: 0rem auto; margin-bottom: 2rem">
    <img src="/assets/2021-03-31/jwt-everywhere.jpg" />
</div>

<p>Entire code of this demo app:</p>

<script src="https://gist.github.com/philenius/641aebd1ba56769829e1fc7771326bf8.js"></script>]]></content><author><name></name></author><category term="web-development" /><category term="Passport" /><category term="Socket.IO" /><category term="WebSocket" /><category term="Authentication" /><category term="JWT" /><category term="Express" /><category term="Node.js" /><category term="JavaScript" /><summary type="html"><![CDATA[Passport is an authentication middleware for Node.js that provides dozens of pluggable authentication mechanisms. You might already have implemented a Node.js backend with Express (and Passport). And now you’re wondering how you can protect the WebSocket communication between backend and frontend from unauthenticated access? Perhaps you don’t want to implement authentication twice, but want to reuse your existing Passport authentication via JSON Web Tokens (or any other strategy e.g. basic auth)? Then, this blog post is meant for you!]]></summary></entry><entry><title type="html">How to develop Flutter apps for Windows desktop (August 2020)</title><link href="https://philenius.github.io/app-development/2020/08/23/how-to-develop-flutter-apps-for-windows-desktop-august-2020.html" rel="alternate" type="text/html" title="How to develop Flutter apps for Windows desktop (August 2020)" /><published>2020-08-23T14:00:00+02:00</published><updated>2020-08-23T14:00:00+02:00</updated><id>https://philenius.github.io/app-development/2020/08/23/how-to-develop-flutter-apps-for-windows-desktop-august-2020</id><content type="html" xml:base="https://philenius.github.io/app-development/2020/08/23/how-to-develop-flutter-apps-for-windows-desktop-august-2020.html"><![CDATA[<p>Flutter is an awesome framework for developing native cross-platform apps with a single code-base for Android, iOS, Windows, Linux, and macOS. In this blog post, I’ll explain how to set up your computer for developing Flutter apps which you can run on Windows desktop.</p>

<p>As recently <a href="https://medium.com/flutter/flutter-and-desktop-3a0dd0f8353e">stated by Tim Sneath</a>, the product manager for Flutter, support for Windows desktop <em>“remains in technical preview, and the APIs and tooling are not yet stable”</em>. Despite this, I was surprised how far Flutter’s desktop support has come. Writing Flutter apps for desktop is not much different than doing so for Android / iOS. Build times are fast. Of course, you can use pre-existing widgets or implement your own. Only, there aren’t many packages/plugins with desktop support, e.g. Flutter doesn’t provide access to the Windows file picker for selecting files.</p>

<p>Without further ado, let’s get started:</p>

<ol>
  <li>
    <p>Download the <a href="https://flutter.dev/docs/get-started/install/windows">latest installation bundle for Flutter</a>.</p>
  </li>
  <li>
    <p>Extract the zip file to your favorite destination. I’m going to choose <code class="language-plaintext highlighter-rouge">C:\bin\flutter</code>.</p>
  </li>
  <li>
    <p>Add the Flutter installation directory to your PATH so that you can run Flutter commands from your favorite command line/shell. In my case, it’s <code class="language-plaintext highlighter-rouge">C:\bin\flutter\bin</code>. Make sure that this directory contains the binaries <code class="language-plaintext highlighter-rouge">flutter</code> and <code class="language-plaintext highlighter-rouge">dart</code>.</p>
  </li>
  <li>
    <p>Open a terminal/console and type <code class="language-plaintext highlighter-rouge">flutter doctor</code>. This command evaluates your installations and informs you about missing platform dependencies. The output should look similar to this:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>flutter doctor
 Doctor summary <span class="o">(</span>to see all details, run flutter doctor <span class="nt">-v</span><span class="o">)</span>:
 <span class="o">[</span>√] Flutter <span class="o">(</span>Channel stable, 1.20.2, on Microsoft Windows <span class="o">[</span>Version 10.0.17763.107], locale en-DE<span class="o">)</span>
 <span class="o">[</span>X] Android toolchain - develop <span class="k">for </span>Android devices
     X Unable to locate Android SDK.
     Install Android Studio from: https://developer.android.com/studio/index.html
     On first launch it will assist you <span class="k">in </span>installing the Android SDK components.
     <span class="o">(</span>or visit https://flutter.dev/docs/get-started/install/windows#android-setup <span class="k">for </span>detailed instructions<span class="o">)</span><span class="nb">.</span>
     If the Android SDK has been installed to a custom location, <span class="nb">set </span>ANDROID_SDK_ROOT to that location.
     You may also want to add it to your PATH environment variable.
    
 <span class="o">[!]</span> Android Studio <span class="o">(</span>not installed<span class="o">)</span>
 <span class="o">[</span>√] VS Code <span class="o">(</span>version 1.48.1<span class="o">)</span>
 <span class="o">[!]</span> Connected device
     <span class="o">!</span> No devices available
    
 <span class="o">!</span> Doctor found issues <span class="k">in </span>3 categories.
</code></pre></div>    </div>

    <p>In this case, the output of <code class="language-plaintext highlighter-rouge">flutter doctor</code> shows three issues:</p>
    <ul>
      <li>The missing Android toolchain is not of relevance as we want to develop a Flutter app for Microsoft Windows.</li>
      <li>For developing Flutter apps, you can choose between Android Studio and Visual Studio Code. In my case, I have already installed the latter. I can highly recommend Visual Studio Code. So far, I have experienced no restrictions using VS Code for developing Flutter apps. More than that, VS Code is much more lightweight than Android Studio.</li>
      <li>There are no connected devices available. We’ll fix this issue later.</li>
    </ul>
  </li>
  <li>
    <p>Install either Visual Studio Code or Android Studio. If you chose to install Visual Studio Code, then you’ll need to install the Flutter plugin manually. For this purpose, open VS Code, go to the <em>Extensions</em> tab and search for <em>Flutter</em>. This plugin brings support for Dart and starting/stopping/debugging Flutter applications.</p>
  </li>
  <li>As of August 2020, you need to be on the <em>master channel</em> of Flutter, if you want to develop apps for Windows. Per default, the <em>stable</em> channel is activated. Let’s change this by running these two commands:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> flutter channel master
 flutter upgrade
</code></pre></div>    </div>
  </li>
  <li>Create your Flutter app and enable Windows desktop support:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> flutter create myapp
 <span class="nb">cd </span>myapp/
 flutter config <span class="nt">--enable-windows-desktop</span>
</code></pre></div>    </div>
  </li>
  <li>If you try to run the previously created app, Flutter will complain that there are no compatible devices. Run <code class="language-plaintext highlighter-rouge">flutter doctor</code> to find out why:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nv">$ </span>flutter doctor
 Doctor summary <span class="o">(</span>to see all details, run flutter doctor <span class="nt">-v</span><span class="o">)</span>:
 <span class="o">[</span>√] Flutter <span class="o">(</span>Channel master, 1.22.0-2.0.pre.48, on Microsoft Windows <span class="o">[</span>Version 10.0.17763.107], locale en-DE<span class="o">)</span>
 <span class="o">[</span>X] Android toolchain - develop <span class="k">for </span>Android devices
     X Unable to locate Android SDK.
       Install Android Studio from: https://developer.android.com/studio/index.html
       On first launch it will assist you <span class="k">in </span>installing the Android SDK components.
       <span class="o">(</span>or visit https://flutter.dev/docs/get-started/install/windows#android-setup <span class="k">for </span>detailed instructions<span class="o">)</span><span class="nb">.</span>
       If the Android SDK has been installed to a custom location, <span class="nb">set </span>ANDROID_SDK_ROOT to that location.
       You may also want to add it to your PATH environment variable.
    
 <span class="o">[</span>X] Visual Studio - develop <span class="k">for </span>Windows
     X Visual Studio not installed<span class="p">;</span> this is necessary <span class="k">for </span>Windows development.
       Download at https://visualstudio.microsoft.com/downloads/.
       Please <span class="nb">install </span>the <span class="s2">"Desktop development with C++"</span> workload, including all of its default components
 <span class="o">[!]</span> Android Studio <span class="o">(</span>not installed<span class="o">)</span>
 <span class="o">[</span>√] VS Code <span class="o">(</span>version 1.48.1<span class="o">)</span>
 <span class="o">[</span>√] Connected device <span class="o">(</span>1 available<span class="o">)</span>
    
 <span class="o">!</span> Doctor found issues <span class="k">in </span>3 categories.
</code></pre></div>    </div>

    <p>Oops, there’s a new issue. Developing Flutter apps for Windows requires the installation of Visual Studio.</p>
  </li>
  <li>
    <p><a href="https://visualstudio.microsoft.com/downloads/">Download and install Visual Studio</a> (the <em>Community</em> edition is free to use). <strong>Important:</strong> as in the output of <code class="language-plaintext highlighter-rouge">flutter doctor</code> denoted, please install the <em>Desktop development with C++</em> workload, including all of its default components.</p>

    <p><a class="img" href="/assets/2020-08-23/visualStudioInstallationWizard.png">
     <img src="/assets/2020-08-23/visualStudioInstallationWizard.png" alt="" />
 </a></p>
  </li>
  <li>If you re-run <code class="language-plaintext highlighter-rouge">flutter doctor</code>, then there should be only two issues remaining:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>flutter doctor
Doctor summary <span class="o">(</span>to see all details, run flutter doctor <span class="nt">-v</span><span class="o">)</span>:
<span class="o">[</span>√] Flutter <span class="o">(</span>Channel master, 1.22.0-2.0.pre.48, on Microsoft Windows <span class="o">[</span>Version 10.0.17763.107], locale en-DE<span class="o">)</span>
<span class="o">[</span>X] Android toolchain - develop <span class="k">for </span>Android devices
    X Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you <span class="k">in </span>installing the Android SDK components.
      <span class="o">(</span>or visit https://flutter.dev/docs/get-started/install/windows#android-setup <span class="k">for </span>detailed instructions<span class="o">)</span><span class="nb">.</span>
      If the Android SDK has been installed to a custom location, <span class="nb">set </span>ANDROID_SDK_ROOT to that location.
      You may also want to add it to your PATH environment variable.
    
<span class="o">[</span>√] Visual Studio - develop <span class="k">for </span>Windows <span class="o">(</span>Visual Studio Community 2019 16.7.2<span class="o">)</span>
<span class="o">[!]</span> Android Studio <span class="o">(</span>not installed<span class="o">)</span>
<span class="o">[</span>√] VS Code <span class="o">(</span>version 1.48.1<span class="o">)</span>
<span class="o">[</span>√] Connected device <span class="o">(</span>1 available<span class="o">)</span>
    
<span class="o">!</span> Doctor found issues <span class="k">in </span>2 categories.
</code></pre></div>    </div>

    <p>As stated before, the issues w.r.t. the Android toolchain and Android Studio can be ignored in our case.</p>
  </li>
  <li>
    <p>Finally, we can run our Flutter app. You can do so in Visual Studio Code by opening one of the Dart files in <code class="language-plaintext highlighter-rouge">myapp/lib/</code> and pressing F5 on your keyboard. Alternatively, you can execute <code class="language-plaintext highlighter-rouge">flutter run</code> in the root directory of your app:</p>

    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>flutter run
Launching lib<span class="se">\m</span>ain.dart on Windows <span class="k">in </span>debug mode...
Building Windows application...
Waiting <span class="k">for </span>Windows to report its views...                           2ms
Syncing files to device Windows...                               1,030ms
    
Flutter run key commands.
r Hot reload.
R Hot restart.
h Repeat this <span class="nb">help </span>message.
d Detach <span class="o">(</span>terminate <span class="s2">"flutter run"</span> but leave application running<span class="o">)</span><span class="nb">.</span>
c Clear the screen
q Quit <span class="o">(</span>terminate the application on the device<span class="o">)</span><span class="nb">.</span>
An Observatory debugger and profiler on Windows is available at: http://127.0.0.1:51854/YXlEoTVxsOE<span class="o">=</span>/
</code></pre></div>    </div>
  </li>
  <li>Congratulation, you successfully created your first Flutter app for Windows desktop!!! You should see the Flutter demo app:</li>
</ol>

<p><a class="img" href="/assets/2020-08-23/flutterDemoApp.png">
    <img src="/assets/2020-08-23/flutterDemoApp.png" alt="" />
</a></p>

<div style="height: 2rem"></div>

<blockquote>
  <p>Final notes: you can run <code class="language-plaintext highlighter-rouge">flutter build windows</code> to build an executable file without debug banner. The resulting EXE file can be found at <code class="language-plaintext highlighter-rouge">./build/windows/runner/Release/myapp.exe</code>. Further information on Flutter desktop shells can be found at <a href="https://github.com/flutter/flutter/wiki/Desktop-shells">the Flutter wiki</a>. Happy coding!</p>
</blockquote>

<div style="display: table; margin: 0rem auto">
    <div style="width: 100%">
        <iframe src="https://giphy.com/embed/LmNwrBhejkK9EFP504" width="100%" height="300px" frameborder="0" class="giphy-embed" allowfullscreen=""></iframe>
    </div>
</div>]]></content><author><name></name></author><category term="app-development" /><category term="Flutter" /><category term="Dart" /><category term="desktop" /><category term="Windows" /><summary type="html"><![CDATA[Flutter is an awesome framework for developing native cross-platform apps with a single code-base for Android, iOS, Windows, Linux, and macOS. In this blog post, I’ll explain how to set up your computer for developing Flutter apps which you can run on Windows desktop.]]></summary></entry><entry><title type="html">Installing Spin Model Checker including iSpin on Ubuntu 19.04</title><link href="https://philenius.github.io/software-quality/2020/04/09/installing-spin-on-ubuntu-19.html" rel="alternate" type="text/html" title="Installing Spin Model Checker including iSpin on Ubuntu 19.04" /><published>2020-04-09T09:00:00+02:00</published><updated>2020-04-09T09:00:00+02:00</updated><id>https://philenius.github.io/software-quality/2020/04/09/installing-spin-on-ubuntu-19</id><content type="html" xml:base="https://philenius.github.io/software-quality/2020/04/09/installing-spin-on-ubuntu-19.html"><![CDATA[<p>For the lecture <em>Advanced Software Quality</em> at Aalen University, I had to demonstrate the usage of Spin model checker for the verification of software models. iSpin represents a graphical interface for editing and executing models and for analyzing the results. iSpin invokes Spin commands in the background and graphically presents the results.</p>

<p><a class="img" href="/assets/2020-04-09/ispin.png">
   <img src="/assets/2020-04-09/ispin.png" alt="" />
</a></p>

<p>Following these steps, I managed to run Spin model checker on Ubuntu:</p>

<ol>
  <li>Download the Spin binary from Github and extract it to your favorite installation directory:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">INSTALL_DIR</span><span class="o">=</span>/opt
<span class="nb">cd</span> <span class="nv">$INSTALL_DIR</span>

wget https://github.com/nimble-code/Spin/archive/version-6.5.2.tar.gz
<span class="nb">tar </span>xfz version-6.5.2.tar.gz
</code></pre></div>    </div>
  </li>
  <li>Build the Spin binary:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt-get <span class="nb">install</span> <span class="nt">-y</span> build-essential byacc flex graphviz
<span class="nb">cd</span> <span class="nv">$INSTALL_DIR</span>/Spin-version-<span class="k">*</span>/Src
make
<span class="nb">sudo cp </span>spin /usr/bin/
</code></pre></div>    </div>
  </li>
  <li>This is how you can execute a simple model written in <a href="http://spinroot.com/spin/Man/Quick.html">Promela</a>, the modeling language for Spin, from the command line:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spin <span class="nv">$INSTALL_DIR</span>/Spin-version-<span class="k">*</span>/Examples/hello.pml
</code></pre></div>    </div>
  </li>
  <li>The usage of iSpin requires the installation of <em>tk</em>, a graphical user interface toolkit:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt-get <span class="nb">install</span> <span class="nt">-y</span> tk
<span class="nb">cd</span> <span class="nv">$INSTALL_DIR</span>/Spin-version-<span class="k">*</span>/optional_gui/
./ispin.tcl
</code></pre></div>    </div>
  </li>
</ol>]]></content><author><name></name></author><category term="software-quality" /><category term="spin" /><category term="ispin" /><category term="model checker" /><category term="software quality" /><category term="verification" /><summary type="html"><![CDATA[For the lecture Advanced Software Quality at Aalen University, I had to demonstrate the usage of Spin model checker for the verification of software models. iSpin represents a graphical interface for editing and executing models and for analyzing the results. iSpin invokes Spin commands in the background and graphically presents the results.]]></summary></entry><entry><title type="html">Nextcloud desktop client gets stuck on ‘Account access’ view / during login</title><link href="https://philenius.github.io/cloud/2019/10/24/nextcloud-desktop-client-stuck-at-login-loop.html" rel="alternate" type="text/html" title="Nextcloud desktop client gets stuck on ‘Account access’ view / during login" /><published>2019-10-24T00:00:00+02:00</published><updated>2019-10-24T00:00:00+02:00</updated><id>https://philenius.github.io/cloud/2019/10/24/nextcloud-desktop-client-stuck-at-login-loop</id><content type="html" xml:base="https://philenius.github.io/cloud/2019/10/24/nextcloud-desktop-client-stuck-at-login-loop.html"><![CDATA[<p>Having set up Nextcloud with Docker and secured over TLS, I wanted to connect my Netcloud desktop client. I was successfully able to login into Nextcloud in my browser. But somehow, the desktop client would always get stuck on the login page of Nextcloud.</p>

<h2 id="scenario">Scenario</h2>

<p>I use NGINX to act as a reverse proxy for Nextcloud. TLS termination happens at the NGINX. That is, the connection between NGINX and Nextcloud is not secured.</p>

<p><a class="img" href="/assets/2019-10-24/scenarioTLSTermination.png">
   <img src="/assets/2019-10-24/scenarioTLSTermination.png" alt="" />
</a></p>

<h2 id="gotcha">Gotcha</h2>

<p>There are different versions of the Nextcloud desktop client (snap package vs. Ubuntu/Debian packages PPA). Using the snap package of the Nextcloud desktop client, I was able to figure out why the desktop client would always get stuck on the login page:</p>

<p>Somehow, the login website contains an HTML form with an action that points to a URL with plain HTTP connection (no TLS encryption). At the same time, the website itself is served through a TLS encrypted connection. My browser, here Firefox, then denies this insecure HTTP request.</p>

<p><a class="img" href="/assets/2019-10-24/nextcloudDesktopLoginStuckAtGrantAccess.png">
   <img src="/assets/2019-10-24/nextcloudDesktopLoginStuckAtGrantAccess.png" alt="Nextcloud desktop login" />
</a></p>

<p>It turned out, that Nextcloud cannot know that it is served by a reverse proxy which encrypts all HTTP traffic to the outside world. Therefore, some URLs generated by Nextcloud lack the HTTP<strong>S</strong>.</p>

<h2 id="solution">Solution</h2>

<p>To solve this problem with incorrect URLs, Nextcloud offers a configuration parameter <code class="language-plaintext highlighter-rouge">overwriteprotocol</code> which tells Nextcloud that it is running behind a reverse proxy with TLS encryption. If one sets this value to <code class="language-plaintext highlighter-rouge">https</code>, Nextcloud will automatically generate correct URLs.</p>

<p>Unfortunately, Nextcloud only offers a few environment variables for configuration. The majority of configuration can only be
done through files. Changing configuration files of dockerized applications is always a struggle.</p>

<p>As the documentation states, the main configuration of Nextcloud is stored inside the file <code class="language-plaintext highlighter-rouge">config/config.php</code>. But, because the official documentation doesn’t mention an absolute file path, I tried to edit <code class="language-plaintext highlighter-rouge">/var/www/html/config/config.php</code> at first hand (<strong>DO NOT EDIT THIS FILE! That’s the wrong way!</strong>).</p>

<p>Here are the correct steps to <em>edit</em> or - better extend - <code class="language-plaintext highlighter-rouge">config.php</code> for Nextcloud inside the Docker image:</p>

<ol>
  <li>
    <p>The documentation of Nextcloud states:</p>

    <blockquote>
      <p>Nextcloud supports loading configuration parameters from multiple files. You can add arbitrary files ending with .config.php in the config/ directory, for example you could place your email server configuration in email.config.php. This allows you to easily create and manage custom configurations, or to divide a large complex configuration file into a set of smaller files. These custom files are not overwritten by Nextcloud, and the values in these files take precedence over config.php.</p>
    </blockquote>
  </li>
  <li>
    <p>So, I created a separate configuration file <code class="language-plaintext highlighter-rouge">tls.config.php</code> and added my specific configuration:</p>

    <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span>
<span class="nv">$CONFIG</span> <span class="o">=</span> <span class="k">array</span> <span class="p">(</span>
  <span class="s1">'overwriteprotocol'</span> <span class="o">=&gt;</span> <span class="s1">'https'</span><span class="p">,</span>
<span class="p">);</span>
</code></pre></div>    </div>
  </li>
  <li>Then, I created a Dockerfile in which I extended the official docker image and added my individual coniguration:
    <div class="language-Dockerfile highlight highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> nextcloud:17.0.0-apache</span>
   
<span class="k">COPY</span><span class="s"> tls.config.php /usr/src/nextcloud/config/</span>
<span class="k">RUN </span><span class="nb">chmod </span>440 /usr/src/nextcloud/config/tls.config.php
</code></pre></div>    </div>
  </li>
  <li>If you use plain Docker, you’re ready to build your new image and run it:
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker build <span class="nt">-t</span> nextcloud:17.0.0-apache-customized <span class="nb">.</span>
docker run <span class="nt">-p</span> 8080:80 nextcloud:17.0.0-apache-customized
</code></pre></div>    </div>

    <p>If you use Docker Compose, then you need to add a build step to your <code class="language-plaintext highlighter-rouge">docker-compose.yml</code> and change the image name so that Docker won’t remove the tag from the original Nextcloud Docker image:</p>
    <div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nn">---</span>
<span class="na">version</span><span class="pi">:</span> <span class="s1">'</span><span class="s">3'</span>
<span class="na">services</span><span class="pi">:</span>
  <span class="na">nextcloud</span><span class="pi">:</span>
    <span class="na">image</span><span class="pi">:</span> <span class="s2">"</span><span class="s">nextcloud:17.0.0-apache-customized"</span>
    <span class="na">build</span><span class="pi">:</span> <span class="s">.</span>
    <span class="na">restart</span><span class="pi">:</span> <span class="s">always</span>

<span class="nn">...</span>
</code></pre></div>    </div>
    <p>Make sure to add the <code class="language-plaintext highlighter-rouge">--build</code> option when starting your stack:</p>
    <div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker-compose up <span class="nt">--build</span>
</code></pre></div>    </div>
  </li>
</ol>]]></content><author><name></name></author><category term="cloud" /><category term="nextcloud" /><category term="docker" /><category term="configuration" /><summary type="html"><![CDATA[Having set up Nextcloud with Docker and secured over TLS, I wanted to connect my Netcloud desktop client. I was successfully able to login into Nextcloud in my browser. But somehow, the desktop client would always get stuck on the login page of Nextcloud.]]></summary></entry><entry><title type="html">Transfer files using a USB cable to the reMarkable</title><link href="https://philenius.github.io/devices/2019/10/18/transfer-files-using-a-usb-cable-to-remarkable.html" rel="alternate" type="text/html" title="Transfer files using a USB cable to the reMarkable" /><published>2019-10-18T20:00:00+02:00</published><updated>2019-10-18T20:00:00+02:00</updated><id>https://philenius.github.io/devices/2019/10/18/transfer-files-using-a-usb-cable-to-remarkable</id><content type="html" xml:base="https://philenius.github.io/devices/2019/10/18/transfer-files-using-a-usb-cable-to-remarkable.html"><![CDATA[<p>I decided to try out a so called <em>digital notepad</em> for university so that I can take notes on my lecture manuscripts. I ended up buying the <a href="https://remarkable.com/">reMarkable</a> because it seemed to be the most complete concept (there are a lot of hacks available on Github to fix some of the <em>reMarkable</em>’s flaws).</p>

<p><a class="img" href="/assets/2019-10-18/reMarkableNote.jpg">
    <img src="/assets/2019-10-18/reMarkableNote.jpg" alt="reMarkable Note" />
</a></p>

<p>One flaw of the <em>reMarkable</em> is the file synchronization between your computer and the <em>reMarkable</em>. You cannot connect your <em>reMarkable</em> to Dropbox, Google Drive, OneDrive, ownCloud, Nextcloud, etc. There are only two ways to sync your files:</p>

<ol>
  <li>
    <p>Use the <em>reMarkable</em> app (no Linux support) for synchronization through their cloud service.</p>
  </li>
  <li>
    <p>Experimental feature: transfer files over USB.</p>
  </li>
</ol>

<p>In this blog post, I will briefly share my experiences with the file sync over USB to help people with their decision whether the <em>reMarkable</em> fits their needs.</p>

<h1 id="steps">Steps</h1>

<ol>
  <li>
    <p>Turn on your <em>reMarkable</em>.</p>
  </li>
  <li>
    <p>Go to the device settings, select the tab <code class="language-plaintext highlighter-rouge">Storage</code> and make sure that <code class="language-plaintext highlighter-rouge">Enable USB web interface (Beta)</code> is turned on:
<a class="img" href="/assets/2019-10-18/reMarkableEnableWebInterface.jpg">
<img src="/assets/2019-10-18/reMarkableEnableWebInterface.jpg" alt="" class="image-margin" />
</a></p>
  </li>
  <li>
    <p>Connect your <em>reMarkable</em> to your computer using the micro USB cable.
<a class="img" href="/assets/2019-10-18/gretaMeme.jpg">
<img src="/assets/2019-10-18/gretaMeme.jpg" alt="" class="center-image" />
</a></p>
  </li>
  <li>
    <p>Open your favourite browser and navigate to <code class="language-plaintext highlighter-rouge">http://10.11.99.1/</code>:
<a class="img" href="/assets/2019-10-18/reMarkableWebUiScreenshot.png">
<img src="/assets/2019-10-18/reMarkableWebUiScreenshot.png" alt="" class="center-image" />
</a></p>
  </li>
  <li>
    <p>You can now download and upload PDF and EPUB files:</p>
    <video width="100%" src="/assets/2019-10-18/screencaptureUploadDownload.mp4" loop="" autoplay="" controls=""></video>
  </li>
</ol>

<h1 id="observations">Observations</h1>

<ul>
  <li>You can only upload files through drag-and-drop.</li>
  <li>You must refresh the website in your browser to check whether the file was successfully uploaded.</li>
  <li>You can only download files through a righ click on a file and then through selecting the context menu entry <code class="language-plaintext highlighter-rouge">Download</code>.</li>
  <li>Depending on the file size and the amount of annotations &amp; notes in your file, it might take a few seconds until the download starts.</li>
  <li>Folders that have been created on the <em>reMarkable</em> are visualized on this website. You can navigate this folder structure and upload / download documents to each specific folder.</li>
</ul>

<blockquote>
  <p><strong>Good news:<br />
downloaded PDF files from the <em>reMarkable</em> automatically contain all annotations, notes, drawings &amp; sketches.</strong></p>
</blockquote>

<h1 id="special-gift-pdf-file-size">Special gift: PDF file size</h1>

<p>I created a dummy PDF file with images and a total of 5 pages. I uploaded this file to the <em>reMarkable</em>, added a lot of notes and sketches on it and then downloaded it again. The file size increased by about 50 % (from 0.886 MB to 1.3 MB). This observation contrasts with several <a href="https://www.reddit.com/r/RemarkableTablet/comments/8ihqbs/massive_problem_when_exporting_notespdfs_from/">comments on Reddit</a> about exploding file sizes after exporting PDF files from the <em>reMarkable</em>.</p>

<p>To form your own opinion, have a look at the <a href="/assets/2019-10-18/loremIpsum.pdf">original PDF file</a> and the <a href="/assets/2019-10-18/loremIpsumEdited.pdf">edited PDF file</a>.</p>

<p>For further details, see the <a href="https://support.remarkable.com/hc/en-us/articles/360002661337-Transferring-files-using-a-USB-cable">official documentation</a> for file sync over USB.</p>

<blockquote>
  <p>Write down in the comments below, if you have any questions. I’d be happy to help you.</p>
</blockquote>]]></content><author><name></name></author><category term="devices" /><category term="e-ink" /><category term="table" /><category term="digital notepad" /><category term="file transfer" /><category term="sync" /><category term="reMarkable" /><summary type="html"><![CDATA[I decided to try out a so called digital notepad for university so that I can take notes on my lecture manuscripts. I ended up buying the reMarkable because it seemed to be the most complete concept (there are a lot of hacks available on Github to fix some of the reMarkable’s flaws).]]></summary></entry><entry><title type="html">How to create user interfaces in Unity for Google Daydream VR apps</title><link href="https://philenius.github.io/virtual-reality/2019/03/02/how-to-create-user-interfaces-in-unity-for-google-daydream-vr-apps.html" rel="alternate" type="text/html" title="How to create user interfaces in Unity for Google Daydream VR apps" /><published>2019-03-02T00:41:04+01:00</published><updated>2019-03-02T00:41:04+01:00</updated><id>https://philenius.github.io/virtual-reality/2019/03/02/how-to-create-user-interfaces-in-unity-for-google-daydream-vr-apps</id><content type="html" xml:base="https://philenius.github.io/virtual-reality/2019/03/02/how-to-create-user-interfaces-in-unity-for-google-daydream-vr-apps.html"><![CDATA[<p>In the last two months, I’ve been experimenting with <em>Google Daydream View</em> and I created a few demo applications in Unity. Throughout my experiments, I stumbled upon the difficulty to create user interfaces in VR. In this blog post, I’ll describe how you can create UIs for your VR app build with the <a href="https://github.com/googlevr/gvr-unity-sdk">Google VR SDK for Unity</a>.</p>

<h2 id="types-of-ui">Types of UI</h2>

<p>In general, you can differentiate between two different kinds of user interfaces.</p>

<h3 id="non-diegetic">Non-diegetic</h3>
<p>Non-diegetic UIs are typically used to display the player’s health or score. This kind of UI is often referred to as HUD (Heads Up Display). The UI sticks to the user’s view by following the head movement of the user.</p>

<p><a class="img" href="/assets/2019-03-01/nonDiegeticUserInterfaceVirtualRealityDaydream.gif">
  <img src="/assets/2019-03-01/nonDiegeticUserInterfaceVirtualRealityDaydream.gif" alt="" />
</a></p>

<h3 id="spatial-ui">Spatial UI</h3>
<p>Spatial UIs are user interfaces that are placed somewhere in the 3D environment of your scene. They stick to a certain position in the world so that the user has to move its head in order to see it.</p>

<p><a class="img" href="/assets/2019-03-01/spatialUserInterfaceVirtualRealityDaydream.gif">
  <img src="/assets/2019-03-01/spatialUserInterfaceVirtualRealityDaydream.gif" alt="" />
</a></p>

<h2 id="steps-for-creating-a-user-interface">Steps for creating a user interface</h2>

<p>At this point, I assume that you have already setup a scene in Unity that contains all game objects required for Daydream.</p>

<p><a class="img" href="/assets/2019-03-01/unityBasicSetupDaydreamScene.png">
  <img src="/assets/2019-03-01/unityBasicSetupDaydreamScene.png" alt="" />
</a></p>

<p>Now, to create a user interface follow these steps:</p>

<ol>
  <li>Add a canvas to the hierarchy of game objects.
    <ul>
      <li>If you want to create a non-diegetic UI, then add the canvas as a child of your main camera like so:<br />
<a class="img" href="/assets/2019-03-01/unitySceneHierarchyNonDiegeticUI.png">
 <img src="/assets/2019-03-01/unitySceneHierarchyNonDiegeticUI.png" alt="" />
</a></li>
      <li>If you want to create a spatial UI, then place the canvas as a sibling to the player’s game object like so:<br />
<a class="img" href="/assets/2019-03-01/unitySceneHierarchySpatialUI.png">
 <img src="/assets/2019-03-01/unitySceneHierarchySpatialUI.png" alt="" />
</a></li>
    </ul>
  </li>
  <li>
    <p>Select the canvas and set the <code class="language-plaintext highlighter-rouge">Render Mode</code> in the inspector tab to <code class="language-plaintext highlighter-rouge">World Space</code>.</p>
  </li>
  <li>
    <p>Resize the canvas until it fits the player’s view.</p>
  </li>
  <li>
    <p>Add the script <code class="language-plaintext highlighter-rouge">GvrPointerGraphicRaycaster</code> to the canvas. If you have multiple canvas’ in your scene, then you must add this script to each canvas.</p>
  </li>
  <li>
    <p>Remove the script <code class="language-plaintext highlighter-rouge">Graphic Raycaster</code> from the canvas. Otherwise, your main camera will draw a second raycast on your UI and will interfer with the raycasts send by your controller / laser.</p>
  </li>
  <li>
    <p>Select the main camera and add the script <code class="language-plaintext highlighter-rouge">GvrPointerPhysicsRaycaster</code>.</p>
  </li>
  <li>Now, you can add a button or any other UI control to your canvas. The player can automatically interact with the UI elements with the Daydream controller.</li>
</ol>]]></content><author><name></name></author><category term="virtual-reality" /><category term="Google Daydream" /><category term="Unity" /><category term="VR" /><category term="virtual reality" /><summary type="html"><![CDATA[In the last two months, I’ve been experimenting with Google Daydream View and I created a few demo applications in Unity. Throughout my experiments, I stumbled upon the difficulty to create user interfaces in VR. In this blog post, I’ll describe how you can create UIs for your VR app build with the Google VR SDK for Unity.]]></summary></entry></feed>