function61.com

LabelCloud

Fri, 17 Mar 2017 10:55:00 +0000

Use case

You need to print labels from your web-based application, whether you are in healthcare, logistics or some other sector. You have two solutions regarding the implementation:

a) Build the label rendering and printing logic inside your application.
b) Call some other software to do the label rendering and printing for you.

LabelCloud is the software that does everything for you from rendering to producing the label printer specific control code and pushing that code to the printer.

LabelCloud is available both as a cloud service or as a on-premises version where you require tighter data privacy controls or want to control availability yourself.

Demo

Supported printer manufacturers

Toshiba TEC
Zebra ZPL/ZPL2 based printers
PCL-based printers (most laser printers print in PCL, some label printers)
Other printer manufacturers via PDF + GDI-based printing
Other control language implementations considered on a case-by-case basis if you’re ordering a license.

Architecture: how does it work?

LabelCloud is a webservice into which you can make a HTTP+JSON call with the text/barcode contents of the data you want to print as a label.

Basically, you do this:

POST /labelcloud HTTP/1.1
Content-Type: application/json

[
	{ label data to print + into which type of printer }
]

And it responds with the correct printer controlling code that can be pushed into the printer. There’s a client-side JavaScript library for that. All this is done with HTML5 + JavaScript with no browser plugins required - thus it works in every browser.

The solution requires you to install NX Print (provider for low-level access to the printer from web applications) software for the workstation, which listens on a HTTP port (not allowed to be contacted from outside of the computer), to provide access to Windows’ printers.

Licensing

Security

Thu, 16 Mar 2017 22:25:00 +0200

Security policy

Whenever our users use our products, there’s an implicit trust agreement between our user and us not to screw things up. We’d like to make this agreement explicit by promising the following:

We do everything in our power to secure access to our systems by keeping systems up-to-date, using good authentication practices like strong, unique passwords everywhere and hardware security modules where applicable.
Use encryption where it’s sensible to do so (transport layer stuff like HTTPS, data encryption at rest where it’s practical)
Never to store your password without current best-practices hashing (we won’t event know your password). Never to store your credit card data on our servers but use partners with PCI compliant systems.
We will design our systems rather in a paranoid manner than a trusting manner: even datacenter-internal traffic between nodes is encrypted. LAN is not a magical safe place separated by a firewall from the outside bad world.
We keep tabs on security researchers’ blogs and tweets.
Any severe security incidents will be disclosed publicly (in our blog), to our customers via email and listed on this page.

Vulnerability reporting

We highly appreciate responsible vulnerability disclosures.

If you would like to report a vulnerability, or have any security concerns with our product, please reach out to us by email. Our PGP key is on Keybase.

For non-critical matters, we prefer that you open an issue with the appropriate product:

For open source products, use the GitHub issues.
For all other products, file a ticket in our support system.

Security incident history

No security incidents. Let’s keep it that way!

Introduction to WAL (write-ahead logging)

Thu, 23 Feb 2017 12:24:00 +0200

WAL is a concept to achieve atomicity & durability - usually found in filesystems and database systems.

Both atomicity (the “A” in ACID) and durability (the “D” in ACID) are properties of ACID - the set of properties used to describe database systems.

What is atomicity & durability? Why do I want it?

When you add or insert data to, say an SQL database, when the database system responds with “OK - I saved your data”, it has to be sure that even in the face of a power loss or application crash, the updated data is still found from the file. This is known as durability.

When a database saves data, it almost always changes more than one value. If the operation fails for some reason, we don’t want the end result to be one value ending up as changed but the other value as unchanged. We want either all things to happen or none at all. This is known as atomicity. Imagine a banking system where a money transfer debits from your account but is never credited to the receiver. Not atomic.

Imagine ordering something from an online shop like Amazon, paying for it and never receiving the order.

When you ask the customer service about it, they don’t find your order. It turns out that during the order their computer crashed and the saved order was just lost. Not durable.

You would be pretty bummed about this, so competent tech companies use systems for saving data that guarantee atomicity & durability so we don’t suffer these problems.

The same goes for filesystems - you don’t want to lose all your files if your computer crashes.

Aren’t file writes atomic & durable by default?

Unfortunately, no! Yes I know I previously said that filesystems are atomic & durable, but those guarantees only apply at the filesystem level. The filesystem is protected against corruption, but the individual files are not. This is something that has to be achieved at the application level!

To begin explaining this, let’s oversimplify how a database system might write to a file. We might have a “users” table, stored in a file in a filesystem as users.txt:

id=1 username=joonas.fi registered=2017-02-01

Now, let’s do an insert into the database: INSERT INTO users (id, username, registered) VALUES (2, "hello", "2017-02-23")

After that write the file should look like this:

id=1 username=joonas.fi registered=2017-02-01
id=2 username=hello     registered=2017-02-23

The database system used pseudocode like this to write that change to the file:

fwrite("users.txt", "id=2 username=hello     registered=2017-02-23")

But, the system write call used to write to the file is not atomic (the “A” in ACID). Atomicity in short means that either the whole write call succeeds or the whole write call fails. And we don’t have that, so in the face of power loss or application crash we can easily end up with this in users.txt:

id=1 username=joonas.fi registered=2017-02-01
id=2 userna

Or even this:

id=1 username=joonas.fi registered=2017-02-01
           me=hello     registered=2017-02-23

Because nothing guarantees that the bytes written to the file are written in the order that you specified.

Writing to a file can always fail. There is no getting around this.

But I’ve heard about fsync, it fixes this right?

fsync alone does not fix this. fsync() is only a mechanism to ask the operating system to wait until all the pending writes are actually confirmed as written on the disk.

So, without fsync have this pseudocode:

fwrite("users.txt", "id=2 username=hello     registered=2017-02-23")

print("Changes were successfully written to users.txt")

This code is lying, since fwrite() can under the hood do anything it wants. Usually for performance reasons the operating system either:

Queues the write to the disk as a background task. It will finish soon, maybe.
Or even batches/buffers the write and waits for more writes to come so it can more efficiently write more data as a single operation.

This is where we can fix the above code with fsync()

fwrite("users.txt", "id=2 username=hello     registered=2017-02-23")

fsync("users.txt")

print("Changes were successfully written to users.txt")

Now the code is correct. fsync() tells the OS to wait until all the buffers are flushed to disk. When we receive the success message, we can be sure we’re not being lied to.

Okay fsync() solves durability, i.e. confirmed data written to disk is durable!

But what about atomicity? What if the fwrite() fails while we’re waiting for the fsync() to finish?

Good question! Like we explained before, nothing still guarantees that the fwrite() call succeeds or fails as a whole.

The fwrite() call can always land on the disk only partially when an application/operating system crashes or the power goes out. So with fsync() we get atomicity & durability but only if it succeeds. And it is not guaranteed to succeed and thus we can still end up with corrupted data (no atomicity).

In short, sequence of events:

fwrite() start
– queue write op 1
– queue write op 2
fwrite() returns
fsync() start
– wait for write op 1 to finish
– wait for write op 2 to finish
fsync() returns (atomicity & durability boundary only achieved here)

Remember: the power can go out in any of those states. Only in state #8 we are atomic & durable. In all the other states before we will end up with corrupted data if we’re not careful.

Ok, so how does WAL achieve atomicity & durability?

The problem boils down to:

We don’t want incomplete data in users.txt. We either want the write to fail as a whole or succeed as a whole.

That is human speak for wanting atomicity and durability.

So finally, this is where the subject, write-ahead logging, comes in!

WAL in essence: before writing the line to users.txt, we write first to write-ahead-log.txt:

wal-entry line=2 content=[id=2 username=hello     registered=2017-02-23]

Sidenote: WAL’s usually use byte offsets instead of line offsets. This is just easier to explain.

After we have fsync()ed the WAL, we can start writing to users.txt. If during that write the power goes out and we end up with:

line 1 | id=1 username=joonas.fi registered=2017-02-01
line 2 | id=2 userna

After we restart the database system, it scans the write-ahead-log.txt first and notices that according to the WAL the line 2 in users.txt is not what it should be and repairs it:

line 1 | id=1 username=joonas.fi registered=2017-02-01
line 2 | id=2 username=hello     registered=2017-02-23

It doesn’t matter if the repair operation fails because the power goes out again during the reparing fwrite(), because after database restart the repair operation will start again and eventually reach valid state.

As long as the WAL is intact, we are safe.

Okay that’s cool, but what if write to the WAL fails?

Excellent question again, champ! So we know that every fwrite() can fail mid-write, so writing the “repair entry” to WAL can fail as well and no amount of fsync() fixes that?

Correct. The trick is implementing the WAL in such a way that it is resilient to corruption. I.e. we need to be able to detect corrupted WAL entries - the ones that can occur before fsync() completes and power goes out. Corrupted WAL could look like this:

wal-entry line=2 content=[id=2 username=hello     registered=2017-02-23]
wal-entry line=3 content=[id=3 user

Ok, clearly that is corrupted. So when trying to write WAL entry for line 3 the fsync() never finished and we wound up with that.

But because the fsync() didn’t finish we didn’t confirm to the user that the change was applied - instead the user got an error (or timeout because the power went out), and she can know for sure that the operation didn’t go through. Or perhaps the application even automatically re-tries with a healthy server and the user never suffers from this error.

Remember:

We never acknowledge writes to database without fsync() to WAL.

We never write to users.txt before the WAL entry is fully fsync()ed, because only durable & atomic WAL entries can repair the tracked file.

Moving on, because we didn’t finish writing the WAL entry, the original users.txt was not even tried to be modified and thus is in entirely valid state. Only succesfully written WAL entries will be written to users.txt because only those entries can be repaired.

Broken WAL entries are discarded (either removed or ignored) and writes to users.txt correctly reported to clients as either succeeding or failing as a whole atomic operation, in a durable fashion.

Atomicity and durability. Thanks to WAL! Congratulations, you now understand one of the most fundamental aspects of a database system! :)

Recapping

With WAL:

The tracked file (users.txt) is effectively (after possible repair) always fully atomic & durable.
The log file (write-ahead-log.txt) as a whole is never atomic & durable.
- Individual WAL entries though are atomic & durable.
- We just need to be able to detect and deal with corrupt WAL entries.

Reverse-engineering with strace

Thu, 12 Jan 2017 12:00:00 +0200

Let’s learn some reverse-engineering with strace by inspecting how Docker interacts with its server API (Docker uses client-server model).

Docker has API documentation so it’s not like this is necessary, but this serves as a good example on how to reverse-engineer black boxes.

Install strace

First, make sure you have strace, curl and jq installed:

$ apt-get install -y strace curl jq

System calls? What and why?

strace stands for “system trace”. With strace we can trace system calls. System calls are calls that programs make to communicate with the Kernel.

Kernel calls are needed for any I/O such as disk, network etc. Why are they needed? Because the only way for a program to read/write to/from disk or to/from network is to utilize disk/network drivers, and it would be stupid for each program to implement those drivers themselves. Instead, the drivers are implemented only once and they are hosted in the Kernel. Userspace programs (= “normal programs”) communicate with those drivers by calling functions in the Kernel (“syscall”).

Why would it be stupid to implement disk/network/etc drivers ourselves? If you used to play DOS games in ye oldie (but golden) times, you probably recall that you had to choose and configure a sound card driver before you could play that game. That meant that each game had to contain sound card drivers for each type of device. That was inefficient in both time and money, as each game developer had to develop drivers for each sound card (or even video card) and also probably purchase those cards and test the drivers against them. It also slowed the pace of innovation and competition, as it took some time for new games to support newer sound cards that came to the market.

So in short: system calls are awesome because they let your program just abstractly ask the Kernel to read a file, and not care about whether:

The disk is a HDD, SSD, CD-ROM or Blu-ray.
It’s attached via SATA, IDE, USB or even as a network drive.
The filesystem is FAT32, NTFS, ext4 etc.
The block level on the disk is encrypted or not.
Etc etc.

Quick introduction to strace

Okay, let’s run a short program first without using strace:

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial

cat is a program that prints a file to stdout - usually which is connected eventually to your screen.

Now, let’s run that very same program but with strace:

$ strace cat /etc/os-release
execve("/bin/cat", ["cat", "/etc/os-release"], [/* 21 vars */]) = 0
... snipped ...
open("/etc/os-release", O_RDONLY)       = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=274, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6f4c4e000
read(3, "NAME=\"Ubuntu\"\nVERSION=\"16.04.1 L"..., 131072) = 274
write(1, "NAME=\"Ubuntu\"\nVERSION=\"16.04.1 L"..., 274NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial
) = 274
read(3, "", 131072)                     = 0
munmap(0x7fb6f4c4e000, 139264)          = 0
close(3)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

strace dumps its tracing output to stderr, while stdout is directly connected to the running program.

That means that:

By running strace cat /etc/os-release we see both the output from strace and the original program (as pictured above).
By running strace cat /etc/os-release > /dev/null we ignore stdout => we only see output from strace.
By running strace cat /etc/os-release 2> /dev/null we ignore stderr => we only see output from the original program (this is effectively useless).

Demonstration:

$ strace cat /etc/os-release 2> /dev/null
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial

$ strace cat /etc/os-release 1> /dev/null
execve("/bin/cat", ["cat", "/etc/os-release"], [/* 21 vars */]) = 0
... snipped ...
open("/etc/os-release", O_RDONLY)       = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=274, ...}) = 0
fadvise64(3, 0, 0, POSIX_FADV_SEQUENTIAL) = 0
mmap(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f171a572000
read(3, "NAME=\"Ubuntu\"\nVERSION=\"16.04.1 L"..., 131072) = 274
write(1, "NAME=\"Ubuntu\"\nVERSION=\"16.04.1 L"..., 274) = 274
read(3, "", 131072)                     = 0

The “ignore stdout” is relevant because if your original program produces much output, we might not be interested in that at all.

So, from the above output we can see that cat program:

Opens a file (open) /etc/os-release as readonly (O_RDONLY).
Issues read calls to file descriptor #3, receiving the file contents (NAME="Ubuntu"\n...).
Other file descriptors would be #0 for stdin, #1 for stdout, #2 for stderr and 3..N are other open file descriptors.
In this case the file that was specified in the only argument to cat ended up as fd #3.
Immediately after reading the content, cat outputs the same content into its own stdout: write(1, same content as what was read...).

Okay, now that we know the basics of strace, let’s move on.

Using strace with real-life programs

Okay, we have this docker command, that we want to reverse engineer:

$ docker service ps whoami
ID                         NAME      IMAGE                     NODE    DESIRED STATE  CURRENT STATE              ERROR
aps1k88b6rv0dcxnh6tstgjc4  whoami.1  emilevauge/whoami:latest  master  Running        Running about an hour ago

That command interrogates the Docker API to list tasks (whoami.1, identified by aps1k88b6rv0dcxnh6tstgjc4) running for a specific service (whoami).

Okay, since we know that docker is a client for a client-server model, therefore it accesses the server API, probably over TCP or a Unix socket. Both of those we can crudely inspect with strace.

We know that Docker uses HTTP protocol to communicate with the API, so we’ll search for a string GET (= start of most HTTP requests) to validate that assumption:

$ strace docker service ps whoami | grep GET
execve("/usr/bin/docker", ["docker", "service", "ps", "whoami"], [/* 21 vars */]) = 0
uname({sysname="Linux", nodename="master", ...}) = 0
brk(NULL)                               = 0x2360000
brk(0x23611c0)                          = 0x23611c0
... snipped ...

Okay, we made our first mistake. So we grepped for GET but none of the output lines contain that keyword, so what’s going on?

Remember that strace (as each program) outputs two streams: stdout and stderr. The log output we wanted to grep from is in stderr, but the shell | operator pipes stdout to the next program. So we were grepping from the wrong stream. This drawing illustrates the problem:

+------------------------------+      +-------------+                  +-------------------+
|                         +------+    |NAME="Ubuntu"|     +------+     |                   |
|                         |stdout+---->...          +----->$ grep+----->                   |
|                         +------+    +-------------+     +------+     |                   |
| $ strace cat /etc/os-release |                                       |    Your screen    |
|                         +------+    +-------------+                  |                   |
|                         |stderr+---->execve()     +------------------>                   |
|                         +------+    |...          |                  |                   |
+------------------------------+      +-------------+                  +-------------------+

So, knowing that, let’s redirect strace’s stderr to stdout so we can grep it (2>&1):

$ strace docker service ps whoami 2>&1 | grep GET

Nothing, so we were expecting a write call with a string GET, but there was no matches. Let’s try just listing the write calls:

$ strace docker service ps whoami 2>&1 | grep write

Nothing, that’s weird! When running again:

$ strace docker service ps whoami 2>&1 | grep write
write(3, "GET /v1.24/services/whoami HTTP/"..., 95) = 95

Why are those write calls randomly showing up and sometimes not? Well, that relates to the underlying programming language used by Docker, the Go language. Go takes advantage of processors with multiple cores by spawning multiple processes, and sometimes that write call shows up in the main process that we launched, and sometimes not.

There’s an option to strace that we can use:

-f      follow forks

This means that when Go spawns child processes, strace monitors those as well. We probably should’ve used that switch by default, so we don’t have to take guesses about the inner workings of each program to debug from now on. Continuing with that:

$ strace -f docker service ps whoami 2>&1 | grep write
[pid  9337] <... write resumed> )       = 95
[pid  9337] write(3, "GET /v1.24/tasks?filters=%7B%22s"..., 160 <unfinished ...>
[pid  9337] <... write resumed> )       = 160
[pid  9336] write(3, "GET /v1.24/services/7g1xbqf95ly6"..., 114 <unfinished ...>
[pid  9336] <... write resumed> )       = 114
[pid  9338] write(3, "GET /v1.24/nodes/7xynabmkp3r117p"..., 111 <unfinished ...>
... snipped ...

Observation: each outbound HTTP request had its own process (-f option prefixes each line with a process id). So they must take heavy use of Go’s goroutines for easy parallelization of computation. :)

Now, each time we run that same command we consistently see all the writes, awesome!

Now moving forward with our initial plan of only scanning for the GET strings to identify only the calls relating to outbound HTTP GET requests:

$ strace -f docker service ps whoami 2>&1 | grep GET
[pid  9361] ioctl(2, TCGETSstrace: Process 9365 attached
[pid  9361] ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
[pid  9361] ioctl(1, TCGETS, 0x7ffecdce30c0) = -1 ENOTTY (Inappropriate ioctl for device)
[pid  9364] write(3, "GET /v1.24/services/whoami HTTP/"..., 95 <unfinished ...>
[pid  9363] write(3, "GET /v1.24/tasks?filters=%7B%22s"..., 160 <unfinished ...>
[pid  9364] write(3, "GET /v1.24/services/7g1xbqf95ly6"..., 114) = 114
[pid  9364] write(3, "GET /v1.24/nodes/7xynabmkp3r117p"..., 111) = 111

Okay, strace seems to truncate the write command, because we’re not seeing the write argument in its entirety - we are not seeing the entire URLs which we are interested about. There’s a switch for that:

-s strsize     limit length of print strings to STRSIZE chars (default 32)

So:

$ strace -s 64 -f docker service ps whoami 2>&1 | grep GET
[pid  9377] ioctl(2, TCGETS <unfinished ...>
[pid  9377] ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
[pid  9377] ioctl(1, TCGETS <unfinished ...>
[pid  9381] write(3, "GET /v1.24/services/whoami HTTP/1.1\r\nHost: docker\r\nUser-Agent: D"..., 95) = 95
[pid  9381] write(3, "GET /v1.24/tasks?filters=%7B%22service%22%3A%7B%227g1xbqf95ly6bt"..., 160 <unfinished ...>
[pid  9380] write(3, "GET /v1.24/services/7g1xbqf95ly6btrwyglymrigt HTTP/1.1\r\nHost: do"..., 114) = 114
[pid  9380] write(3, "GET /v1.24/nodes/7xynabmkp3r117pj9rxhln9k9 HTTP/1.1\r\nHost: docke"..., 111) = 111

# better. let's also:
# - filter only to the `write` lines that contain keyword GET
# - increase string length to 96 because we were not seeing the /tasks?filters=... URL fully

$ strace -s 96 -f docker service ps whoami 2>&1 | grep write | grep GET
[pid  9442] write(3, "GET /v1.24/services/whoami HTTP/1.1\r\nHost: docker\r\nUser-Agent: Docker-Client/1.12.3 (linux)\r\n\r\n", 95 <unfinished ...>
[pid  9439] write(3, "GET /v1.24/tasks?filters=%7B%22service%22%3A%7B%227g1xbqf95ly6btrwyglymrigt%22%3Atrue%7D%7D HTTP"..., 160 <unfinished ...>
[pid  9442] write(3, "GET /v1.24/services/7g1xbqf95ly6btrwyglymrigt HTTP/1.1\r\nHost: docker\r\nUser-Agent: Docker-Client/"..., 114 <unfinished ...>
[pid  9442] write(3, "GET /v1.24/nodes/7xynabmkp3r117pj9rxhln9k9 HTTP/1.1\r\nHost: docker\r\nUser-Agent: Docker-Client/1.1"..., 111 <unfinished ...>

Okay, we know that Docker makes a HTTP request to path /v1.24/services/whoami. Does it use HTTP or a Unix socket?

Searching from the full output of strace just before the first write we find:

$ strace -s 64 -f docker service ps whoami 1>/dev/null | less -S
... snip ...
[pid  9401] setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
[pid  9401] connect(3, {sa_family=AF_LOCAL, sun_path="/var/run/docker.sock"}, 23) = 0
... snip ...
[pid  9401] write(3, "GET /v1.24/services/whoami HTTP/1.1\r\nHost: docker\r\nUser-Agent: D"..., 95 <unfinished ...>
... snip ...

Okay, we saw that it uses a Unix socket and its location in the filesystem is /var/run/docker.sock.

Lets try connecting to Docker API ourself without the official Docker client:

$ curl --unix-socket /var/run/docker.sock http:/v1.24/services/whoami
# snipped: non-pretty JSON output in one line

# if you have jq installed, you can pipe JSON into it and have it pretty-printed
$ curl --unix-socket /var/run/docker.sock http:/v1.24/services/whoami | jq .
{
  "ID": "7g1xbqf95ly6btrwyglymrigt",
  "Version": {
    "Index": 808
  },
  "CreatedAt": "2017-01-11T12:37:55.171414273Z",
  "UpdatedAt": "2017-01-11T13:19:20.515932494Z",
  "Spec": {
    "Name": "whoami",
    "Labels": {
      "traefik.enable": "true",
... snipped
}

Now, checking out the /tasks?filters... endpoint:

$ curl --unix-socket /var/run/docker.sock http:/v1.24/tasks?filters=%7B%22service%22%3A%7B%227g1xbqf95ly6btrwyglymrigt%22%3Atrue%7D%7D | jq .
[
  {
    "ID": "aps1k88b6rv0dcxnh6tstgjc4",
    "Version": {
      "Index": 728
    },
    "CreatedAt": "2017-01-11T12:37:55.17994955Z",
    "UpdatedAt": "2017-01-11T12:38:02.142646711Z",
    "Spec": {
      "ContainerSpec": {
        "Image": "emilevauge/whoami:latest",
        "Env": [
          "VIRTUAL_HOST=whoami._CLUSTER_.fn61.net"
        ]
      },
... snipped ...
    },
    "ServiceID": "7g1xbqf95ly6btrwyglymrigt",
    "Slot": 1,
    "NodeID": "7xynabmkp3r117pj9rxhln9k9",
    "Status": {
      "Timestamp": "2017-01-11T12:38:02.097532639Z",
      "State": "running",
      "Message": "started",
      "ContainerStatus": {
        "ContainerID": "a748e970d01cf55fea3550c29f769f2ccd81f018e12b002b4cfbedb9f60a1f0c",
        "PID": 27839
      }
    },
    "DesiredState": "running",
... snipped ...
  }
]

And /nodes/7xynabmkp3r117pj9rxhln9k9:

curl --unix-socket /var/run/docker.sock http:/v1.24/nodes/7xynabmkp3r117pj9rxhln9k9 | jq .
{
  "ID": "7xynabmkp3r117pj9rxhln9k9",
... snipped ...
  "Description": {
    "Hostname": "master",
    "Platform": {
      "Architecture": "x86_64",
      "OS": "linux"
    },
... snipped ...
}

Okay cool, so we managed to make the same requests as the Docker client made, with only the curl tool. HTTP is awesome! :)

So, recapping from the output of strace we reverse-engineered the logic for getting the output of $ docker service ps whoami:

Make request to /services/whoami.
Find out that the service ID is 7g1xbqf95ly6btrwyglymrigt.
Make request to /tasks?filters={"service":{"7g1xbqf95ly6btrwyglymrigt":true}} (for clarity I urldecoded the garbage that looked like this: %7B%22service%22..).
Make request to /nodes/7xynabmkp3r117pj9rxhln9k9 to find out the details of the node the task is running on.

So from above HTTP requests we found the data from the JSON responses that go into the final ps output:

$ docker service ps whoami
ID                         NAME      IMAGE                     NODE    DESIRED STATE  CURRENT STATE        ERROR
aps1k88b6rv0dcxnh6tstgjc4  whoami.1  emilevauge/whoami:latest  master  Running        Running 3 hours ago

The task ID aps1k88b6rv0dcxnh6tstgjc4 comes from response from /tasks (Task.ID).
Task name is combination of service.Name and task.Slot (whoami + 1 => whoami.1).
Image we know from /tasks (Task.Spec.ContainerSpec.Image).
Node name was from /nodes/7xynabmkp3r117pj9rxhln9k9 (Node.Description.Hostname).
Desired state was from /tasks (Task.DesiredState).
Current state was from /tasks (Task.Status.State).
Error would be from /tasks as well (would be Task.Status.State=failed, Task.Status.Err=task: non-zero exit (1)).

Closing words

strace is awesome, as it helps you understand how a program works behind the scenes (reverse engineering), without needing access to the source code. And it doesn’t matter whether the program you are inspecting is open or closed source, or written in Python, C++ or Go, because in the end almost everything happens via syscalls.

Running a company-internal certificate authority

Mon, 09 Jan 2017 18:35:00 +0200

If you have not read my previous post on How do SSL certificates work?, go read it first!

Moving on, this post is about setting up an certificate authority for company-internal use.

“Company-internal” CA?

Most CA setups are for company-internal usage, unless you are planning on starting multi-million dollar CA business by investing years and some serious cash to get through start-of-operation audits, periodical audits, setting up hardcore secure infrastructure with tight access controls, TPM modules and such. :)

Company-internal essentially means that we use a self-signed root certificate to provide certificates only internal to our company. That means that because we don’t have to (or want to) fill all the requirements of those above mentioned public CA’s, we can easily use all the benefits (like client certificate authentication) of public key infrastructure to our advantage.

We use this to provide authentication and TLS-encryption for internal (“backoffice”) applications.

Security implications?

Even though this is for internal usage, our CA is used to authenticate into mission critical internal services that must not be compromised. Thus, the encryption keys backing the certificates must be protected really well. In addition to that, one should think of:

Set up the root CA to be in an offline and air-gapped setup (think Raspberry Pi without network).
Use multi-level CA (online or offline, depending on needs) architecture (“intermediate CA”), so intermediate CAs can be revoked.
Implement an online certificate revocation list, possibly hosted on AWS S3.
Have the CA’s private key stored in a TPM (or equivalent) module.

We don’t currently implement the above points, but we should look into that in the future.

This sounds hard - why do it?

Good question! Not all companies use SSL certificates - probably because it’s hard to set up and manage. But the tooling is getting better and easier, so I hope client cert authentication is here to stay. Here are a few benefits:

Once the CA is set up, configuring server apps to use it is usually pretty damn easy.
Client certificate authentication is implemented in pretty much every programming language and HTTP server library there is. You don’t have to reinvent the authentication/authorization layers, and thus they are probably more secure than anything else you or anyone else would write.
If you have 10 services where the user can log in and you hire a new employee, granting access does not have to mean adding the user manually to those 10 services. If those services use client cert authentication, all you have to do is issue the certificate to the user and not touch those services. Certificate authentication is effectively offline access control, as the server does not have to contact the CA to ask if this certificate has authorization to this service. The CA signs this authorization once, and the user effectively delivers this signature herself to the service.
Since you’re implementing authentication on top of TLS, you get guaranteed encryption for free. This means that even if your employess log in from an unsecured network from a coffee shop, your data is safe.
Client certificate authentication protects you from keyloggers, as you don’t have to type in a username and password. Though this fact can be diluted by the fact that if a keylogger compromises your computer, another malicious program could as well steal your private keys related to your client certs (unless you use a hardware security module).

How do I use the certificates issued by a CA?

For concrete usage, see our loadbalancer configuration.

Choosing a toolkit for CA management

Choosing a toolkit for this task was not easy. I was not fond of using OpenSSL, since it requires keeping more state than should be required: verbose and complex config files and counters which more modern toolkits statelessly implement with randomization. OpenSSL did not felt modern and as automation friendly as I had hoped.

Ultimately, Cloudflare’s cfssl fit this task rather nicely. It supports automation out of the box very nicely via JSON and it doesn’t require any configuration, if all you want to do is call if via command line. And it’s a nice bonus that I already trust Cloudflare as their long-time customer.

However, their documentation was really lacking (thanks CoreOS for having a better tutorial!), and the compiled binaries on their official website were outdated and there was bug or feature lacking in -initca feature that did not respect the expiration date in the certificate signing request. I had to statically compile the binaries myself. Why statically? So they work on Alpine linux that has a different standard library.

More alternatives and research is documented on the below mentioned repository!

How do I set up a CA?

Since I had to take notes while setting up a CA anyway, I figured I’d open source my research in the form of an example CA (pretty much 1:1 our production setup): github.com/function61/certificate-authority

If you’re reading this far, you are probably interested of this subject. Check out that repo to learn more!

How do SSL certificates work?

Wed, 04 Jan 2017 22:48:00 +0200

In this article I try to explain how the increasingly encrypted internet works!

Disclaimer: I am intentionally simplifying a few things, since this is already a complicated subject.

What is PKI?

PKI (public key infrastructure) is the basis of TLS (Transport Layer Security, previously known as SSL, but we still speak of SSL certificates).

PKI is basically a collection of conventions on dealing with public key cryptography and trust relationships between public keys by using certificates. You could say that PKI = public key crypto + certificates.

Okay, but what do we use PKI for?

+-------+   +------+   +-------+
|       |   |      |   |       |
| HTTPS |   | FTPS |   | IMAPS |
|       |   |      |   |       |
+-------+   +------+   +-------+

+------------------------------+
|                              |
|             TLS              |
|                              |
+------------------------------+

+------------------------------+
|                              |
|             PKI              |
|                              |
+------------------------------+

From the above diagram, you can see the relationships between these technologies:

TLS is what makes encrypted HTTP (called HTTPS), FTP and IMAP possible.
Therefore, TLS is not just about HTTPS.
PKI is what makes TLS possible.

Taken further, public key crypto is not just for PKI and PKI is not just for TLS:

+-------+   +------+   +-------+
|       |   |      |   |       |
| HTTPS |   | FTPS |   | IMAPS |
|       |   |      |   |       |
+-------+   +------+   +-------+

+------------------------------+   +-----------+
|                              |   |           |
|             TLS              |   |    VPN    |
|                              |   |           |
+------------------------------+   +-----------+

+----------------------------------------------+   +-------+
|                                              |   |       |
|                      PKI                     |   |  SSH  |
|                                              |   |       |
+----------------------------------------------+   +-------+

+----------------------------------------------------------+
|                                                          |
|                Public key cryptography                   |
|                                                          |
+----------------------------------------------------------+

Observations:

SSH does not use PKI (because it does not use certificates).
But PKI and SSH both use public key crypto, so they are related technologies.
VPN also uses certificates, so it uses PKI.

What actually is a certificate

So, previously we learned that HTTPS is built on TLS which is built on PKI, which means public key crypto + certificates.

Certificates are basically just:

A name (like www.google.com).
Our public key.
Issuer’s public key (like from Comodo’s CA).
Issuer’s signature (proves that issuer vouches that this certificate is legit => details like our name and public key are true).

Also directly related to our certificate is our private key, but like the name implies it is always kept secret and is never transmitted to anybody else. But the public and private key are mathematically related and thus by having the private key we can prove that we are the owner of the public key. By knowing the public key you cannot know the private key. That essentially is the basis of public key cryptography.

So each certificate has an issuer (almost 100 % of certs, except for root certificates which we’ll cover soon), and only one issuer. This means that each issuer has probably issued many certificates. This means that you can think of certificates like it was a directory structure in a filesystem:

+---------+
|         |
| Root CA |
|         |
+----+----+
     |
     |    +-----------------+
     |    |                 |
     +----> Intermediate CA |
          |                 |
          +-------------+---+
                        |
                        |   +---------------+
                        |   |               |
                        +---> Server cert 1 |
                        |   |               |
                        |   +---------------+
                        |
                        |   +---------------+
                        |   |               |
                        +---> Server cert 2 |
                        |   |               |
                        |   +---------------+
                        |
                        |   +---------------+
                        |   |               |
                        +---> Client cert 1 |
                            |               |
                            +---------------+

From above picture:

Issuer of “Root CA” = (nobody)
Issuer of “Intermediate CA” = “Root CA”
Issuer of “Server cert 1” = “Intermediate CA”

Also from the above picture you’ll see that there are a few types of certificates:

Root certificates,
Intermediate certificates,
Server certificates and
Client certificates.

These are basically just a convention, and they technically don’t differ much from each other - they all are just certificates. Here are the differences:

Server and client certificates can not issue certificates (the issuer decides this restriction when signing a certificate), for security reasons.
Server certificates use their hostname as certificate name to prove that you are connecting to the correct server. This means that the issuer also vouches that you own that hostname, and therefore you have to prove domain ownership to SSL cert issuers when getting server SSL certs.
Client certificate’s name has no meaning, as the client presents the certificate to the server during connection to prove its identity only - the client doesn’t have a hostname (or one that has any meaning).
Intermediate certificate is basically like a server certificate but it is given the permission to issue sub certificates.
Root certificate is like intermediate certificate but it is the only type of certificate that nobody has issued, i.e. it does not have a parent certificate. That’s why it is called a root certificate. These are also called self-signed certificates.

So how do SSL certificates work with HTTPS connections

When you connect to “www.google.com”, the server must present a server certificate saying that its name is really “www.google.com”. If your browser receives a certificate for any other name (like bing.com), or something else is wrong with it (like signatures not matching or certificate having expired), your browser refuses to connect to the website and warns you that this seems dangerous.

In addition to the name, the server must also sign the certificate with its private key to prove, that the server really is authoritative for that certificate. Remember that certificates are public knowledge, so just by downloading Google’s certificate you cannot claim to be Google! (unless you have the private key as well)

Okay, but can’t anybody just make a certificate with the name “www.google.com” but with a different public and private key, since the browser mainly cares about the name in the certificate but the public key has no direct meaning? Yes, anybody can make such a certificate, but no trustworthy certificate authority will sign it for you (signing = issuing). Therefore your malicious certificate would only be:

Self-signed (root certificate), but your browser only trusts a small list of trustworthy roots => no trust OR:
Properly signed by some some malicious certificate authority but its certificate would not anchor to a trustworthy root => no trust.

PKI boils down to this:

You only trust certificates that anchor to trusted root certificates.

Anchoring just means that when you walk the parent issuer path to the root, you can trust the certificate if you trust the root certificate. And the root certificates are owned by trusted certificate authorities that promise to do good job of issuing certificates only to entities that prove that they own the hostname they are asking the certificate to be issued for, and also that all other details in the certificate hold true.

Thus, if www.google.com presents these certificates (this is called a trust chain):

www.google.com -> Google Internet Authority -> GeoTrust Global CA

.. your browser goes through this thought process:

I do not trust “www.google.com”, but it has “Google Internet Authority” as an issuer.
I do not trust “Google Internet Authority”, but it has “GeoTrust Global CA” as an issuer.
I do trust “GeoTrust Global CA”!
Therefore I know that “www.google.com” can be trusted, awesome!

This can be summed up with:

                            +--------------+
                     trusts |              |
    +-----------------------+ Your browser | +----------------+
    |                       |              |                  |
    |                       +--------------+                  |
    |                                                         |
    |                                                         | indirect trust
    |                                                         |
    |                                                         |
    |                                                         |
    |                                                         +
    |
+---v---------+            +---------------+            +----------------+
|             | trusts     |               | trusts     |                |
| GeoTrust CA +------------> Google CA     +------------> www.google.com |
|             |            |               |            |                |
+-------------+            +---------------+            +----------------+

This trust chain can technically be really long, but in practice it’s only a few levels.

Tell me more about root certificates

So, in the previous section’s example I told that your computer trusts “GeoTrust Global CA”, why is that?

Your computer/device/whatever holds a static list of root CAs that are configured as trustworthy (= trust anchors).

Take a look yourself, on Windows hit start menu and search for Manage computer certificates, go to Trusted Root Certification Authorities > Certificates and you’ll see this list:

Unsurprisingly, “GeoTrust Global CA” was on that list. :)

Where does that list come from, and who manages that list? Biggest software vendors have their own list of root certificates that these vendors (certificate authorities) have to apply to be included into:

Mozilla CA Certificate program - Firefox & most open source ecosystem.
Microsoft Trusted Root Certificate Program - most applications that run on Windows use this list.
Apple Root Certificate Program - Apple’s ecosystem.

The process is quite heavy, costly and requires third party audits initially and regularly. But it’s a good business - SSL certs have been somewhat expensive and it’s a huge industry.

For PKI to work as a whole, there has to be some sort of consensus on what vendors of root CAs are included in the list. If your root cert is on Mozilla’s list but not Microsoft’s, nobody would buy your CA’s issued certificates because your certificate would only work on some browsers and some devices. And different vendors take different amount of time to update the list to the end products (operating system updates, device updates, software updates etc.), so it takes quite a while even after you’ve been accepted to the vendors’ root programs for your issued certificates to become so widespread that your customers would buy your issued certificates.

Sidebar: I’ve used the term “CA” and “root certificate” quite interchangably. Technically, the root certificate represents the CA (Certificate Authority) and that certificate is used to issue sub certificates. That certificate also represents the vendor that (probably sells) issues the sub certificates, so the vendor itself could also be called a CA.

Can we trust the CAs? Isn’t PKI broken by design?

Good questions! So yes, the whole thing breaks down if even one of the CAs gets compromised or acts knowingly maliciously. And these things do happen:

In 2015 Google beat up and quite rightly humiliated Symantec after discovering that during testing Symantec had generated certificates posing as Google services.
In 2012 Trustwave issued a man-in-the-middle certificate that compromised the integrity of the whole PKI ecosystem, with Mozilla reacting with a patch denouncing any Trustwave issued sub-CA certs.
In 2011 DigiNotar went bankrupt for having a security incident that compromised the whole internet and having quite rightly been booted off all vendors’ root CA programs - a clear death blow to any CA business.

So yes, if there are so many instances of misuse made public, think of all the malicious activity that has not made it to the public, and think of all the government actors, NSA being a dick etc.

I’m with you if you’re thinking that this system is broken by design, it seems so weird to build a centralized system of power which we have to blindly trust!

The good thing is that the ecosystem is being actively improved by introducing Google’s certificate transparency (“CT”) project - essentially requiring all CA-issued certificates to be publicly disclosed, and that way if any foul play happens, it will be noticed and dealt with accordingly.

In the centralized design of PKI we cannot prevent malicious use, but by moving 100 % to CT we can at least detect malicious activity, kill malicious/incompetent CA vendors and start trusting PKI again.

Of course CT is a transitional project and not all CAs yet implement CT, but things will gradually improve and eventually all trusted CA certificates will either implement CT or be booted off all vendors’ lists.

How much do SSL certificates cost?

Doesn’t cost much, depending on certificate type (there are subtypes to server certificates but it’s not that important), cheapest go for ~ 8 USD/year.

Of course the cost if relative - for a business that’s nothing, but for a hobby site or a personal blog that might be too much, and it’s the reason most of the web is not encrypted yet and thus we are still using insecure connection channels.

Of course progress is being made and this landscape is also changing by making SSL certs finally free and making the web democratic again:

Let’s Encrypt - finally making SSL certs free and accessible to everyone.
Cloudflare - offering even their free customers free SSL certificates.

Summing up

There’s no denying that PKI/SSL is a complicated subject, also part of the reason why not all websites are encrypted yet (this is too hard), and we didn’t even discuss the encryption that goes into actually securing your traffic between the devices on the internet.

Honestly, there is probably a lot more about PKI/SSL certs that I don’t know yet and I’ve probably made some mistakes trying to explain the mess that is this subject. But I hope you gained some insight, and if you have any comments/questions, drop them below! :)

Introducing Buildbot (open source)

Wed, 28 Dec 2016 13:51:00 +0300

Buildbot is a build server like Jenkins, but stateless, super lightweight and aimed for building Dockerized apps.

Head over to our GitHub repository for more details!

Website launch

Fri, 23 Dec 2016 16:38:53 +0300

function61.com got its website launched! Hope you enjoy it, and any feedback would be appreciated.

Even Ron Swanson would approve of this site:

Disclaimer: Ron Swanson might or might not approve of this site.

Huge thanks to:

HTML5 UP for the design
Hugo for the modern static website engine
Unsplash for the imagery

About our values

Thu, 22 Dec 2016 22:24:34 +0200

Building software is a craft

Building great software is really hard - software fails all the time either to work or keep up with expectations. Most people may not realize it but software engineering is a craft with almost endless amount of things to learn. You can never make perfect software, but the real talent is trying your very best to be far above average and always learn how to improve things by having an open and critical mind.

User experience

We are passionate about making the user experience so good that you’ll like using our software. Especially in corporate settings most software tend to be boring and even hated because of poor usability. We’re trying our hardest to change that! Try our software to see it for yourself!

Security

Concrete steps we take to respect your security and privacy:

All of our websites are served over secure channel only. Unencrypted http is auto-upgraded to https.
We use Cloudflare for DDOS protection and additional security of our and our customers’ sites.
We closely follow the work of security researchers like Troy Hunt to keep up with the ever-changing landscape.
We understand how vulnerabilities in OWASP top10 project work and thus know how to defend against them.
Our user accounts in all important systems are protected with strong passwords.
We’ve designed our login systems in such a way that we may not need your password at all (if you sign in via Facebook, Twitter, Google etc.)
If we need to store your password, we’ll use the current state-of-the-art bcrypt with an appropriate cost factor. Even Yahoo failed this by using a weak algorithm. We are not going to make the same mistakes.
If your use case requires stronger security, our systems support two-factor authentication (Google Authenticator)
We really, really care about security.
We use USB security keys (from Yubico) to protect our credentials, so even if an attacker got access to our laptops, they would not gain authorization to our production servers.

Please, we urge you to ask your other software vendors how they are working to protect your security and privacy. We as users deserve better.

Trust by transparency

It’s easy to sweep bad development practices, security issues, bad software design, uptime problems and user satisfaction under the carpet. Many companies have huge issues that their users might be unaware of.

If a company is actively trying to make these details public, it builds trust because users know that the things we make public will be under public scrutiny and thus hurts our image if done improperly. What we’ve already made transparent are:

The quality of our software by publishing many open source projects. Our software architecture and code quality is free for anybody to review.
In fact, we have open sourced our most critical security infrastructure like our loadbalancer configuration and certificate authority.
Our focus and thoughts on importance of security.
Our promise that if we ever get hacked or anything important gets compromised, we will disclose it (Buffer handled their issue beautifully).
Any issues with our services being unreachable.

In the future we’d like to be transparent about our finances. Buffer set the gold standard for this and we’d love to do something like this in the future.

Consulting

Thu, 22 Dec 2016 22:24:34 +0200

Need help with something?

Our experience

Stuff we feel comfortable working with:

Software integrations
Most AWS services. In-depth experience with EC2, SQS, S3, IAM, SNS, Lambda, DynamoDB, Route53.
Printer drivers / printer infrastructure / label printing / barcodes
Distributed systems (CoreOS, Docker Swarm, Consul)
Container-based infrastructure and build systems
Security-oriented thinking
Public key infrastructure + SSL
EventSourcing -based architecture + CQRS
JSON/REST
Computer vision
Reverse engineering (strace, Wireshark, Docker diff)
Network protocols such as HTTP, HTTP/2, DNS, TCP/IP, UDP
Software loadbalancers such as HAProxy and Nginx
Laboratory information systems (healthcare)

NX Print

Thu, 22 Dec 2016 22:24:34 +0200

Printing infrastructure for browser-based apps.

Use cases

Need to print from the browser automatically? Your use case simply does not allow for your end users to invoke the print dialog, choose a printer and possibly printer settings?
NX Print provides you low-level access to Windows’ printing infrastructure from JavaScript.
Print PDF-based reports/order confirmations/packing slips etc. from browser.

Technical details

NX Print is built on our NX technology (an open source framework for calling native apps from JavaScript in a secure manner).

Do you need low level or higher level?

If you don’t need to access the low level printing primitives from JavaScript, and/or you’d just rather print some labels, we have a product for that as well: LabelCloud. LabelCloud is basically NX Print + extra stuff for printing labels.

Documentation + integration usage example

Integration is easy, with just a few lines of JavaScript. Head over to documentation.

Here’s the rough API in pseudocode:

// list printers
nx.nxprint_1_0.list_printers()

// print PDF
nx.nxprint_1_0.print_pdf_http()

// raw print
nx.nxprint_1_0.print_file_http()

For additional details, contact us!

Licensing

While you can download NX Print freely for evaluation, you have to get a license for production use. Contact us for licensing!

Every website will get hacked - how to prepare for it

Fri, 23 Sep 2016 11:29:36 +0300

Every website will get hacked?

Yes. It is starting to look like every website - even the major ones, will get hacked. It is not a question of “if”, but “when”.

Just now in the news is that Yahoo was compromised and the result was a leak of names, usernames, passwords and other personal data of 500 million accounts - almost twice the United States population.

Here are examples of a few widely known breached websites (up to September 2016 - this list grows constantly):

Website	Breached accounts, millions
Yahoo	~500 M
MySpace	359.4 M
LinkedIn	164.6 M
Adobe	152.4 M
VKontakte (Russian Facebook)	93.3 M
Dropbox	68.6 M
Tumblr	65.4 M
Ashley Madison (dating site for cheaters)	30.8 M
Last.fm	37 M
Snapchat	4.6 M
Trillian	3.8 M
Patreon	2.3 M
Forbes	1 M
Comcast	0.6 M
Yahoo (old breach)	0.4 M
Avast	0.4 M

This list is just the publicly known breaches, and I believe there are plenty that go unreported and/or have already happened, but are yet to be known to the public. Many breaches have a lead time (= time it takes from the breach until it becomes public knowledge) in years.

More complete list (and sources for the claims) of breaches are found from haveibeenpwned.com, operated by security researcher (and my personal hero) Troy Hunt.

Have my details been leaked online?

HaveIBeenPwned is a free website where you can enter your email to see if your account details have been publicly leaked online.

I know, in this day and age entering your primary email into one more website doesn’t sound good - but the operator of the website is as trustworthy and as responsible as they get.

Different types of websites have different risks

Type	Example site	Sign-in via	Impact of breach
Blog	Tumblr		Low: people probably don’t have much private content in Tumblr blogs.
Social	Twitter		Low: Twitter messages are public anyway (though some people use private messages as well)
Social	Twitter	Yes	Severe: all your “Sign in via Twitter” -websites are compromised as a result.
Social	LinkedIn		Low: my profile and private messages there are not so sensitive.
Social	Facebook		Severe: your private messages in Facebook should probably be private.
Social	Facebook	Yes	Severe: all your “Sign in via Facebook” -websites are compromised as a result.
Dating	match.com		Severe: you probably have privacy-sensisive conversations there.
Dating	AshleyMadison		Catastrophic: you being a cheater and your sexual preferences become public knowledge.
Email	GMail	Yes	Catastrophic: “Reset password” links are sent to email. Your email gives full access to all your other services.
Health	E-health provider		Catastrophic: your health data, possibly sexual history, awkward illnesses will be made public.

BUT: if you are one of the 59 % who use the same password for different sites, then the impact of even a Tumblr breach could turn up to be catastrophic because the same password lets hackers into your GMail as well (and from there, to other websites).

Note: above are generalizations. People use websites differently, for example you could use Twitter’s private messages a lot more than I do, so you could be a lot more concerned about a breach than I would be. Or you could not have sensitive private messages in Facebook.

You don’t believe me about the catastrophic impact? Read more to learn for example how vulture-like people set up the AshleyMadison records (sexual preferences etc.) for public search and extorted money from the victims. Yes, you could argue the victims being cheaters, they deserved it - and you’d probably be right.

I believe it’s only a matter of time before a huge breach reveals people’s really intimate data like health records. Philippines voting data and US government personnel data have already been exposed. Health record data will be breached, and it will get ugly.

As a user, how do I prepare for this?

Super important: use different passwords for every service. Password managers help with this, like the free Keepass and others.
Use strong passwords. Or better yet, passphrases.
Use multi-factor authentication where possible. Google Authenticator is awesome and works for non-Google websites as well!
Know the risks (= the data might/will get public) when giving personal data to online websites.
If your software vendor doesn’t do things responsibly, ask them for improvements.

As a website operator, how do we prevent this and/or prepare for this?

Take your and your users’ security and privacy very seriously. Your users deserve better.
Support HTTPS. Force HTTPS. CloudFlare handles it all for you, or if you don’t use CloudFlare SSL certs nowadays are free anyway. No excuses!
Don’t use company-wide accounts for any sensitive systems. Have each employee have a unique account so access can be audited and privileges managed granularly. If this is painful, you are lacking automation.
Enforce strong and unpredictable passwords for all employees that have access to critical systems. I’ve witnessed first hand companies having systematic (= predictable) master passwords to critical production systems. Even former employees knew the new passwords because they knew the formula the passwords were generated with. A huge no-no!!
There are many important issues you need to be aware of when building a password reset feature (which you probably need if you are handling user accounts).
Support multi-factor authentication.
If your team isn’t aware of issues like SQL injection, cross-site scripting, CSRF, or are not familiar with issues documented in OWASP top ten, you are not being responsible and should not be developing software for other people’s use.
Keep your systems up-to-date and patched. Huge issues like Heartbleed or Shellshock pop up and it’s your job to follow the changing landscape and react to issues FAST. Even keeping up doesn’t quarantee immunity: see zero-day vulnerabilities.
Implement rate limiting for logins to prevent brute-forcing passwords online. (PBKDF2 et al. sorta fixes this too..)
Prepare for the inevitable and protect your users’ passwords with current best practices. Hash + salt is nearly useless nowadays. Look into adaptive hashing formulas like PBKDF2.

And most importantly: foster a culture of security in your team. Many companies don’t really care, and I hope those companies go out of business once they lose their reputation - which you can only lose once.

One example of downplaying important issues: SnapChat was warned about a security issue and they called the attack vector “theoretical”. Theory turned into practice and 4.6 million usernames and phone numbers were exposed. SnapChat deserved the hack but the users didn’t. Be responsible!

function61.com

LabelCloud

Use case

Demo

Supported printer manufacturers

Architecture: how does it work?

Licensing

Security

Security policy

Vulnerability reporting

Security incident history

Introduction to WAL (write-ahead logging)

What is atomicity & durability? Why do I want it?

Aren’t file writes atomic & durable by default?

But I’ve heard about fsync, it fixes this right?

Ok, so how does WAL achieve atomicity & durability?

Okay that’s cool, but what if write to the WAL fails?

Recapping

Reverse-engineering with strace

Install strace

System calls? What and why?

Quick introduction to strace

Using strace with real-life programs

Closing words

Further reading

Running a company-internal certificate authority

“Company-internal” CA?

Security implications?

This sounds hard - why do it?

How do I use the certificates issued by a CA?

Choosing a toolkit for CA management

How do I set up a CA?

How do SSL certificates work?

What is PKI?

What actually is a certificate

So how do SSL certificates work with HTTPS connections

Tell me more about root certificates

Can we trust the CAs? Isn’t PKI broken by design?

How much do SSL certificates cost?

Summing up

Introducing Buildbot (open source)

Website launch

About our values

Building software is a craft

User experience

Security

Trust by transparency

Consulting

Need help with something?

Our experience

NX Print

Use cases

Technical details

Do you need low level or higher level?

Documentation + integration usage example

Licensing

Every website will get hacked - how to prepare for it

Every website will get hacked?

Have my details been leaked online?

Different types of websites have different risks

As a user, how do I prepare for this?

As a website operator, how do we prevent this and/or prepare for this?