It saddens me to say that the way the Covid-19 vaccinations have been handled in Romania is, for lack of a better word, complete shitstorm .

The main problem came not from the technology behind the platform, though we'll get to talk about that a bit as well. As is usually the case with Romanian authorities, the communication side of things is where things break down.

For a ;tldr you can jump straight to the timeline or, for the geeks data scientists out there, to the code behind part of the analysis.

I'd suggest though at least skimming through the full article.

A short intro

You see, in Romania, like in most other countries, the population was split into three groups in order to more efficiently distribute the vaccine:

1. Category 1: persons working in healthcare and social services;
2. Category 2: persons with high risk (over 65 years old, people diagnosed with chronic conditions, etc.) and essential workers;
3. Category 3: the general population.

To better handle this, towards the end of December 2020 an online platform is launched which initially allows individuals from category 1 to make vaccination appointments at any of the vaccination centers with open spots. Following that, around the middle of February, people from category 2 can do the same.

An issue slowly creeps

Playing a bit with the official data from the government Covid-19 vaccination transparency report, something interesting shows up.

Despite not being officially possible to get a vaccine if you're not in these two categories, looking over the data we can spot something:

Based on the stats from the 14th of March we can notice that already 1950 people from Category 3 had at least one dose of vaccine administered to them.

As is usual in Romania, the official communications said that only essential and at-risk people are able to get vaccinated, and yet the data says otherwise.

I agree that maybe I'm being nitpicky about the issue considering that when looking at the total vaccination numbers Category 3 is insignificant compared to the other two.

The problem is though that the data generally ignores the social impact: after a long period of lockdown where most of the population respected the rules it can be frustrating to see that others cut ahead in line.

And this exact issue of a queue, so dreaded in the Romanian culture, is what will come to bite us again in a bit.

The announcement for the masses

Around the 13th of March the officials announce that starting from the 15th, the general population will be able to sign up for a vaccine, or at least, get on the waiting list for one.

Good communication, right? We have a date, people can prepare, your tech team can finish doing maintenance and testing considering that a large part of the population will be using the website.

Except that on the 14th of March at around 6:30PM the website was wide open for registrations.

As expected, a large number of people flooding the website brings it down, but not before being available for almost an hour during which the waiting lists for the few vaccine centers in Bucharest filled up with hundreds of people.

To address the situation, the officials in charge of the platform announced that they're going to do a staged rollout the next day based, if I remember correctly, on county infection metrics. Based on that, the waiting lists for Bucharest would be open at 3PM on May 15th.

Unexpectedly, at around 4am on the 15th, people could yet again schedule themselves not only on the waiting list, but also to get actual vaccines in some centers.

The rest of the day goes on without surprises, with the platform being either online or offline depending on the actual load.

Communication issues

My main frustration comes from the issues the government had around clearly communicating the scope and timelines of the Covid-19 vaccination program. By not respecting its set schedules it eroded the trust people had in them and their ability to manage the situation.

This was yet again clearly seen when it came to adding (and removing) vaccination centers on the platform: it seemed to happen haphazardly and randomly, without any clear details.

This lead to situations such as people being the 7000th person on the waiting list for a center while new centers were being added with open waiting lists. What's even worse, existing centers were being removed and people reassigned to others without knowing what position you're holding in the queue.

To address this communication problems different websites popped up.

Date la zi

Datelazi.ro prezintă infografii actualizate periodic cu datele furnizate de autoritățile competente.

COVID-19: Date La Zi

Date la zi offers a great overview dashboard of the Covid-19 situation in Romania, and what's more important, it's an open source project.

Vaccin.live

Informatii pentru locuri libere in centrele de vaccinare

Vaccin.live

Vaccin.live is the most popular of them, offering a per-county list of available centers, their waiting lists and the vaccine used (something which the official website doesn't display, but makes indirectly available in the response payload). Initially it also had the advantage of being updated more often than the map present on the platform itself.

vaccinica

Statistici vaccinare Romania

vaccinica is a newer appearance, but a very useful one, providing a data dashboard around the vaccination effort based on the official Covid-19 transparency report.

The timeline

Bellow is a timeline of the events, but please note that it's mostly from my recollection, so there might be discrepancies around the time and dates of early events (platform launch, etc.)

The code

You too can also play with the data if you're interested using the following Jupyter Notebook.

GitHub: fuzzmz/rocovidstats/main

Click to run this interactive environment. From the Binder Project: Reproducible, sharable, interactive computing environments.

The source code for it is also available for those so inclined.

fuzzmz/rocovidstats

Exploratory JupyterLab notebook for the Covid-19 transparency data provided by the Romanian government. - fuzzmz/rocovidstats

GitHubfuzzmz

Looking over some scientific code written in Python with a friend doing his Ph.D. I couldn't not notice how bad its quality was. This in turn lead to a discussion around coding culture and a biased scientific stance on coding.

The coding culture

One of the possible reasons for horrible code is that, for most of it, academic code tends to be throwaway code. In academia you write chunks of code (sometimes fairly big chunks of code) to test your hypothesis, or to prove that some algorithm works, but rarely if ever will anyone, most likely including the one who wrote the code, will ever read or modify it; this in turn means that it doesn't make any sense to optimize for maintainability, weed out bugs or use proper engineering practices such as doing code review, writing documentation, tests, CI/CD flows, etc.

This coding culture goes hand in hand with...

Scientific upbringing

Maths (and by extension physics and other hard sciences) is terse, so it's more natural for someone heavily steeped in math to begin writing code that resembles it (also a reason I think Python was adopted is because it looks more like math than other languages). That means terribly unreadable code for someone not intimately acquainted to the problem space; so you'll see lots of single character variables in functions that look like submissions to a code golf competition.

To give even more context to terse variable names, sometimes they are certainly cultural (e.g. fhat = ... for Fourier coefficients), but more often than not scientists are implementing variable names that read straight out of a paper; this makes sense, as it makes it easier to follow the code in parallel to the paper. There's little chance to read some paper talking about a quantity x_ij and name it something else, there's usually just no point. Even if it did read smoother in the code alone, you can lose some readability that goes along with the paper.

Of course when the quantity is something physical and simple it's not a big deal, like temperature instead of T, but often these variables represent some really random thing that would be silly to name. I mean if we're talking about a second order mixed partial there's no getting around it, it's getting named f_xy, code reviewers be damned.

This means that once you know the underlying math, a lot of scientific programming is pretty straight-forward. There usually aren't a lot of inter-connected moving parts, so they never really have the need to discover design patterns, standards, and best practices common in the software world, which takes us back to the coding culture.

A bit of history

Most people might think of EXE files when asked "what is a program on Windows?", but before thta the only programs that existed on Windows land were COM files. The COM format, inherited from CP/M wasn't actually a format, it was just a memory image which got loaded into memory unchanged and then ran from the first byte.

The downside of COM files, and what eventually lead to the creation of EXE was that the programs couldn't be bigger than 64KB (I guess this had something to do with register segments being around 64KB on the 8086), and slowly COM was forgotten.

Of memory and processor instructions

Getting to how to restart a PC using a COM file, I ran accross it by accident, by renaming a .bat file which had something like cd c:\games in it to COM, which reseted the PC.

Now, knowing tha COM files are actually memory images, the restart actually makes sense. cd disassembles into arpl [si+0x20],sp which despite looking like a priviledged instruction, it actually isn't; more than this, it isn't even a valid instruction in real or virtual mode, which causes the illegal instruction exception and restarts the PC.

Another fun fact is that the code for the warmest reboot on PC is 0xcd 0x19 (int 0x19) which jumps back into the BIOS code when it starts looking for the operating system to boot. You could then have files containing INT 19; (CD 19) which was the fastest way to soft reboot when the file was run from a MSDOS mode shortcut.

But wait, there's more! This has the side-effect of confusing the NT NTLDR, which spits out a "your computer does not have enough free conventional memory to run Windows NT" error message.

Symantec has been caught yet again failing to make its due diligence when issuing certificates: this time it looks like they have failed to properly validate at least 30.000 certificates.

In response, the Chrome team announced that it will stop recognizing the extended validation status of all certificates issued by Symantec-owned certificate authorities, and to nullify all currently valid certificates in a staggered approach by decreasing the maximum age of Symantec-issued certs over a series of releases, ending with a validity period of only 9 months by Chrome 64' release.

The web of trust

The problem with certificates authorities is that you need to trust them in order for the whole system to work. That is, you trust that Symantec, or Thawte, Verisign, Equifax and others (all bought by Symantec in the past years) actually checked that whoever requested that certificate is the actual owner of that business.

The issue here is not the validity of the certificates but the trust under which they were issued. EV certificates promise Extended Validation of the legal entity requesting the certificate: check with BBB and double check with Dun & Bradstreet, for instance. If you can skip the 'extended validation'-part by paying more, EV becomes meaningless because a fraudster with deep pockets can get an EV-certificate for his bankofamurica.com-website.

For an example of the process someone goes through to get an EV certificate you can read the great post written by Troy Hunt, Journey to an extended validation certificate.

TooBigToFail™

The problem is these entities getting TooBigToFail™, which then makes them almost invulnerable to meaningful actions.

By consolidating so many certificate authorities under one umbrella it makes it almost impossible to revoke their certificates without breaking a large part of the Internet. To put this into perspective, Symantec certificates represent more than 30 percent of the Internet's valid certificates by volume in 2015 and around 42% of all certificate validations.

In a perfect world companies who have their root certificates entrusted as part of the TLS core infrastructure need to have better checks and balances than to simply say "oops, we done goofed" after the fact. If they demonstrate - as Symantec has demonstrated - that they can't manage that, their root certificates need to be yanked out of the chain of trust as soon as possible.

It will be interesting to see how the customers who have paid for their certs in good faith react to this move by Google, and hopefully the backlash will make Symantec take better care in the future.

What next?

The question now is what happens next? What do we do to ensure that we're in the clear?

Well, a solution would be to move away from Symantec and the rest and towards Let's Encrypt, especially if you don't need EV certificates.

But Let's Encrypt really does almost NO validation what-so-ever, I hear you say. As long as you are in control of the web server at said domain, you can get a certificate. How many web servers are compromised on a daily basis at any one domain?

This is only sometimes helpful in practice, but it's important to remember the distinction between "what you promise" and "how much of what you promise you deliver".

Let's Encrypt doesn't promise all that much; their system is pretty much just designed to ensure that the entity that requested the certificate has (or very recently had, their certificates do last a modest period of time after issue) operational control of the host for which the certificate is issued.

That isn't a terribly grand promise: it doesn't imply anything about the real-world owner of the site; tell you whether or not somebody compromised the host, etc. but it has the virtue of both being relatively easy and cheap to automatically verify and of being something where the most common failure mode (compromised host) isn't a threat that SSL is supposed to protect against, no matter how exhaustively vetted (it is supposed to protect the channel between you and the server from 3rd parties, not assure you that the guy running the server is trustworthy); so failures there aren't terribly serious.

To the best of my knowledge, while LE doesn't promise much, they have so far delivered it; and have avoided failures that threaten other people's sites. If my little VPS gets hacked, or my shoddy admin dashboard has an exploit, somebody else getting an LE cert for my site is quite plausible; but that's because somebody else does, indeed, have operational control of my site. Getting an LE cert without demonstrating operational control is what would be really worrisome.

And, that is where Symantec has been trouble: they've been caught issuing enormously powerful certs for high profile domains without the request of the domain holder, which are potent weapons for MiTM attacks. That is bad.

Per-display graphics card?

This whole post started after we received new laptops at work equipped with both an Intel HD Graphics 530 integrated GPU (iGPU) and an AMD FirePro W5170M (discrete GPU - dGPU). One of my colleagues was wondering if, when running multiple monitors, it is possible to get one or the other of the displays to be driven exclusively by the dGPU.

Sharing is caring

The short answer to the above question is no. It's impossible because only the iGPU is connected to the display; the discrete GPU only renders to video memory, and the integrated GPU displays the image on the screen.

Remember that a graphics chip has two jobs: fill the frame buffer with data and output that data to the monitor. The first and the second job actually happen at the same time, but in different locations in memory; the GPU is actually reading and writing to VRAM at the same time. The part of the memory being currently displayed is called the frame buffer, and the part being filled up is called the back buffer.

On a switchable graphics system the iGPU does both jobs most of the time: it takes commands like "draw a red line from here to there" and sets the value of the pixels along that line to the color red while another part of the iGPU reads the data out to the video port. This second part is actually fairly slow and takes the same amount of time, no matter what data is in video memory. When you operate in high power mode, the dGPU fills the back buffer and the iGPU reads it.

So in a push-pull configuration like this, the dedicated video chip is not ever actually connected to the video port. Instead, it simply feeds data into the frame buffers that the Intel Graphics Processor uses to display the images. Because of that, there's really no way for you to connect one GPU to one monitor - there's literally no connection between the GPU and the monitor at all. All video goes through the iGPU.

Version control systems are really important tools in the day to day life of a software developer. What happens though when you have to move both code and people off of one system to another? Let's just say that you'll need time, patience and the desire to write your own tools...

As you know, I'm a big fan of git, so when work decided to move our source control from AccuRev to git I was one of the first ones to jump up in excitement. Don't get me wrong, AccuRev isn't a bad tool per se, but it does have enough downsides which make it a pain to work with, especially if you're used to something more lean such as git.

AccuRev downsides

My main issues with AccuRev are mainly the following:

1. it's slow as hell, especially the GUI
2. command line options aren't as good compared to git
3. user hooks are a joke compared to git, which in turn leads to
4. harder to ensure coding style and local tests by running scripts before committing/pushing code
5. harder to integrate correctly with continuous integration systems
6. getting history is a pain in the ass: for example it's next to impossible from the output of accurev hist to figure out if a file was added or deleted
7. not at all conductive to agile development: branches (streams) can't be re-used if deleted, workspaces need to be manually moved to track another stream, which usually leads to "one workspace per stream syndrome"
8. not portable: workspaces are hardcoded to the system (they have the PC name in their metadata); this means that if you'll switch PC's you'll need to either manually trawl through your list of workspaces and update them one by one, or try and script it (which doesn't work if you're switching from Windows to Linux and vice-versa).
9. still on the workspace part, if you work on multiple PCs you'll end up having more than one workspace on a given stream, because workspaces can't be reused (seems like nothing in AccuRev can be reused).

Migrating developers

All of the issues from above cascade and make the developer behave in a certain way in order to accommodate the tool.

The best example to give in order to show the change of mentality needed to move from AccuRev to git is caused by issues number 7 in my list. Because it's hard to make a workspace track another stream (equivalent to a git branch) users started creating separate folders for each stream; this in turn allowed them to simply diff the folders in order to check the differences between the development and stable streams for example. When moving to git, the first question I had was what happens when they change the branch they're on and how to see the differences between branches.

Another bad behavior was caused by how slow AccuRev can be when it comes to branching: in order to reduce the time spent waiting around, most users pushed changes directly to the development (or worse, master) branch instead of having feature branches; this then made it hard to push just specific changes to the master branch when you wouldn't want to integrate all changes, or one of them needed to wait for another fix.

Most of these workflow differences got solved via trainings as well as due to git's popularity, which means that most questions are just a Google search away.

Migrating code

The next step in making the move was actually getting the code into git. Some teams decided that history wasn't important for them, so they'll just dump all of their existing code on the stable branch and then go from there. For us though that was unacceptable, so I decided to write something that would migrate our AccuRev history to git.

Before going into the code let's talk a bit about some AccuRev terms, and compare them to git.

AccuRev has the concept of depots, which should map to git repositories, but in our workplace we usually assigned them per team, so they ended up holding completely different components and systems. This way, depots are better mapped to projects in our specific case.

Next are the streams, which are the branches from git. Streams can optionally have a parent, which is the equivalent to git branching, or they can start from scratch, which is how we translated repositories.

Basically we would have the following AccuRev structure:

TEAM_DEPOT -> Wizard -> Wizard_stable -> Wizard_develop

TEAM_DEPOT -> Toolchain -> Toolchain_stable -> Toolchain_develop

In the TEAM_DEPOT we'd have two components, Wizard and Toolchain, which start from the parent (empty) stream of Wizard and Toolchain, which in turn has the child of Wizard/Toolchain_stable on which actual code resides.

If we are to map this to git, we'd have the following:

Wizard repository -> master branch -> develop branch

Toolchain repo -> master branch -> develop branch

Instead of having a single repository which contains both projects each of them is split into it's own repo, with it's own history, permissions and so on.

In order to migrate our code we had to do the following steps:

1. select an AccuRev stream to migrate (this was usually the stable stream for each particular project)
2. get the full history for that stream
3. for each historical event get the author, message and timestamp
4. for each historical event get the actual files and commit them to a git repo with the info obtained at the previous step
5. keep doing 3 and 4 until you're up to date

Optionally, because for a period of time both AccuRev and git would be available, with development being done in AccuRev, allow updating a git repository with the new history from AccuRev:

1. get the stream history
2. check the latest commit from git and map it to AccuRev history
3. start migrating from the newest transaction in AccuRev not available in git

The first issue we ran across was due to workspaces and how they're attached to streams: we needed to check if the location used was already associated with an existing workspace, as that would prevent us from creating the new workspace.

The second one was having to move the migration workspace once created in case the user had to perform multiple migrations on different streams in the depot.

Then we found out that AccuRev doesn't really sanitize the messages in any way, which can in turn lead to failures when trying to commit the changes into git.

Another strange case is that, while AccuRev insists that all streams start with the depot name, that match isn't case sensitive, so you could have a depot named Project and the stream could start with project, which in turn caused my script to fail.

But the worst thing is that you can't tell if a file is deleted or added from the AccuRev history file. This in turn lead to my first implementation being slow for large repositories, because I would have to get all the files for each history step (not just the changed ones), copy the files to the local git repository, commit them, then deleting all the files in the git repo before copying the next round of files from AccuRev in order to detect deleted files.

To fix this we decided to move to a stream and workspace implementation, where we would have a pass-through stream tied to the one we would migrate and a workspace tied to this stream that points locally to git repo folder. Pass-through streams are interesting, as they allow you to change the history element they're pointing at without modifying the original stream; what this means is that, by having an AccuRev workspace tied to it, we wouldn't have to get all the files each time, but just the changed ones by simply modifying the transaction at which the pass-through stream was pointing and then updating our workspace.

Get it while it's hot!

In case anyone has to go through this themselves I've made the code available on Github and GitLab, and each and every contribution is appreciated.

Unfortunately the tests can't be made available because you'd need a reference depot which isn't portable. You can see the Making private tests public article for more information about this.

github.com/fuzzmz/accurev-to-git

gitlab.com/fuzzmz/accurev-to-git

To give some more context, the tests are specific to a tool I wrote for migrating history from AccuRev to git. The problem here is that AccuRev is a tool which isn't widely available, and to make things even more fun, we're talking about migrating history, so we need to have a known good state in the tests in order to check the migration was done successfully.

Bellow is a simple test which will fail if the user doesn't have AccuRev installed on his machine:

def test_accurev_login_error_handling():
    assert not accurev_login('badtest', 'badtest')

That's a simple one, but we can get into even more complicated ones, like comparing the history obtained by the migration script to a known good configuration:

def test_get_history(set_fixture):
    historyfile = set_fixture[2]
    modhist = set_fixture[3]
    acchist = get_history('Test_gitmigrate_testhist')

    with open(acchist, 'r') as fin:
        content = fin.read().splitlines(True)
    with open(modhist, 'w') as fout:
        fout.writelines(content[4:])
    assert filecmp.cmp(modhist, historyfile)

The problem here is that even if the developer has AccuRev installed on his system, there's no way for me to provide him with a valid stream history which will allow the tests to pass because, well, it's history, which contains information like the user who commited the code, when it was commited, what the message was and so on.

I'm wondering what's the proper way to make the tests for such a project available.

• Should I make them public and just note that they'll fail?
• Write a guide on how to modify the tests in order for them to work? But then, doesn't this pose the risk of sabotaging your own tests, so that they'll always pass no mater what?
• Simply mark all AccuRev specific tests are failable and tell possible contributors to enable them if they have AccuRev? But then I'll have to be careful during pull requests to not merge any of the changes done to make the tests work for their specific environment.
• Not publish any tests at all? Or just leave in the very generic python ones?

I know that this is mitigated to some level by the fact that if someone is interested in this migration script they're most likely going to have all the software installed, but it's still a bit of a headache. Not to mention not being able to have automated testing via Travis-CI once the code is made public, which means even more fun with running manual tests in case of a pull request.

So, how would you handle this? Hit me up on Twitter (just reply to the tweets embedded above) or leave me your thoughts on the bellow Google+ post.

This feature is a nice starting point for Google Photos going social, but it needs a bit more feature for it to be truly awesome (see what you did, I'm almost agreeing with The Verge here!).

After playing with the new feature for a bit I've found the following:

• Controls appear to be all-or-nothing. I can control whether an album allows people to add or not, but no way to give people different links with different controls to the same album
• There's no way to remove a person from a shared album
• A contributor can add/remove their own photos, but cannot remove other people's
• The owner can remove other people's photos that were added to the album
• If a contributor joins an album, adds photos, then leaves the album, all of their photos are removed with them
• The owner (and presumably anyone else) has the option to add any of the photos from a shared album to their library
• Each picture gets labeled with the person who added its name
• There's no notification that someone left an album
• If you turn "collaborate" off on a previously shared album, it removes all other people that have joined & their photos

I was hoping that turning collaborate off might "freeze" the album in its current state, and still allow people to get notifications if I added photos, but it looks like this is not the case.

One other feature that I'd personally really like to see would be the ability to tap someone's picture (which is present in the list of who's in the album at the top) and see all the photos they added.

Upstream fast and large, Android slow

One of the most cited reasons for this is that Android kernels still carry a lot of custom patches which are not available in the upstream kernel.

Say that an upstream developer introduces an API change; the first step he should do after this is to check that all existing code compiles and works without regressions. However, he's not obligated to take custom kernel modules (like grsecurity, rt or Android) into account, nor is he responsible for third party drivers.

As such, all of the workload of bringing in upstream changes to the Android kernel lays solely on Google developers, and considering the fast pace of kernel development this task is time consuming.

Except that's not it

The problem is that all of the above simply isn't that accurate. For example, for an Android device running Linux 3.18, there are only 700 patches to Android-ify your kernel (adding modules like cpufreq_interactive, binder, SElinux fixes, etc.). You can see the list of kernel branches supported here to get an idea on how much, or little, they differ from upstream.

One of the reasons might be that SoC vendor brings up the kernel on a new chip (so Qualcomm for example ensures that everything is working on the SD808/810). The vendor invests and a lot of time and money in making sure everything is stable. The issue here is that the kernel changes a lot of hands: upstream (Linus) provides the base kernel, Google provides the core Android patches on top of a specific version (3.10, 3.14, 3.18, 4.1), the SoC vendor brings up their chip with their own set of patches, and then the actual device OEM will apply the final changes on top to support a specific device.

This means that SoC vendors don't really have an incentive to bring new kernels to an existing chip. Also note that the development time from when a SoC starts getting worked on and when it's released is pretty long, so your kernel might have actually been not that old when it was selected. It's much easier from a SoC vendor perspective to deal with a new kernel when you're working on bringup for a new SoC. Likewise, even if the SoC vendor upgrades a kernel, the OEM would have to pick that up, rebase their device-specific patches on top o it, then go through all the testing and bug fixing that entails.

Realistically that won't happen though, as the SoC vendor is already doing bringup on the next generation of chips, and the OEM is working on their next devices.

Does it matter though?

Well, it doesn't really matter if you're running 3.10 when upstream is at 4.4. If you're on a desktop and there are bugfixes or new drivers that improve some piece of hardware in your system, great, a new kernel might improve your life significantly. For your phone though, the kernel has been developed specifically for your device because the hardware isn't interchangeable.

If there are better drivers or bugfixes from 4.4, they've probably been backported to your device on 3.10 - and yes, those will come in with your standard OTAs. You don't need a full kernel version bump to take advantage of those.

In fact you don't want a full kernel version bump because all of your device and SoC specific code can break in mysterious ways. Watch as some mmc refactoring upstream randomly breaks your wifi driver! Missed one armv8 patch? Boom, random SIMD register corruption that causes programs to randomly segfault. People are especially sensitive to kernel stability on their phones, so the kernels need to be rock solid. Backporting patches to a kernel the SoC vendor has spent a lot of time on making stable is always the safer route.

But what about PCs?

The only reason that this is agnostic to Android devices, and not say, desktop PC's, is because of how drivers/modules are implemented on Android.

Because there is no standardized bootloader on Android, the kernel cannot go about detecting hardware on the platform, and drivers need to be baked into the kernel itself (and thus, loses modularity). This is why it's such an intensive process to upgrade the kernel, as drivers are statically written in, rather than dynamically being linked.

In short, this would be much less of a problem if there were a standardized bootloader, and easy hardware detection in Android Kernel. Once that happens, dynamic linking of modules could be implemented, and much less customization would need to be done to the kernel itself.

In theory, implementing dynamic linking would also be the gateway for Google to provide universal updates to Android devices. It is because of these deep, inflexible changes that Google cannot even begin to push universal updates.

Read more

If you've reached this point and still want more, then you can read Running a mainline kernel on a cellphone by Jonathan Corbet on the struggles and fun you'd have to face in having the latest and greatest.

The slides

The video

The abstract

The goal of this talk is to share our experience in making things happen between clicking on Help - Install New Software and Finish and how the features we developed evolved from one another.

The talk will present an overview of the following custom Equinox p2 actions, as well as how they integrate into shipping both an Eclipse based IDE and enabling future updates.

The following custom actions are treated:

1. Freescale Install - enabling placing the binary contents of extensions outside of the Eclipse home folder and ensuring that even files that are write-locked get updated
2. Freescale Copy - a faster Freescale Install by disabling the write-lock check and acknowledging the caveats that come with this
3. Freescale Remove - at uninstall make sure that all binary files added using Freescale Install or Copy are correctly removed
4. Freescale Execute - allowing execution of files during the install phase, on both Linux and Windows
5. Freescale Merge - XML files are fun and useful, and at times they need to be updated.
6. Freescale Process Check - conditional installation denial based on running processes
7. Freescale Common Deploy - installing features and plugins outside of the Eclipse location

Geeking out after switching back the blog to Pelican and hosting it on GitHub Pages, I decided to set up automated publishing of the HTML content from the markdown articles without manually running Pelican on my own PC. So what better way to do this than plug everything into Travis CI, because continuous integration doesn't automatically imply code.

Configuring the job

I'm not going to go through the whole set-up of having Pelican up and running, nor about how to configure GitHub pages in order to publish your blog, though those are some nice ideas for future articles :fa-lightbulb-o:.

The publishing flow would be as follows:

1. Create a separate article/article-name branch on which to work on the article
2. Submit a pull request to the source branch once the article is finished
3. On pull request merge run a Travis CI job to get the latest changes, publish using Pelican and push the output back to the master branch in order to make the changes public.

I am fully aware that this sounds so much more complicated compared to publishing an article using Wordpress for example, but it fits well into my mostly command line world, love and general geekiness.

The actual implementation can be split in two:

1. Getting Travis CI to notice that a merge was made
2. Telling Travis CI what to do once 1 happens

1. Travis integration

Actually setting up Travis CI is pretty easy and their docs are amazing, so in short you flip a setting in your GitHub repository options and add a .travis.yml file which basically configures how a Travis CI job works.

My .travis.yml file is as follows:

language: python
python:
  - '2.7'
branches:
  only:
  - source
before_install:
  - "export TRAVIS_COMMIT_MSG=\"$(git log --format=%B -n 1 | head -c23)\""
install:
  - pip install -r requirements.txt
  - pip install git+https://github.com/fuzzmz/[email protected]
script:
  - git config --global user.name "Serban Constantin"
  - git config --global user.email [email protected]
  - make publish
  - make github
env:
  global:
  - secure: ASDASDADS

This file tells us that the scripts will run on Python 2.7, and to only run when changes happen to the source branch.

The next step is to tell Travis that before installing our required packages he should export the git commit message which prompted the build to the TRAVIS_COMMIT_MSG variable, which will then be used when pushing the HTML files to the master branch.

After that we go ahead and install all the packages required to publish the blog (Pelican, Markdown, etc), as well as a custom fork of one of the packages and configure git so that the correct author is displayed in the repository master branch after pushing the published content to it.

We then go ahead and actually publish the content via make publish and push it to GitHub using make github, functions which will get explained a bit further down.

One more thing that needs done is to have a way to let Travis push the changes back to the repo. This is done by generating a token on GitHub and encoding it using the travis ruby application so that others can't use said token to push to my repository.

The last thing that needs to be done is to disable pull request builds in Travis CI to prevent the blog being updated by a pull request.

2. Travis commands

The actual steps to publish the blog and push it back to GitHub are detailed in the Makefile and called via make publish and make github.

PELICAN=pelican

BASEDIR=$(CURDIR)
INPUTDIR=$(BASEDIR)/content
OUTPUTDIR=$(BASEDIR)/output
CONFFILE=$(BASEDIR)/pelicanconf.py

clean:
    [ ! -d $(OUTPUTDIR) ] || rm -rf $(OUTPUTDIR)

publish:
    $(PELICAN) $(INPUTDIR) --debug --output $(OUTPUTDIR) --settings $(CONFFILE)

github:
    ghp-import -n -b master -m "${TRAVIS_COMMIT_MSG}" $(OUTPUTDIR)
    @git push -fq https://${GH_TOKEN}@github.com/$(TRAVIS_REPO_SLUG).git master

In the Makefile we define the current directory, the directory which holds the articles in markdown, the output directory which will store the generated HTML files as well as the path to the Pelican configuration file.

To publish the blog we simply run Pelican.

To push the changes to the repo so they become live we use ghp-import to simplify the process of committing the files and setting the correct branch (master in this case) and commit message, and then do a git push.

And that's it, now you just have to wait for the Travis CI build to take place, after which your content will be automatically made available.