cketti

Goodbye K-9 Mail

Wed, 26 Feb 2025 00:00:00 +0100

TL;DR: I quit my job working on Thunderbird for Android and K-9 Mail at MZLA.

My personal journey with K-9 Mail started in late 2009, shortly after getting my first Android device¹. The pre-installed Email app didn’t work very well with my email provider. When looking for alternatives, I discovered K-9 Mail. It had many of the same issues². But it was an active open source project that accepted contributions. I started fixing the problems I was experiencing and contributed these changes to K-9 Mail. It was a very pleasant experience and so I started fixing bugs reported by other users.

In February 2010, Jesse Vincent, the founder of the K-9 Mail project, offered me commit access to the Subversion³ repository. According to my email archive, I replied with the following text:

Thank you! I really enjoyed writing patches for K-9 and gladly accept your offer. But I probably won’t be able to devote as much time to the project as I do right now for a very long time. I hope that’s not a big problem.

My prediction turned out to be not quite accurate. I was able to spend a lot of time working on K-9 Mail and quickly became one of the most active contributors.

In 2012, Jesse hired me to work on Kaiten Mail, a commercial closed-source fork of K-9 Mail. The only real differences between the apps were moderate changes to the user interface. So most of the features and bug fixes we created for Kaiten Mail also went into K-9 Mail. This was important to me and one of the reasons I took the job.

In early 2014, Jesse made me the K-9 Mail project leader⁴. With Kaiten Mail, end-user support was eating up a lot of time and eventually motivation to work on the app. So we stopped working on it around the same time and the app slowly faded away.

To pay the bills, I started working as a freelancing Android developer⁵. Maybe not surprisingly, more often than not I was contracted to work on email clients. Whenever I was working on a closed source fork of K-9 Mail⁶, I had a discounted hourly rate that would apply when working on things that were contributed to K-9 Mail. This was mostly bug fixes, but also the odd feature every now and then.

After a contract ended in 2019, I decided to apply for a grant from the Prototype Fund to work on adding JMAP support to K-9 Mail⁷. This allowed me to basically work full-time on the project. When the funding period ended in 2020, the COVID-19 pandemic was in full swing. At that time I didn’t feel like looking for a new contract. I filled my days working on K-9 Mail to mute the feeling of despair about the world. I summarized my 2020 in the blog post My first year as a full-time open source developer.

Eventually I had to figure out how to finance this full-time open source developer lifestyle. I ended up asking K-9 Mail users to donate so I could be paid to dedicate 80% of my time to work on the app. This worked out quite nicely and I wrote about it here: 2021 in Review.

I first learned about plans to create a Thunderbird version for Android in late 2019. I was approached because one of the options considered was basing Thunderbird for Android on K-9 Mail. At the time, I wasn’t really interested in working on Thunderbird for Android. But I was more than happy to help turn the K-9 Mail code base into something that Thunderbird could use as a base for their own app. However, it seemed the times where we had availability to work on such a project never aligned. And so nothing concrete happened. But we stayed in contact.

In December 2021, it seemed to have become a priority to find a solution for the many Thunderbird users asking for an Android app. By that time, I had realized that funding an open source project via donations requires an ongoing fundraising effort. Thunderbird was already doing this for quite some time and getting pretty good at it. I, on the other hand, was not looking forward to the idea of getting better at fundraising.
So, when I was asked again whether I was interested in K-9 Mail and myself joining the Thunderbird project, I said yes. It took another six months for us to figure out the details and announce the news to the public.

Once under the Thunderbird umbrella, we worked⁸ on adding features to K-9 Mail that we wanted an initial version of Thunderbird for Android to have. The mobile team slowly grew to include another Android developer, then a manager. While organizationally the design team was its own group, there was always at least one designer available to work with the mobile team on the Android app. And then there were a bunch of other teams to do the things for which you don’t need Android engineers: support, communication, donations, etc.

In October 2024, we finally released the first version of Thunderbird for Android. The months leading up to the release were quite stressful for me. All of us were working on many things at the same time to not let the targeted release date slip too much. We never worked overtime, though. And we got additional paid time off after the release ❤️

After a long vacation, we started 2025 with a more comfortable pace. However, the usual joy I felt when working on the app, didn’t return. I finally realized this at the beginning of February, while being sick in bed and having nothing better to do than contemplating life.
I don’t think I was close to a burnout – work wasn’t that much fun anymore, but it was far from being unbearable. I’ve been there before. And in the past it never was a problem to step away from K-9 Mail for a few months. However, it’s different when it’s your job. But since I am in the very fortunate position of being able to afford taking a couple of months off, I decided to do just that. So the question was whether to take a sabbatical or to quit.
Realistically, permanently walking away from K-9 Mail never was an option in the past. There was no one else to take over as a maintainer. It would have most likely meant the end of the project. K-9 Mail was always too important to me to let that happen.
But this is no longer an issue. There’s now a whole team behind the project and me stepping away no longer is an existential threat to the app.

I want to explore what it feels like to do something else without going back to the project being a foregone conclusion. That is why I quit my job at MZLA.

It was a great job and I had awesome coworkers. I can totally recommend working with these people and will miss doing so 😢

I have no idea what I’ll end up doing next. A coworker asked me whether I’ll stick to writing software or do something else entirely. I was quite surprised by this question. Both because in hindsight it felt like an obvious question to ask and because I’ve never even considered doing something else. I guess that means I’m very much still a software person and will be for the foreseeable future.

During my vacation I very much enjoyed being a beginner and learning about technology I haven’t worked with as a developer before (NFC smartcards, USB HID, Bluetooth LE). So I will probably start a lot of personal projects and finish few to none of them 😃

I think there’s a good chance that – after an appropriately sized break – I will return as a volunteer contributor to K-9 Mail/Thunderbird for Android.

But for now, I say: Goodbye K-9 Mail 👋

This leaves me with saying thank you to everyone who contributed to K-9 Mail and Thunderbird for Android over the years. People wrote code, translated the app, reported bugs, helped other users, gave money, promoted the app, and much more. Thank you all 🙏

My youngest brother gave me his “old” HTC Magic ❤️ ↩
This is not surprising when you learn that K-9 Mail was forked from the AOSP Email app in October 2008. ↩
The K-9 Mail project was initially hosted on Google Code where Git wasn’t available at the time. ↩
Jesse moved on to build keyboards. ↩
As a freelancer I rarely worked more than 30 hours a week. I made sure there was always enough time to work on K-9 Mail. ↩
There’s surprisingly many of these forks around. Yet hardly any of the teams working on them ever contributed back to K-9 Mail 😞 ↩
The app currently doesn’t support JMAP. However, the code for basic JMAP support can be found in the repository. ↩
You can find monthly progress reports about this in the mobile section of the Thunderbird blog. ↩

My second year as a full-time open source developer

Sat, 22 Jan 2022 00:00:00 +0100

2020 was my first year as a full-time open source developer. Writing about it in this blog was a nice way to reflect on the year. And some people seemed to have enjoyed it.

So here’s a summary of how my 2021 went.

K-9 Mail

Most of my “open source time” is spent on K-9 Mail. I first started contributing to the project in 2010 and became the project lead in 2014.
The reason I decided to dedicate all of my time to open source is because I wanted to spend more time working on K-9 Mail. There’s certainly enough issues in the bug tracker to keep me busy for a couple of years.

The goal for 2020 was to release a new version of the app (the last stable version was from September 2018). If you’ve read my blog post reviewing that year, you know that I didn’t manage to accomplish that goal. But I’m happy to report that 2021 was the year that finally saw the release of a new stable version - K-9 Mail 5.800.

Even though it took a very long time to get that version done, it was far from perfect. When we built up the new user interface, not all features of the old UI made it into the new version right away. It was always the goal to re-implement those features. But development dragged on. So at some point I decided it was more important to release a new version than it was to get to feature parity.
Of course, not all users agreed. Some hated the new user interface. Some really missed the features that didn’t make the cut. Some shared their opinions on how to run an open source project 🙄 But there were also a lot of users who liked the new look.

Work on the next release is ongoing, but somewhat slow. It doesn’t help that some device vendors decided their Android version will do things differently for no good reason 😭

Read more on what the project has been up to in 2021 in Review.

Blog posts

cketti from last year believed he was getting better at writing blog posts. It looks like his assessment has not been entirely accurate.

The only post I published in 2021 was a list of things I did the year before that 😐

My first year as a full-time open source developer

Contributions to open source projects

Even though I spent almost all of my time working on K-9 Mail, I still managed to make a couple of small contributions to other open source projects.

Own projects

AdbScreenAwake - Inspired by @ErikHellman complaining that there should be an Android developer option called “Keep screen on while ADB is connected”, I hacked up a quick prototype app to do just that. You can read a bit more about it in this Twitter thread.
Jitsi Hacks - I updated some of the hacks to make them work with newer versions of Jitsi Meet. (Un)fortunately, the Jitsi team is evolving Jitsi Meet at an incredible pace. Whenever they change the user interface, there’s a good chance some of the hacks will stop working 😢
MailToCompat - I deprecated the library because the functionality is now available as androidx.core.net.MailTo. Check out the blog post android.net.MailTo is broken for more details.
The Great Suspender (fork) - After the extension in the Chrome Web Store was suspected to contain malware, I forked the repository to build my own version with all of the “phone home” functionality removed.
Please note that I have no intention of maintaining this extension for anyone but myself. Feel free to use it. But don’t tell me about it if it doesn’t work for you 😜

Other projects

AndroidStudyGroup/conferences - added a few events; but as one of the maintainers, I mostly click the “Merge pull request” button
android/storage-samples - various bug fixes
c-base/c-base-org-lektor - uwekamper and I rewrote the c-base website
Element Android - fixed the preview when sharing images
ErikHellman/base45-kotlin - small improvements
EventFahrplan - removed unused code and reviewed a couple of pull requests
FastAdapter - added a ‘reorder via drag & drop’ screen to the sample app; made small changes to the library to support this use case
jakewharton.com - I create pull requests to fix typos in blog posts if the site is built from a Git repository 😀
MaterialDrawer - small fix to support loading images via URIs using a custom scheme
signald - small changes and fixes
signald.org - fixed some links
SlimSocial for Twitter - fixed a bug in the code handling the Share intent

Donations to open source projects

Aside from recurring donations set up the previous year, 2021 was the year of spontaneous donations. I mostly sponsored projects and individuals because I liked their work, not because I was using the software or service they had created.

The donations to The Tor Project, WebRTC.rs, pixelfed, Simply Secure, and gitea I made via FundOSS.

I also sponsored the following individuals:

bleeptrack - creates art using code
chrisrhymes - creator of the bulma-clean-theme Jekyll theme I used for the Jitsi Hacks website
johnjohndoe - maintainer of the EventFahrplan Android app
Mariatta - does lots of things in the Python community; gave an inspiring talk at the Global Maintainer Summit 2021
mikepenz - creator of the MaterialDrawer and FastAdapter libraries we use in K-9 Mail
Slavi Pantaleev - creator of matrix-docker-ansible-deploy, which I use to manage my Matrix server

Financials

In 2020 my main source of income was a grant from the Prototype Fund. But applying for grants and the associated paperwork is… not fun. So in 2021 I tried something else.

K-9 Mail

On I love Free Software Day I published the blog post K-9 Mail is looking for funding. I was asking users for money so I could dedicate 80% of my time to working on the app.

The post was picked up by various news sites and in February alone K-9 Mail’s Liberapay account received around 19,000 EUR 🎉

At the end of the year a total of 51,002.55 EUR had been donated via Liberapay, GitHub Sponsors and bank transfers. Check out Donations in 2021 for more details.

Liberapay

In 2021 my personal Liberapay account received 58.46 EUR from 5 patrons 💖

Total

The donations for K-9 Mail together with the donations to my personal Liberapay account add up to 51,061.01 EUR.

Reflections

If it was a regular job, a salary of €51k would be very disappointing to me. But it is not. I get to do what I love every day. And on days where I don’t feel the love, I just do something else.
It’s great to see that what I create is useful to others. It’s awesome that so many people find this work valuable enough to give money 💖

My plan was to use 80% of my time to work on K-9 Mail. But in 2021 it was much more than that. It hasn’t been a problem so far. But I think I need to find a healthier balance.

I feel relieved that I finally managed to release a new stable version of K-9 Mail. Hopefully it won’t be another 3 years until the next release 🤞

Plans for 2022

The goals for 2022 are simple. Keep working on K-9 Mail to make the app better. And convince enough people to sponsor my open source work, so I can keep doing this full time.

If you want to help, become a sponsor 😃

Thanks

None of this would have been possible without the people donating to the K-9 Mail project and the ones sponsoring me directly. Thank you! ❤️

My first year as a full-time open source developer

Thu, 14 Jan 2021 00:00:00 +0100

Pandemic aside, 2020 has been an exciting year for me.

For more than 10 years I was working on open source software in my spare time. Then, in late 2019, a grant from the Prototype Fund allowed me to work on K-9 Mail full-time for six months. And it was awesome! So afterwards I decided to just keep going. I was working as a freelancer, so there wasn’t really a job I had to quit first.

Here’s a probably incomplete list of open source-y things I did in 2020. It’s also rather brief because I still haven’t figured out how to write blog posts without rewriting most of it, then rewriting it some more, and then throwing away half of it because I don’t like those parts anymore.
It feels like I’m slowly getting better, though 🤞

K-9 Mail

I’ve spent most of my time working on K-9 Mail. The app was in a difficult state. The last stable release was in September 2018. My goal was to publish a new stable version in 2020. By releasing 22 beta versions we got closer to that goal. But the app is still not in a state where I’m comfortable cutting a new stable release 😞

As K-9 Mail’s maintainer I’m also dealing with all the non-development-related aspects of the project:

In 2020 the website got a new domain name – k9mail.app – and a redesign, thanks to Anxhelo Lushka.
I set up a discourse instance to replace the mailing list: forum.k9mail.app
And the project now has a presence on Twitter (@k9mail_app) and Mastodon (@[email protected]).

Jitsi Hacks

Due to the pandemic much of my social life took place in video conferences, powered by the wonderful open source project Jitsi. At some point I started poking around in the browser’s developer tools trying to do things the web interface wasn’t built to do. This resulted in multiple “hacks” that I shared with friends.

Since by then I had convinced myself that I was an “open source developer” I figured I should publish this little side project. So I created a small website containing “installation” and usage instructions for the different “hacks”.
I am quite proud of myself because I refrained from buying a domain for the project. A sub-domain does just fine for a small side project: jitsi-hacks.cketti.eu

Blog posts

In 2020 I published five blog posts. That is five more than the year prior 😀

Contributions to open source projects

In 2020 I contributed to quite the variety of open source projects.

Own projects

Aside from the ones previously mentioned I also worked on these projects of mine:

MailToCompat - because Android’s MailTo class is broken I created a fixed version
SafeContentResolver - after existing for 4 years as version 0.9.0 without any issues I figured it was time to promote the library to version 1.0.0
smartwatch2048 - open sourced an old and abandoned project

Other projects

android/storage-samples - corrected some inaccuracies in code/comments related to Android’s URI permissions 🧐
AndroidStudyGroup/conferences - added some conferences, improved the CI workflow, and cleaned up the generated HTML/JavaScript
androidx - fixed a memory leak, fixed the MailTo class, fixed ShareCompat.IntentBuilder 🐛
beautiful-jekyll - various minor improvements I wanted to use on cketti.de
bulma-clean-theme - very minor fix to the generated HTML
c-calendar - improved the c-base calendar page
dbbahnhoflive-android - some minor fixes; apparently the first community contribution 🥇
discourse - fixed a bug in the quick start guide I noticed when setting up K-9 Mail’s new forum
EventFahrplan - various fixes and refactorings
FairEmail - replaced usage of the MailTo class with the one from the AndroidX Core library
gstreamer/gst-plugins-bad - make curlsmtpsink use a less wrong date format when generating emails
iNPUTmice/jmap - various fixes and small improvements
jakewharton.com - fixed number format in a blog post; apparently the first real external contribution 🎉
jitsi/handbook - small improvements to the documentation on how to build Jitsi Meet from source
kotlin - fixes to the parameter popup; tweet including video 🎥
leakcanary - make it harder to accidentally include LeakCanary in release builds
moshi-gson-interop - small documentation fixes
okhttp - fixed code to read length of DER-encoded values
TokenAutoComplete - small fixes/improvements needed for K-9 Mail

Donations to open source projects

I usually use the I love Free Software Day as a reminder to financially support the open source projects that are important to me. In 2020 I gave money to the following projects:

andOTP
Dino
Fosstodon
KeePassXC
Let’s Encrypt
Liberapay
lichess.org
Matrix
Signal
The Great Suspender - Sadly, the software seems to have changed owners since then. And the extension might contain malware now 😞
Thunderbird
Tusky
Ubuntu

I also sponsored the following individuals via GitHub sponsors:

chrisrhymes - creator of the bulma-clean-theme Jekyll theme I used for the Jitsi Hacks website
johnjohndoe - maintainer of the EventFahrplan Android app
mikepenz - creator of the MaterialDrawer and FastAdapter libraries we use in K-9 Mail

I do most of my development work in IntelliJ IDEA-based IDEs. The open source applications IntelliJ IDEA Community Edition and Android Studio provide all the functionality I need. But to do my part in ensuring the company behind IntelliJ IDEA and the creators of the Kotlin programming language stick around, I purchased a license for IntelliJ IDEA Ultimate edition.

Financials

Most of my income in 2020 came from the Prototype Fund project. But some of it came from people who donated money using one of these platforms:

GitHub Sponsors

I signed up to the GitHub sponsors program as soon as I heard about it. On my sponsors page users could select one of the following tiers: $1 a month, $5 a month, $25 a month, $1000 a month. There are no rewards associated with any of the tiers.
To no one’s surprise, nobody signed up for the $1000 tier. But people did sign up for all of the other tiers. In total I had 29 sponsors. Some of them canceled their sponsorship after a month or two, but most of them kept it going ❤️
The total payout was 966.82 EUR. Some of that was from GitHub matching contributions. But, honestly, I can’t tell if there’s a system when GitHub is matching contributions and when they’re not.

Liberapay

In 2020 K-9 Mail’s Liberapay account received 4,794.91 EUR from ~250 patrons. That is more than double the amount received in 2019 📈

Total

That’s a total of 5,761.73 EUR in donations. Not bad. But certainly not enough to live on in Germany. It looks like someone needs to come up with a better plan for 2021 😀

Conclusion

I was disappointed with how my first year as a full-time open source developer went. I didn’t accomplish the one goal I had: releasing a new stable version of K-9 Mail.

But writing this blog post made me feel better. It made me realize that small contributions add up to quite a bit. Another way to see that is by comparing my GitHub contribution graphs for 2019 and 2020.

During the past 8 years I haven’t done much web development. Working on Jitsi Hacks made me realize that I still enjoy it quite a bit. I’ll probably end up doing more of that in the future.

Plans for 2021

I have quite a few ideas I want to work on this year. But my most important goals remain the same as they were in 2020:

Finally release a new stable version of K-9 Mail.
Figure out how to be a full-time open source developer without using up all of my savings.

Thanks

My thanks goes to everyone who has contributed to one of my projects, to the people who reviewed my blog posts, and to everyone who has helped me stay sane this last year.
A special thanks goes to all of the sponsors who have made this journey possible ❤️

Avoid Intent.resolveActivity()

Thu, 03 Sep 2020 00:00:00 +0200

Until recently PackageManager allowed every app to access information about every other installed app. While this information is vital for apps like launchers, most apps have no need to know which other apps are installed. So Android 11 makes it a little bit harder to retrieve this information. For more details see Package visibility in Android 11.

Most apps should not be affected by this change because they don’t need to perform any PackageManager queries. But, sadly, many apps contain code to start implicit intents similar to this:

val intent = … // Some implicit intent, e.g. ACTION_VIEW

if (intent.resolveActivity(packageManager) != null) {
    startActivity(intent) // or startActivityForResult(…)
}

Due to the new limitations when targeting Android 11 the Intent.resolveActivity() call might now return null even though using startActivity() with the intent would work just fine.

To make this work you could add a <queries> entry to AndroidManifest.xml as explained in the documentation. But my suggestion is to avoid using Intent.resolveActivity() in such a case.

Why avoid Intent.resolveActivity()?

I always cringe when people recommend this “resolveActivity() then startActivity()” pattern. You’re asking the system to perform the same operation twice. First “find the app to handle this Intent”, then “find the app to handle this Intent and then start that app using this Intent”.

The problem with code like this is that the environment might change between the two method invocations. There might be a suitable app installed when resolveActivity() is executed and it could be gone by the time startActivity() is executed. Granted, in practice it is very unlikely to run into this case. Still, I think we can do better.

Catch ActivityNotFoundException instead

Instead of using Intent.resolveActivity() just call startActivity() and deal with ActivityNotFoundException.

val intent = … // Some implicit intent

try {
    startActivity(intent)
} catch (e: ActivityNotFoundException) {
    // Display some error message
}

In the past, using one method over the other was mostly a matter of taste. With the changes in Android 11 this method has the big advantage that you don’t have to add anything to AndroidManifest.xml. It just works.

Always avoid Intent.resolveActivity()?

My advice only applies to this particular startActivity() use case. Of course Intent.resolveActivity() still has its uses. For example, you might want to hide a ‘take picture’ button if no camera app is installed. In that case resolveActivity() comes in handy.

Why I hide GitHub issue comments

Tue, 07 Jul 2020 00:00:00 +0200

I am the maintainer of a somewhat popular open source email app for Android, called K-9 Mail. We use GitHub to host the app’s source code and use its issue tracker to manage bug reports and feature requests.
Lately, I’ve found myself hiding issue comments more and more. Sooner or later someone will ask “why?”. And now I can point them to this blog post 😀

If you didn’t know that team members can hide comments, this is what it looks like:

There are all kinds of issue comments that I hide:

“+1” comments
Comments asking when this feature will (finally) be implemented
Comments asking why this bug hasn’t been fixed yet
“Please add this feature, it is really important to me” type of comments
Totally off-topic comments where people ask general questions about the project
“I have a similar but different problem” type of comments
Comments where people (colorfully) express their dissatisfaction that a feature still hasn’t been implemented after X years
My own comments telling people to please…
- …use the support forum for tangential discussions
- …open a new issue because their bug/feature request is totally unrelated

All of these have one thing in common. For the developer who ends up implementing the feature or fixing the bug these comments don’t add any useful information.

Especially issues for feature requests can remain open for years. When I go back to the issue page I don’t want to read through those unhelpful comments again (and again, and again). So that’s why I hide them.

I’d love to learn how other open source maintainers deal with such comments. Let me know on Mastodon or Twitter.

Thanks to @tbsprs for proofreading this post.

android.net.MailTo is broken

Mon, 22 Jun 2020 00:00:00 +0200

The MailTo class has been part of the Android Platform API from the very start. And the class hasn’t been changed in a significant way since then. This could mean the API is very good and the code doesn’t contain any known bugs. Sadly, that’s not the case. android.net.MailTo is broken and shouldn’t be used. Let’s find out what’s wrong exactly.

But I’m guessing most Android developers have never heard of this class before. So, first, let’s have a quick look at the documentation for MailTo to find out what it is about.

This class parses a mailto scheme URL and then can be queried for the parsed parameters. This implements RFC 2368.

And here’s a quick example of how the class is used:

MailTo mailToUri = MailTo.parse(
    "mailto:[email protected]?subject=Hello%20World&body=Hi");

String recipient = mailToUri.getTo();    // [email protected]
String subject = mailToUri.getSubject(); // Hello World
String body = mailToUri.getBody();       // Hi

What is broken exactly?

The bugs are in MailTo.parse(). Let’s look at them one by one.

Use of Uri.parse()

After making sure the supplied url starts with mailto:, Uri.parse() is invoked to parse the remainder of the url string.

// Strip the scheme as the Uri parser can't cope with it.
String noScheme = url.substring(MAILTO_SCHEME.length());
Uri email = Uri.parse(noScheme);
// …
String address = email.getPath();

So if the input to MailTo.parse() was mailto:[email protected]?subject=Hello, Uri.parse("[email protected]?subject=Hello") is called.

Now you might be asking yourself two things:

Why is the mailto: prefix removed before passing the URL to Uri.parse()?
Why doesn’t Uri.parse() complain that the input is missing a scheme identifier?

The answer to the first question is that many Uri methods only work for hierarchical URIs. Those are URIs where the scheme identifier is followed by ://, e.g. https://authority/path?query#fragment. If the MailTo class were using Uri.parse("mailto:[email protected]?subject=Hello") there would be no way to extract [email protected] using one of Uri’s methods.

Reading the documentation (or source code) of the Uri class reveals the answer to the second question. The Uri class also supports relative URIs, i.e. ones without a scheme identifier. These are automatically treated as hierarchical URIs. And that’s why email.getPath() returns the recipient address [email protected].

Wanting the input to be parsed as relative URL is the source of the first bug. If one of the query parameters contains an unencoded : character, the input won’t be treated as a relative URL. Instead everything up to the colon will be treated as scheme identifier.

Example:

Uri parsed = Uri.parse("[email protected]?body=oh:no");

String scheme = parsed.getScheme();                     
// Result: [email protected]?body=oh

String schemeSpecific = parsed.getSchemeSpecificPart();
// Result: no

To be fair, RFC 2368 requires all URL reserved characters (and that includes :) in a header value to be percent-encoded (%3A). So an RFC 2368-compliant URL should look like this: mailto:[email protected]?body=oh%3Ano. However, this restriction has been relaxed in RFC 6068 which replaced RFC 2368 in 2010. Now the colon and some other characters are allowed to be used without encoding inside header field values. This also frequently happens in real world scenarios. The result is that mailto URIs containing an unencoded : are not parsed properly by the MailTo class. And this is not limited to the header field containing the colon. None of the URI is parsed correctly.

Use of Uri.getQuery()

The next issue is the use of Uri.getQuery().

// Parse out the query parameters
String query = email.getQuery();
if (query != null ) {
  String[] queries = query.split("&");
  for (String q : queries) {
    String[] nameval = q.split("=");
    if (nameval.length == 0) {
        continue;
    }
    // insert the headers with the name in lowercase so that
    // we can easily find common headers
    m.mHeaders.put(Uri.decode(nameval[0]).toLowerCase(Locale.ROOT),
        nameval.length > 1 ? Uri.decode(nameval[1]) : null);
  }
}

Uri.getQuery() returns the query part of the URI with percent-encoded characters decoded. So with mailto:?body=Q%26A%3Dnice&subject=Hi as input we end up with body=Q&A=nice&subject=Hi in the variable query. Reading a bit further in the code snippet we can see how this becomes a problem when the string is split on the & character. We end up with an array containing the elements body=Q, A=nice, subject=Hi, breaking the body text apart. The mistake is that Uri.getQuery() was used instead of Uri.getEncodedQuery() to get the encoded query.

Double decoding

After extracting the query parameters Uri.decode() is used to percent-decode the individual strings. But since Uri.getQuery() already decoded the query once, we now end up with double decoding. For example, the body value in mailto:?body=%2525 is first decoded to %25, then again to %. This double decoding bug is only a side-effect of the previous bug. Once that one is fixed, the code that is decoding the query actually produces the expected result.

We should get the class fixed, right?

Over the years these bugs have been reported numerous times via Android’s issue tracker. They have either been closed during an issue tracker cleanup or closed because the issue wasn’t understood properly. I have created a new issue that includes a link to my MailToBugs repository which contains failing tests for all of these bugs.
But even with a fix in a new Android version, apps would still need to work around the bugs as long as they want to support older platform versions. And we all know it would probably be 5+ years before we could drop the workarounds. 😞

Fixed version

If you are using the platform’s MailTo class and are looking for a drop-in replacement without these bugs, check out ~~my MailToCompat library~~ androidx.core.net.MailTo.

Update: The fixed class will also be part of the AndroidX Core library starting with version 1.5.0-alpha02. Big thanks to Ian Lake for suggesting I make this a contribution to AndroidX Core and then shepherding the change on r.android.com ❤️
As soon as the class will be included in a stable release of AndroidX Core I’ll retire my MailToCompat library.

Update: Starting with AndroidX Core 1.5.0 the fixed MailTo class is available in stable releases of the library. So I marked my MailToCompat library as deprecated and archived the GitHub repository.

Should I be using `MailTo`?

Unless you’re writing an email client, probably not. Yet, I’ve seen lots of snippets that try to handle clicks on mailto URIs in a WebView. They look something like this:

// DON'T DO THIS
public class MyWebViewClient extends WebViewClient {
  // …

  @Override
  public boolean shouldOverrideUrlLoading(WebView view, String url) {
    Uri uri = Uri.parse(url);

    if (uri.getScheme().equalsIgnoreCase("mailto")) {
      MailTo mailTo = MailTo.parse(url);

      // DON'T DO THIS
      Intent intent = new Intent(Intent.ACTION_SEND);
      intent.setType("message/rfc822");
      intent.putExtra(Intent.EXTRA_EMAIL, new String[] { mailTo.getTo() });
      intent.putExtra(Intent.EXTRA_TEXT, mailTo.getBody());
      intent.putExtra(Intent.EXTRA_SUBJECT, mailTo.getSubject());
      intent.putExtra(Intent.EXTRA_CC, mailTo.getCc());

      context.startActivity(intent);
      return true;
    }

    // …
  }
}

That’s unnecessary. Email apps handle mailto: URIs just fine. WebView’s default behavior of using ACTION_VIEW with the URI works just fine. Alternatively, you could use ACTION_SENDTO with the unmodified URI. For more information check out my blog post Sending Email using Intents.

Summary

The fact that android.net.MailTo hasn’t been fixed in all those years is a good indicator that the class isn’t widely used. The main audience is authors of email clients. And we all had to either learn the hard way that we can’t rely on this class or never heard about it and went with our own implementation right from the start.

The takeaway is that you shouldn’t be using android.net.MailTo. Instead use androidx.core.net.MailTo. And hopefully some day we can mark android.net.MailTo as @Deprecated and forget all about it. ⚰️

Update (2020-07-09): I updated the section “Fixed version” with information about the code being included in the AndroidX Core library.

Update (2021-07-19): My MailToCompat library is now deprecated because androidx.core.net.MailTo is available in a stable release of the AndroidX Core library.

content:// URIs and OkHttp

Sat, 23 May 2020 00:00:00 +0200

Over a year ago I typed up an implementation of OkHttp’s RequestBody that supports Android’s content:// URIs. I called it ContentUriRequestBody and made it available as a Gist. The other day a comment on the Gist made me realize that this code might be helpful to more people than the person for whom it was originally created; probably more so when accompanied by some form of explanation. So here we go.

Uploading a document

Using Android’s Storage Access Framework to allow the user to select a document is fairly straight-forward. We do it like this:

val openDocumentIntent = Intent(Intent.ACTION_OPEN_DOCUMENT).apply {
    type = "*/*"
    addCategory(Intent.CATEGORY_OPENABLE)
}

startActivityForResult(openDocumentIntent, REQUEST_UPLOAD_DOCUMENT)

When the user has selected a document we get the result in onActivityResult(). The data field of the result Intent contains a content:// URI that can be used to access its contents.

We’ll get back to this in a minute. First, let’s look at how to upload a file using an HTTP POST request with OkHttp.

val request = Request.Builder()
    .url("https://domain.example/upload")
    .post(requestBody)
    .build()

val response = okHttpClient.newCall(request).execute()

So, we need a RequestBody instance that provides the content to be uploaded. If we look closer, we notice it is an abstract class and none of the implementations that come with OkHttp seem to fit our use case. But that can easily be fixed. There’s only two functions we need to implement:

class ContentUriRequestBody(
    private val contentResolver: ContentResolver,
    private val contentUri: Uri
) : RequestBody() {

    override fun contentType(): MediaType? {
        val contentType = contentResolver.getType(contentUri)
        return contentType?.toMediaTypeOrNull()
    }

    override fun writeTo(bufferedSink: BufferedSink) {
        val inputStream = contentResolver.openInputStream(contentUri)
            ?: throw IOException("Couldn't open content URI for reading")

        inputStream.source().use { source ->
            bufferedSink.writeAll(source)
        }
    }
}

The contentType() function provides the media type (also called MIME type, or content type) for the content, e.g. image/jpeg or text/plain. Our content:// URI can be queried for the media type of the document. This is done by calling ContentResolver.getType(contentUri).

To send the actual bytes that make up the document the writeTo() function is called by OkHttp. Using ContentResolver.openInputStream() gives us a java.io.InputStream instance to read the data from. OkHttp is also using streams to read data from and write data to the network. But not the java.io API. Instead it is built on top of a library called Okio, that implements the same concept, but in a much nicer way. Fortunately, Okio comes with an easy way to treat java.io.InputStream and java.io.OutputStream like the corresponding types in the Okio world, Source and Sink. When calling the writeTo() function, OkHttp is handing us a BufferedSink and expects us to write the data to it. Since the document could be very large, we don’t want to read all of the bytes into memory and the write them to the BufferedSink. Instead we do what streams were designed to support: read a small amount of bytes from the document into memory, then write them to the BufferedSink so OkHttp can send them over the network. This step is repeated until we have read and subsequently written all of the document bytes. But because copying data from a source stream to a destination stream is such a common operation, there’s functionality in Okio that does all the heavy lifting for us. All we need to do is call the extension function source() to turn the InputStream we got from ContentResolver.openInputStream() into a Source. Then we call BufferedSink.writeAll(Source) and Okio will copy all bytes from the source to the sink.

And that’s about it. This is more or less everything you need to send content referenced via a content:// URI as body of an HTTP request using OkHttp.

Writing downloaded content to a document

Downloading content and writing it to a document is even easier because it doesn’t require us to extend a class. Displaying the system UI to allow the user to select a location and name for a new document is done like this:

val openDocumentIntent = Intent(Intent.ACTION_CREATE_DOCUMENT).apply {
    type = "image/jpeg"
    addCategory(Intent.CATEGORY_OPENABLE)
    putExtra(Intent.EXTRA_TITLE, "kitten.jpeg")
}

startActivityForResult(openDocumentIntent, REQUEST_DOWNLOAD_DOCUMENT)

Again, we get a content URI in the result Intent delivered to onActivityResult(). We can then make an HTTP request and write the response to the document.

val request = Request.Builder()
    .url("https://placekitten.com/300/300")
    .build()

okHttpClient.newCall(request).execute().use { response ->
    if (response.isSuccessful) {
        response.body!!.source().use { bufferedSource ->
            val outputStream = contentResolver.openOutputStream(contentUri)
                ?: throw IOException("Couldn't open content URI")

            outputStream.sink().use { sink ->
                bufferedSource.readAll(sink)
            }
        }
    }
}

It works similar to uploading, only this time we get a java.io.OutputStream instance when calling ContentResolver.openOutputStream() with our content:// URI. Again, there’s a handy extension function to turn this into one of Okio’s types, in this case Sink. The API surface of Okio’s Source and Sink types is rather limited. The really useful functionality comes with BufferedSource and BufferedSink. We could easily turn our Sink into a BufferedSink by using the extension function buffer(). Then we’d be able to call BufferedSink.writeAll(Source) like we did when uploading a document. But because OkHttp is already handing us a BufferedSource, we can also call BufferedSource.readAll(Sink) to copy all bytes from the source to the sink.

Sample app

I hope this post has helped you to understand how to make OkHttp play nice with content:// URIs. Of course code snippets in blog posts always seem to be missing some crucial bits of information. So I created a small sample app that you can play around with: OkHttpWithContentUri

When URI permissions are in the way

Mon, 03 Apr 2017 00:00:00 +0200

Last year I read the blog post Notifications, Sounds, Android 7.0, and Aggravation by the always well-informed Mark Murphy. In it Mark describes how the ban on file: URIs has made things difficult for developers who want to use custom notification sounds. Apparently, the notification subsystem wasn’t really prepared for content: URIs taking over. It’s currently not possible to use Notification.Builder#setSound(Uri) with a content: URI pointing to a ContentProvider that hasn’t been exported. That means FileProvider, the standard solution when it comes to exposing files via content: URIs, won’t work.

Mark lists a couple of options to work around the problem. Personally, I thought the option he named “The Axe”, using a custom ContentProvider without URI permissions, was the cleanest solution. For unrelated reasons I was reading the source code of FileProvider at that time. With the blog post in mind I thought it should be fairly easy to change the class to support that use case. And that’s what I did. The result was PublicFileProvider, a modified version of FileProvider that doesn’t require URI permissions and that takes special care to only expose files read-only.

For the last 7 months the project was gathering dust on GitHub. But the other day I stumbled across this entry in the bug tracker for the Android O Preview where Mark reiterates that the permission issue should be fixed in the platform rather than worked around by app developers. I agree, but I’m not confident this will be fixed in Android O. Even if it is, there’s still Android 7.0/7.1 we have to deal with. So there might still be some demand for something like PublicFileProvider. Which is why I added a bit more documentation and published the library on Maven Central. If you run into permission issues with custom notification sounds give PublicFileProvider a try.

Check out PublicFileProvider on GitHub. Feedback is always welcome. Open an issue on GitHub or reach out via Twitter. I’m @cketti. Please also let me know if you find another use case for this library 🙂

Share URL to Clipboard

Wed, 15 Jun 2016 00:00:00 +0200

On Android many apps allow the user to share the currently displayed content to another app. Often it’s a link to the content on the web together with some additional information, e.g. the headline and maybe the lead of a news article.

A couple of weeks back I saw the following tweet in my timeline.

Hey native apps: if I share to clipboard, just give me the URL, not the URL & "hey kids check out this cool link" or some other bullshit
— Jake Archibald (@jaffathecake) May 1, 2016

Judging by the reactions, Jake is not the only one frustrated by this. So let’s have a closer look at how sharing to the clipboard works and how the requested feature can be implemented.

Sharing content on Android is as simple as creating an Intent with the ACTION_SEND action. You also have to specify the media type of the content. In this case we’re only interested in text, so the type is text/plain and the content itself goes into the EXTRA_TEXT extra.

Intent shareIntent = new Intent();
shareIntent.setAction(Intent.ACTION_SEND);
shareIntent.setType("text/plain");
shareIntent.putExtra(Intent.EXTRA_TEXT,
    "hey kids check out this cool link\n" +
    "https://example.org/cool-link");

startActivity(Intent.createChooser(shareIntent, "Share with"));

And that’s about all app developers have to do in order to show the Share dialog.

Copy to Clipboard

As you can see in the image below “Copy to clipboard” is an option in the list of available Share targets.

What many people probably don’t know is that this is not a feature built into the Android platform. The Copy to clipboard action is part of the Google Drive app and was only added at the end of 2012.

And of course there exist many more applications that provide the same functionality. So the easiest solution, sharing only the URL when the user selects Copy to clipboard, is not really feasible.

The simplest way to satisfy the original request is to put only the URL into EXTRA_TEXT extra of the Share Intent.

This way only the URL ends up in the clipboard when Copy to Clipboard is selected. But let’s say you want to send a link to a news article to a friend. In that case the headline and a short excerpt would surely be much appreciated by the recipient. And that’s the reason why most apps provide this additional text.

A possible way around this situation is to not use the Share facility and instead provide another way to copy a link to the clipboard. Twitter, for example, added a “Copy link to Tweet” entry to its menu.

Functionality-wise this should make most users happy. Getting only the link is easy enough, while sharing uses additional text to provide some context.

However, this can lead to a confusing user experience. By now users know that Copy to Clipboard can be found in the Share dialog and don’t necessarily look elsewhere.

Fortunately, it’s easy enough to add your own action to the Share dialog. So let’s add a Copy link to clipboard entry that does just that. While all regular Share targets receive the link and additional text.

To make this work, we slightly modify the Share snippet from before.

Using EXTRA_INITIAL_INTENTS we can pass in a list of Intents that will be added to the Chooser dialog. In this case we point to an the Activity CopyToClipboardActivity that we still need to define.

In onCreate() we simply get the URL from the Intent that started the Activity and copy it to the clipboard. Then we finish the Activity right away to return to the screen where the Share event originated.

The icon and label we give CopyToClipboardActivity in the app manifest are the ones being used by the Chooser dialog when displaying our custom entry. Using the Theme.NoDisplay theme disables the activity transition animation when our short-lived Activity is launched.

And here is what it looks like to the user:

Of course there’s still the “Copy to clipboard” provided by the Drive app. And it stills copies all of the text to the clipboard. Maybe sometimes this is just what your users want. For all the other times, when it’s just the URL they’re after, the Copy link to clipboard option is right there at the top.

The source code of a sample application demonstrating all of this can be found on GitHub.

I’m lazy. Any other option?

As a developer, you, of course, created an app that is a shining example when it comes to copying links. But what about the other apps you use? Their creators are busy implementing all of your other feature requests. Now what?

Fortunately, Android’s powerful Intent mechanism means you don’t actually need the cooperation of app authors to get the desired functionality. All you need is an app that hooks into the Share dialog and then extracts links from the text shared to it. Writing such an app is easy enough and will be the topic of a future post.

If you have other ideas on how to make copying links to the clipboard easier, I’d love to hear from you. I’m @cketti on Twitter.

Update: Use Theme.NoDisplay instead of Theme.Translucent.NoTitleBar as suggested by @devangelslondon.

Sending Email using Intents

Fri, 08 Jan 2016 00:00:00 +0100

This post was originally published on Medium: Android: Sending Email using Intents

Many things on Android are as easy as starting an Activity using the right Intent. Sending an email to a specific recipient is one of those things.

Sadly, there is much bad advice out there on how to send emails using Intents. Because I write an open source email app I have a pretty good understanding of what goes on behind the scenes. That’s why I believe I know better than all those other people :)

ACTION_SENDTO

To start the Activity to compose an email in your user’s favorite email app you use an Intent with the ACTION_SENDTO action. Unfortunately, the documentation is very sparse. It simply states:

Activity Action: Send a message to someone specified by the data.
Input: getData() is URI describing the target.
Output: nothing.

This doesn’t mention email at all because ACTION_SENDTO is very generic and can also be used to send e.g. SMS, MMS or XMPP messages. To send an email we have to use a mailto URI as defined by RFC 6068. In its simplest form such a URI consists of mailto: followed by an email address, e.g. mailto:[email protected]. You might recognize this syntax from email links in HTML documents.

Creating such an Intent might look like this:

Most of the time you want to provide more data than just a recipient. And the mailto specification allows us to also specify CC and BCC recipients, a subject, and even text for the email body.

mailto:[email protected][email protected]&subject=Important%20message&body=Hi%20there

Generating such a mailto URI is a bit tedious because you have to percent-encode certain characters as you can see in the example above. Android’s Uri.Builder class would be the perfect fit since it makes it easy to add query parameters while taking care of the encoding. Unfortunately, building Uri instances doesn’t work very well for non-hierarchical URIs like mailto URIs. One way to work around that is to manually construct the URI and dealing with the encoding.

Some code samples advocate the use of EXTRA_SUBJECT, EXTRA_TEXT, etc. Those are certainly easier to use. However, the Android API reference only mentions them in connection with ACTION_SEND and ACTION_SEND_MULTIPLE. Those are more general Intents for sharing content to nobody in particular. The fact that the Extras also work with ACTION_SENDTO was an undocumented “feature” of AOSP Email and the Gmail app. That functionality then had to be copied by all other Android email clients because apps were and are using those Extras. But since all major email clients support the use of query parameters in mailto URIs I see no reason to use an undocumented feature and advice against it.

It is also worth mentioning that you shouldn’t use Intent.createChooser() to launch an email Intent. Users most likely have a preferred email app that they want to be able to select as default app. The standard behavior when resolving an Intent deals with that just fine.

Allow users to select a default app

There are a couple of other things that need to be considered. Line breaks in the body, for example, need to be encoded as %0D%0A (CRLF). Because it’s no fun to deal with all of that when all you want to do is send an email, I wrote a small library to take care of the dirty details for you.

EmailIntentBuilder — Making life easier

The goal of the Email Intent Builder library is to make creating email Intents as easy as possible. You can find it on Maven Central, the source code on GitHub.

Building an email Intent is a simple matter of calling some methods on the builder:

Most of the time you also want to launch the Intent. The start() method will do that for you while also taking care of not throwing an exception if no app could be found to handle the Intent.

What about Attachments?

Unfortunately, the ACTION_SENDTO Intent doesn’t support attachments. If you need to send an email containing one or more attachments you should use Android’s more generic share mechanism, i.e. ACTION_SEND or ACTION_SEND_MULTIPLE. A lot of people have realized that this leads to quite a few more than just email apps showing up in the app chooser dialog. How to properly deal with that is material for another post.

I’d love to hear about your experiences dealing with email on Android. Let me know on Twitter.

cketti

Goodbye K-9 Mail

My second year as a full-time open source developer

K-9 Mail

Blog posts

Contributions to open source projects

Own projects

Other projects

Donations to open source projects

Financials

K-9 Mail

Liberapay

Total

Reflections

Plans for 2022

Thanks

My first year as a full-time open source developer

K-9 Mail

Jitsi Hacks

Blog posts

Contributions to open source projects

Own projects

Other projects

Donations to open source projects

Financials

GitHub Sponsors

Liberapay

Total

Conclusion

Plans for 2021

Thanks

Avoid Intent.resolveActivity()

Why avoid Intent.resolveActivity()?

Catch ActivityNotFoundException instead

Always avoid Intent.resolveActivity()?

Why I hide GitHub issue comments

android.net.MailTo is broken

What is broken exactly?

Use of Uri.parse()

Use of Uri.getQuery()

Double decoding

We should get the class fixed, right?

Fixed version

Should I be using MailTo?

Summary

content:// URIs and OkHttp

Uploading a document

Writing downloaded content to a document

Sample app

When URI permissions are in the way

Share URL to Clipboard

Share to clipboard

Sharing text on Android

Copy to Clipboard

Option 1: Only share the link

Option 2: Don’t share

Option 3: Extend the Share dialog

I’m lazy. Any other option?

Sending Email using Intents

ACTION_SENDTO

EmailIntentBuilder — Making life easier

What about Attachments?

Should I be using `MailTo`?