I've recently come across a situation in Swift, where I needed an object of some kind that could dynamically be extended with stored properties. They had to be stored, because they were non-trivial to compute. And because of this, I also didn't want to compute all of them in places where I might need only a hand full. And finally, I didn't want to deal with Optionals - the computation already throws an error if it fails, so once it succeeds it should be guaranteed to be there and not need another validation (or worse even: an !).
In short: I wanted something that could be pieced together dynamically but still be verified by the compiler.

The Basics

To make this work in Swift, the @dynamicMemberLookup feature is critical. It allows an object to act as a proxy for one or more objects. For details see SE-195 and SE-252. Also, opaque result types (SE-244) and primary associated types (SE-346) are needed.

It all starts with a base protocol (in my case it already had associated types, but that's not needed for the basic example):

public protocol BuilderBase {}

This protocol will act as the base from which the next protocol will inherit and on which the extending function(s) will be defined.
Let's add an implementation for it:

struct MyBuilder: BuilderBase {}

This implementation will be the entry point for the builder chain, but also not much more. In my concrete usecase, this was even a private struct - an implementation detail hidden behind a different entrypoint function.

Now, let's add the magic. For this, we first add a sub-protocol:

@dynamicMemberLookup
public protocol Builder<Base, Lookup>: BuilderBase {
    associatedtype Base: BuilderBase
    associatedtype Lookup

    subscript<T>(dynamicMember basePath: KeyPath<Base, T>) -> T { get }
    subscript<T>(dynamicMember lookupPath: KeyPath<Lookup, T>) -> T { get  }
}

Here we define a protocol that has two associated types (which importantly are also defined as primary). Also, this protocol requires the @dynamicMemberLookup implementation and also defines which subscripts need to be provided. This is actually the magic behind it. The implementing object (which we'll add in a second) will be able to access properties of both the Base and the Lookup types. The Lookup type will contain the new properties we want to add to the builder. The Base, however, will be recursively contain the "parent" builder. And since this "parent" is also able to dynamically lookup properties of its Lookup type, we're now able to add a theoretically unlimited number of properties.

Let's add the implementation:

fileprivate struct _ExtendedBuilder<ParentBuilder, PropertiesLookup>: Builder
where ParentBuilder: BuilderBase
{
    typealias Base = ParentBuilder
    typealias Lookup = PropertiesLookup
    
    let base: Base
    let lookup: Lookup
    
    subscript<T>(dynamicMember basePath: KeyPath<Base, T>) -> T {
        base[keyPath: basePath]
    }

    subscript<T>(dynamicMember lookupPath: KeyPath<Lookup, T>) -> T {
        lookup[keyPath: lookupPath]
    }
}

Notice how this type is fileprivate! It's an implementation detail that users don't need to know about. It's created using the following function:

extension BuilderBase {
    public func withProperties<PropertiesLookup>(of lookup: PropertiesLookup) -> some Builder<Self, PropertiesLookup> {
        _ExtendedBuilder(base: self, lookup: lookup)
    }
}

Since this function is defined on the BuilderBase protocol (the base of all builders), we can also call it on implementations our Builder.

And that's all that's needed.

The Extensions

Now we can start adding properties (e.g. in a different file or even module):

import Foundation

struct Greeting {
    let greeting: String
}

extension BuilderBase {
    func withGreeting(_ greeting: String) -> some Builder<Self, Greeting> {
        withProperties(of: Greeting(greeting: greeting))
    }
}

struct Person {
    let firstName: String
    let lastName: String
}

extension BuilderBase {
    func withPerson(firstName: String, lastName: String) -> some Builder<Self, Person> {
        withProperties(of: Person(firstName: firstName, lastName: lastName))
    }
}

struct Age {
    let birthday: Date
    let ageInYears: Int
}

enum AgeCalculationError: Error {
    case birthdayInFuture
    case unableToCalculateAge
}

extension BuilderBase {
    func withAge(bornAt birthday: Date) throws -> some Builder<Self, Age> {
        guard birthday < .now else { throw AgeCalculationError.birthdayInFuture }
        guard let age = Calendar.current.dateComponents([.year], from: birthday, to: .now).year
        else { throw AgeCalculationError.unableToCalculateAge }
        return withProperties(of: Age(birthday: birthday, ageInYears: age))
    }
}

The Usage

Finally, we can call these new functions and build our values:

do {
    let values = try MyBuilder()
        .withGreeting("Hello")
        .withPerson(firstName: "John", lastName: "Appleseed")
        .withAge(bornAt: Date(timeIntervalSince1970: 20 * 60 * 60 * 24 * 365))

    print(values.greeting, values.firstName, values.lastName)
    print(values.birthday)
    print(values.ageInYears)
} catch {
    print("Failed to build values: \(error)")
}

The compiler will typecheck our code and we only have access to values we previously added:

To prove this, let's remove the call to withGreeting:

The greeting is no longer in the list of available properties! And trying to access it will lead to a compiler error (which isn't pinpointing exactly what's wrong, but that's a different topic):

The Pitfalls

There are a few pitfalls to be aware of, though, that can lead to confusion otherwise.

Overrides

The added properties are added in a flat manner. This means that previously defined properties with the same name will be overridden!

struct Person {
    let name: String
}

struct Dog {
    let name: String
}

extension BuilderBase {
    func withPerson(named name: String) -> some Builder<Self, Person> {
        withProperties(of: Person(name: name))
    }
    
    func withDog(named name: String) -> some Builder<Self, Dog> {
        withProperties(of: Dog(name: name))
    }
}


let values = MyBuilder()
    .withPerson(named: "John Appleseed")
    .withDog(named: "Rex")

print(values.name) // Will print "Rex"!

You can work around this by making the outer structs contain inner structs (or tuples) with the actual values:

struct PersonBag {
    struct Properties {
        let name: String
    }

    let person: Properties

    init(name: String) {
        person = .init(name: name)
    }
}

struct DogBag {
    let dog: (name: String, isAGoodBoy: Bool)

    init(name: String) {
        dog = (name, true)
    }
}

extension BuilderBase {
    func withPerson(named name: String) -> some Builder<Self, PersonBag> {
        withProperties(of: PersonBag(name: name))
    }

    func withDog(named name: String) -> some Builder<Self, DogBag> {
        withProperties(of: DogBag(name: name))
    }
}


let values = MyBuilder()
    .withPerson(named: "John Appleseed")
    .withDog(named: "Rex")

print(values.person.name) // Prints "John Appleseed"
print(values.dog.name) // Prints "Rex"
print(values.dog.isAGoodBoy) // Prints "true"

Performance and Memory

It's worth keeping in mind that the builder instances are recursively nested into each other on every addition.
However, this should not behave too bad memory wise, since these are all structs which take up little memory themselves.
Accessing the built properties has to (in the worst case) move down to the builder at the bottom of the builder stack. Since those are all keypath-lookups, it should have decent performance, though (since KeyPaths access fields directly in most cases).

I haven't verified this in places performance and/or memory is actually critical, though. As always: YMMV.

One more thing: Technically not a builder

Some of you might have correctly realized by now, that this isn't a real builder since it's not finalized (e.g. you can always continue 'building' on values in the examples).
This is true and there is a way to make it like a real builder. However, it didn't bring much value to my usecase which is why I didn't do it.

It's totally possible to add this, however. The base code would undergo some refactoring, though, and finalizing requires walking up the chain of stacked builders. The new base code could look something like this:

@dynamicMemberLookup
protocol BuiltResult<Base, Lookup> where Base: BuiltResult {
    associatedtype Base
    associatedtype Lookup

    subscript<T>(dynamicMember basePath: KeyPath<Base, T>) -> T { get }
    subscript<T>(dynamicMember lookupPath: KeyPath<Lookup, T>) -> T { get  }
}

protocol Builder<Base, Lookup> where Base: Builder {
    associatedtype Base
    associatedtype Lookup

    associatedtype Result: BuiltResult<Base.Result, Lookup>

    func finalize() -> Result
}

struct EmptyResult: BuiltResult {
    typealias Base = EmptyResult
    typealias Lookup = Void

    subscript<T>(dynamicMember basePath: KeyPath<Base, T>) -> T {
        self[keyPath: basePath]
    }

    subscript<T>(dynamicMember lookupPath: KeyPath<Lookup, T>) -> T {
        ()[keyPath: lookupPath]
    }
}

extension Never: Builder {
    typealias Base = Never
    typealias Lookup = Void
    typealias Result = EmptyResult

    func finalize() -> Result { fatalError("unreachable") }
}

struct MyBuilder: Builder {
    typealias Base = Never
    typealias Lookup = Void

    func finalize() -> some BuiltResult<EmptyResult, Lookup> {
        EmptyResult()
    }
}

fileprivate struct _BuiltResult<ParentResult, PropertiesLookup>: BuiltResult
where ParentResult: BuiltResult
{
    typealias Base = ParentResult
    typealias Lookup = PropertiesLookup

    let base: Base
    let lookup: Lookup

    subscript<T>(dynamicMember basePath: KeyPath<Base, T>) -> T {
        base[keyPath: basePath]
    }

    subscript<T>(dynamicMember lookupPath: KeyPath<Lookup, T>) -> T {
        lookup[keyPath: lookupPath]
    }
}

fileprivate struct _ExtendedBuilder<ParentBuilder, PropertiesLookup>: Builder
where ParentBuilder: Builder
{
    typealias Base = ParentBuilder
    typealias Lookup = PropertiesLookup

    let base: Base
    let lookup: Lookup

    func finalize() -> some BuiltResult<Base.Result, Lookup> {
        _BuiltResult(base: base.finalize(), lookup: lookup)
    }
}

extension Builder {
    func withProperties<PropertiesLookup>(of lookup: PropertiesLookup) -> some Builder<Self, PropertiesLookup> {
        _ExtendedBuilder(base: self, lookup: lookup)
    }
}

The extensions now go on Builder (instead of BuilderBase as before) and a finalize call is now needed to access the properties.
The EmptyResult type is needed (and uses Void as Lookup) to provide an empty type as the root. We could use Never here, but that would offer all kinds of (inaccessible) properties on Never (e.g. from it conforming to Identifiable) and lead to runtime crashes if someone actually uses it.

The code using it would then look like this:

struct PersonBag {
    struct Properties {
        let name: String
    }

    let person: Properties

    init(name: String) {
        person = .init(name: name)
    }
}

struct DogBag {
    let dog: (name: String, isAGoodBoy: Bool)

    init(name: String) {
        dog = (name, true)
    }
}

extension Builder {
    func withPerson(named name: String) -> some Builder<Self, PersonBag> {
        withProperties(of: PersonBag(name: name))
    }

    func withDog(named name: String) -> some Builder<Self, DogBag> {
        withProperties(of: DogBag(name: name))
    }
}

let values = MyBuilder()
    .withPerson(named: "John Appleseed")
    .withDog(named: "Rex")
    .finalize()

print(values.person.name) // Prints "John Appleseed"
print(values.dog.name) // Prints "Rex"
print(values.dog.isAGoodBoy) // Prints "true"

Notice how only builder functions can be accessed without the finalize call:

And how after the finalize call, these builder functions are no longer accessible, but only the built values:

This can actually make a nice API addition if needed!

While this year was generally no exception to this process, the chaos that was came along the management of these betas this year, was unlike everything I've witnessed so far. And one big part of this chaos is caused by SwiftUI and Combine. Don't get me wrong, those frameworks are awesome and have great potential. What I don't like is the way they were (and still are) developed. But let me explain a little more.

With Swift Evolution Apple has a way to manage the continuously increasing demand of new and improved features of their programming language Swift. Not too long ago, they further restricted the proposals in Swift Evolution, requiring them to have an implementation ready. Note, that Swift Evolution is mostly a public process. It involves a public discussion in the Swift Forums. Later in the process, there's a private  discussion where the Swift Core Team discusses the change and makes a decision about whether it's going to be added to Swift or not. This has worked well for the last few Swift releases. Also, this caused changes to Swift to be more stable once they make it into the first beta of Xcode. Partially also, because they were tested with snapshots from the master branch before.

Now what's different this time? With Xcode 11, or more precisely with or "for" SwiftUI  and Combine, there were addtions to Swift that did not go through the normal Swift Evolution process before making it into the language. Again, don't get me wrong, those changes are really nice features and I still love where the language is going. But with those features taking a "shortcut", a lot of the problems of this year's betas began.

Let's take the new "Property Wrappers" as an example. They introduce a really cool way of "wrapping" a value within a certain container, whilst not having to deal with the container itself too much. A few common such containers are Lazy, Atomic, Box, etc. I think many projects had one or the other container implementation. Personally, I've even tried to achieve something similar to "Property Wrappers" inside my FFFoundation framework. Now, with Swift 5.1, creating such property wrappers became a piece of cake. Slapping an annotation to the container type and possibly renaming a few properties and initializer and that's it. There is really a lot of potential on them. But they also demonstrate clearly the overall chaos that is present in this year's betas. In the first beta, it's normal to have bugs. And so I wasn't surprised at all that the property wrappers didn't yet work perfectly. However, the property wrappers were one of a few additions to Swift this year, that have not yet gone through Swift Evolution. But they are necessary  in a very substantial way for the two new Swift frameworks SwiftUI and Combine. To be honest, I don't know exactly when the corresponding proposals for those features were effectively created. I think some of them existed (be it in a more verbose form) before the keynote of WWDC. But their process started after the first beta. And with it came a lot of problems. During the public and later private discussion, the naming of the underlying property of a property wrapper was adjusted many times. So was the naming of the initializer and the annotation that has to be applied to the type. In a effort to be backwards compatible, the old naming was supported for a few releases. Even worse, some of the frameworks weren't adjusted in time, so that certain betas shipped with outdated frameworks - throwing warnings that only confused developers.

What I've described here using the new property wrappers basically applies to almost many other new features in Swift as well (namely opaque return types and function builders). Whilst being extremly cool, they went backwards through the normal process resulting in many adjustments and also an unstable final state. In the example of property wrappers, this means that they are currently extremly limited. Many important fixes didn't make it in time for the version of Swift 5.1 that is shipped with Xcode 11 anymore.  With SwiftUI and Combine having to adjust to all these changes, they were partially distracted from making important fixes to the core functionality of these frameworks. Which now results in (IMHO) buggy and only partially stable/usable releases of these frameworks.

Adding to this (more Swift related) chaos, there's the chaos that was introduced when Apple started to distribute an iOS 13.1 beta while iOS 13.0 was not even in GM state. There were reasons for that, sure, but still it doesn't help the quality of this year's betas. To make things worse, this years OS release schedule is scatterd like nothing I've ever witnessed. Apple has split off iPadOS from iOS to distinguish the OS of their iPads more clearly from those of the iPhones. Which is a totally valid decision and was necessary to further improve the iPad experience. But similar to tvOS, iPadOS is still based on iOS (or its core part so to speak) and I would have expected it to be released in a very similar schedule to tvOS and iOS. But I was wrong. This year, all of Apple's OS's are released on a different date (tvOS being the only exception as far as I can tell). iOS is released first, then watchOS, then iPadOS, then macOS. This makes it extremely hard to plan releases for software that runs on more than just one of those OS's.

I'm usually responsible for updating the open source and closed source frameworks of our company (as well as some of our client projects) to new versions of Swift. And basically every year I start with this task before the GM seed is available - mainly because I enjoy working with the new features and also because I know how important early feedback is for fixing bugs and making things more stable. But this year, while I still started early, I quickly gave up trying to keep up to date. With basically every new Xcode beta, something new broke and most old bugs weren't fixed - or at least not completely. Part of me thought "well, maybe there's too much going on this year. Just wait until the GM comes around and see what features actually make it in there". So I did and the GM came. But not only was there already a second GM seed of Xcode (which is not wrong but still doesn't usually happen), but the features included feel as incomplete as they usually do in the midst of beta testing. It's not unusual that I encounter bugs in the GM seed. Some of them I knew from previous betas, some were new. And that's fine. But the number of bugs I encountered this year is enourmous. For basically every new Swift and OS feature I used in the code bases I migrated, I found multiple bugs. Of course, they're all not going to be fixed in the initial release of Swift 5.1 / Xcode 11. There are usually workarounds, of course, but in the cases I've encountered they severly limit the use of the new features. Which is not as bad for closed scope/source projects where the limitations imposed by those workarounds are only noticed in this closed scope. For open source projects it's much worse, since those limitations are also imposed on the users of those open source projects.

It is currently even unclear, what should be used in production and what not. The new function builders feature has not even gone through Swift Evolution yet (and is still using an underline annotation - which usually defines them as being internal), but SwiftUI is very actively using it. You could argue, that the feature is currently taylored for SwiftUI, but still it's a strange inconsistency. And I wouldn't be surprised if there are bugs in SwiftUI that are actually bugs in the new function builders.

Long story short: as I mentioned a few times now, I do love the new features in Swift and the OS's. But this year's chaos is IMHO a real showstopper. It's not really helpful if you encounter bugs whilst still playing with the basics of a new feature. Like the new property wrappers not fully working with static properties, initializer with default values or even worse, the example of a Lazy property wrapper given in the proposal not actually being lazy. I really do hope, that Apple has learned from this year's chaos and will stick to their (usually well-evolved) processes next time. Keeping secrets is fine - heck it's Apple we're talking about - but a secret is only a good surprise, if it works out in the end. Here's to next year's betas...

One thing upfront: the decision to remove those services is sad, but probably based on how they were used (or weren't). And while a UI for configuring those services is nice, it might not justify the amount of work needed from Apple's side. In a nutshell: I'm not critizing the decision here.

The migration guide

Since the profile manager in Server 5.6 on High Sierra seems to have some problems dealing with iOS 12 and macOS Mojave devices, I decided to update my macOS server to Mojave and Server 5.7 this weekend. I needed to migrate / replace DHCP, DNS, VPN, Web and Mail services. As mentioned before, there is a migration guide. Looking through it, I've found that most of them are pretty easy to migrate. So today I started with the migration and migrated DHCP, DNS and VPN services. While the general migration really was only a few steps, it also told me a lot about the migration guide.

It's only made half-hartedly. Lots of copy-pasting happened there. There are mistakes in the listed commands. There are duplicate keys (with contradicting values) in some launch daemon property lists. Overall it really feels like it was done in a rush.

Furthermore, there is no guide on how to set up an Apache web server and use the profile manager in Server 5.7. The migration guide clearly states, that the profile manager has to stay off after the migration described in the document. So essentially, you can't run websites with Server 5.7 if you want to use the profile manager. Unless you're manually dig into the configuration the profile manager does and hack your own configuration around it (what I'll probably do).

Last but not least, there is no guide on how to migrate the mail service. I repeat, the mail service - probably the service with the most user data. I haven't checked, but it seems like every other removed service is covered in the migration guide - except mail. Looking through the config and data directories of the mail service, I cannot help but feel that the mail service is likely the most complicated one. Again, it seems like the migration guide had to be done quickly. And steps to migrate the complex mail service would have just taken too much time.

Dear Apple, if you remove services because they're no longer used enough to justify their development - fine by me. You also started of nicely by listing alternatives. But please put some last effort into it and publish a complete migration guide!

[tl;dr;] The frame of the macOS system HUD is (origin = (x = 620, y = 140), size = (width = 200, height = 200))... (at the time of writing)

In one of the reviews of CapsLocker a user disliked, that the HUDs in CapsLocker do not match the size and position of the HUDs that macOS shows (e.g. when changing the volume or display brightness).
Since I was anyway working on a new release, I said to myself "How hard can it be?" and decided to also adapt CapsLocker's HUD to match the system HUDs. I was already using a NSVisualEffectView to get the blur effect the app also already automatically adapted to macOS's "dark mode". So it was only a matter of adjusting the size and position, right?

(Spoiler alert: I didn't know the rabbit hole that was awaiting me)

Where to start?

At first I thought "Maybe there are macOS docs around the HUDs". But some googling didn't really reveal anything useful. So that didn't seem to get me anywhere.
So I had to find out which process is responsible for showing the HUDs. I threw a few more keywords at Google and eventually found this repository, which contains an alternative HUD. Quickly scanning through the source code, the most interesting part was the section about a OSDUIHelper. OSD and UI sounded promising. Doing a quick sudo find / -name "*OSDUIHelper*" gave away its home: /System/Library/CoreServices/OSDUIHelper.app

The beginning of the adventure

But simply launching it didn't do much. Since it's just an .app I had a look into the package. I figured it had to have some kind of resources which might already help me. And indeed it had resources. PDFs for various HUD contents (e.g. the volume icon). My hope was, that the HUD's size matched the size of the PDF resources. So I took the size of one of said PDFs and used it as my HUD size (170 x 170pt). But unfortunately it didn't really match the size of the system HUDs. I could have just took that as a start and make my way to the exact size using a trial and error approach. But where would be the fun in that? (Also, I wanted the corner radius and position to match as well).

Deeper down the rabbit hole

The next logic step for me was to take apart the binary. My knowledge of Assembly isn't the best, but I thought I give it a try nevertheless. So I started Hopper and disassembled it. Right in the EntryPoint code I found out how the app was communicating with the system (and that it was at least partially written in Swift 🙃):

r14 = objc_allocWithZone(@class(NSXPCListener));
r15 = (extension in Foundation):Swift.String._bridgeToObjectiveC() -> __ObjC.NSString("com.apple.OSDUIHelper", 0x15, 0x0);
rbx = [r14 initWithMachServiceName:r15];
[r15 release];
*qword = rbx;
[rbx setDelegate:*qword];
[rbx resume];

So the OSDUIHelper is listening using a NSXPCListener and the corresponding Mach service name is "com.apple.OSDUIHelper". One of the other results of find was a LaunchAgent (/System/Library/LaunchAgents/com.apple.OSDUIHelper.plist) whose contents confirmed that a XPC connection was used. With that insight, I thought maybe I could use the OSDUIHelper directly - but more on that later.

Looking through the other symbols, I found that there's a OSDRoundWindow (whose mangled name is _TtC11OSDUIHelper14OSDRoundWindow). That sounded pretty much like the window I was interested in.
Looking at its structure I spotted a method named CORNER_RADIUS (at which point I also started wondering if Apple had any coding style guide since this method didn't follow any guidelines of Swift nor Objective-C).
Either way, the implementation was more or less just a constant:

double -[_TtC11OSDUIHelper14OSDRoundWindow CORNER_RADIUS](void * self, void * _cmd) {
    xmm0 = intrinsic_movsd(xmm0, *qword); // qword_100011488
    return xmm0;
}
-> qword_100011488:
dq         0x4032000000000000

Taking 0x4032000000000000 and applying how double is stored according to IEEE754 (1 bit for the sign, 11 bits for the exponent and 52 bits for the fraction) results in 1.8e1 or simply 18.0.

But apart from the corner radius and the structural insight the disassembled binary wasn't of much help. I wasn't able to extract the size and position of the window.

A binary is made for running (and debugging)

Hopper is able to run a binary with a debugger (LLDB) attached. So I decided to give this a try. Unfortunately, OSDUIHelper is within the scope of System Integrity Protection (SIP) - don't ask me why a HUD is so important that it is part of the system's integrity, though.
So I rebooted into the recovery mode and disabled SIP and rebooted again into the normal system. Now things looked better. LLDB was able to run the binary and attach to it.
Knowing the mangled name of the window, I simply created an instance of that class and started poking around with it. But apart from getting the CORNER_RADIUS constant confirmed to be 18.0, there wasn't much useful information to get. The frame was still zero. And I couldn't really get it to layout itself. And triggering a HUD by e.g. changing the volume didn't do anything to my instance of OSDUIHelper.

Debugging the real thing

Since I knew the name of the process, it came to me that there should also be the "live process" which was actually showing the HUDs when I e.g. changed the volume. Running sudo ps aux | grep OSDUIHelper confirmed this thought. So I went right at it and attached lldb to the live process.
Now where to set the breakpoints... CORNER_RADIUS surely isn't of much help since the constant is likely read only once. And apart from that, there were only very few methods: _cornerMask, _backdropBleedAmount and two initializers. The two initializers likely aren't of much use either, because depending how the window is managed, only one is ever alive. And according to the disassembled binary the other two methods have similar semantics as CORNER_RADIUS meaning they are also only called once.
But since OSDROundWindow in the end is also just a NSWindow subclass, there are many methods on NSWindow I could use to set breakpoints. Looking at some runtime headers, I found -[NSWindow _setVisible:] to be the most interesting one. A br set -n "[NSWindow _setVisible:]" and a volume change later, the process actually paused right there.
With the System V x86-64 calling convention, the first six arguments are stored in the %RDI, %RSI, %RDX, %RCX, %R8, %R9 registers. Since every method call in Objective-C passes self as first argument, the pointer to the window had to be in %RDI.
And once I had the pointer, things were pretty easy:

(lldb) po [(NSWindow *)0x00007faa13403300 frame]
(origin = (x = 620, y = 140), size = (width = 200, height = 200))

Finally I had what I wanted. The x position of the frame was rather irrelevant since it's always centered (which the HUD of CapsLocker also always had been). So I took the other values and put them into my HUD in CapsLocker. Build and run and try it out. And hooray, CapsLocker's HUD had the exact same position and size as the system's HUD. Also, the y position seemed to be constant across all display sizes.


With that knowledge I've finished version 1.2.1 and submitted it for review. A little more than a day later it went live on the AppStore.

Using the XPC connection

As previously mentioned, I had the thought of talking to OSDUIHelper directly to get it to show the HUD. The weekend after, I sat down and tried to get this to work. I had to disassemble a few other frameworks that are involved in showing the HUDs for volume changes, brightness changes, etc. to figure out how the XPC connection had to be set up.
A few hours later I had it all figured out and was able to get the HUD to show with the following code:

@objc enum OSDImage: CLongLong {
    case eject = 6
    /* and more cases from 1 to 28 (except 18 and 24) */
}

@objc protocol OSDUIHelperProtocol {
    @objc func showImage(_ img: OSDImage,
                         onDisplayID: CGDirectDisplayID,
                         priority: CUnsignedInt,
                         msecUntilFade: CUnsignedInt,
                         withText: String?)

    /* and more methods from OSDUIHelper.OSDUIHelperProtocol */
}

let conn = NSXPCConnection(machServiceName: "com.apple.OSDUIHelper", options: [])
conn.remoteObjectInterface = NSXPCInterface(with: OSDUIHelperProtocol.self)
conn.interruptionHandler = { print("Interrupted!") }
conn.invalidationHandler = { print("Invalidated!") }
conn.resume()

let target = conn.remoteObjectProxyWithErrorHandler { print("Failed: \($0)") }
guard let helper = target as? OSDUIHelperProtocol else { fatalError("Wrong type: \(target)") }

helper.showImage(.eject, onDisplayID: CGMainDisplayID(), priority: 0x1f4, msecUntilFade: 2000, withText: "🙃 GOT YA HACKED 🙃")

The last call then resulted in the following HUD being shown:



There are other methods to e.g. show an image at a given path, but they only work with paths within the sandbox of OSDUIHelper so I don't think they're of much use. Also, it's probably not a very good idea to use this as a HUD for CapsLocker since this goes into the direction of private API. And on top of that the OSDUIHelper has quite a few bugs when showing a HUD with text. It always draws the text over the last shown text. To get rid of the text, you first need to show a HUD without a text (there's another method without a withText parameter - simply passing nil doesn't do the trick).
But it would definitively be nice if Apple allowed using the HUD for such things by making this a public API.

Swift 4, amongst other things, brought a way to serialize/deserialize data into/from model objects called Codable. Those changes were proposed under SE-0166.

I'm not going to elaborate on how Codable works since it's basically pretty straight forward. If you want to know more about it, I participated in a talk about which you can read more here (in German, though) or check out the sample repository on GitHub.

In this post I want to go into some details of using Codable in combination with CoreData.

Sample structure

To make some things easier to understand, let's create a structure to use for this blog post:

final class Person: NSManagedObject {
    @NSManaged var id: Int32
    @NSManaged var firstName: String
    @NSManaged var lastName: String
    @NSManaged var birthday: Date

    @NSManaged var dogs: Set<Dog>
}

final class Dog: NSManagedObject {
    @NSManaged var id: Int32
    @NSManaged var name: String

    @NSManaged var master: Person
}

Encodable

As you probably know, Codable is a combination of Encodable and Decodable. Using the former with NSManagedObject subclasses is almost as straightforward as you might expect it to be. As long as the conditions are met (that mainly being the properties of your model being also Encodable), you can just slap the Encodable protocol to your NSManagedObject subclass.
But if we have a closer look at the sample, we may notice that we might run into an infinite encoding loop. Why? Because CoreData generates relation properties on models. In our example the Person has a Set<Dog> and the Dog again has a Person. Trouble ahead!
Not to worry, though. Luckily we can just define our own CodingKeys and leave out the relation we don't want. In our case we don't want the Dog to encode its master again:

final class Dog: NSManagedObject, Encodable { ... } // -> Added `Encodable`
private extension Dog {
    enum CodingKeys: String, CodingKey {
         case id, name // -> no case for `master`
    }
}

Now master will not be encoded anymore and the loop is broken. Of course sometimes it's not that easy and you might have to implement the encode(to:) yourself and dynamically decide whether or not to encode a relation.
But that's basically all that has to be said about encoding Core Data model classes.

Decodable

Now on to Decodable. Here's where it gets tricky. Decodable assumes, that your model can be created at anytime and only with the encoded data (no other dependencies). And we all know, that's not true for NSManagedObject: it depends on NSManagedObjectContext. There might be ways to insert an object later, but it still needs to know its entity description.
Another assumption Decodable makes, is that it always creates objects, never updates existing ones. Together with the first assumption, this makes perfectly sense. Yet, since we already struggle with the first assumption, the second one makes things worse for NSManagedObject subclasses. They're usually representing an entry in a database (usually SQLite). And we don't want to continually insert objects and have duplicates. Instead we want to update our existing objects.

While there might be ways to work around all these issues, its probably too much hassle. There might be APIs that allow creating NSManagedObjects and instead of simply inserting them allowing them to update existing entries. There might be ways of making sure, that this also triggers the necessary changes so that existing (parent) contexts can be updated and eventually the new data can be displayed.
If you want to go down that road feel free! Once you succeed, be sure to let me know!

Meanwhile, let's look at another solution, that actually isn't even new: Data Transfer Objects - or in short DTOs!
Why don't we let the decoder create us a lightweight object which we can use to update our models? The downside of course is, that we have to write code for updating our models ourselves. But since Decodable anyways can't update models (only create them), that's something we'd likely have to do anyways. Also, with all the aforementioned issues, we might have anyways had to implement init(from:) ourselves, so we wouldn't have gotten much from the compiler anyways.

CoreDataDecodable

Alright, let's put that approach into some nice protocols:

protocol CoreDataDecodable: Decodable {
    associatedType DTO: Decodable
    
    @discardableResult
    static func findOrCreate(for dto: DTO, in context: NSManagedObjectContext) throws -> Self

    init(with dto: DTO, in context: NSManagedObjectContext) throws

    mutating func update(from dto: DTO) throws
}

We create an associated type for our DTO (that of course needs to be Decodable).
Next, we define a static func to allow us to search for an existing object for a given DTO inside a given NSManagedObjectContext. If we don't find one, we simply create one.
The init(with:in:) defines the initializer for creating object from DTOs in a given object context. And update(from:) is for updating an existing model with a given DTO.
You might notice, that our protocol still inherits from Decodable. Why so? Because then we still have the possibility to create objects directly from the decoder. But we don't want to methods to do that. We could create an extension, but as you might have noticed, our methods all require an NSManagedObjectContext. So if we implement init(from:) in an extension, we need to get a NSManagedObjectContext from somewhere.
To solve this problem, we create a globally accessible context for decoding purposes:

enum CoreDataDecodingError: Error {
    case missingContext(codingPath: [CodingKey])
}

extension NSManagedObjectContext {
    private static var _decodingContext: NSManagedObjectContext? 

    static func decodingContext(at codingPath: [CodingKey] = []) throws -> NSManagedObjectContext {
        if let context = _decodingContext { return context }
        throw CoreDataDecodingError.missingContext(codingPath: codingPath)
    }

    public final func asDecodingContext(do work: () -> ()) {
        NSManagedObjectContext._decodingContext = self
        defer { NSManagedObjectContext._decodingContext = nil }
        performAndWait { work() }
    }
}

To make it easier to debug issues, we also pass down the codingPath ([CodingKey]).

Note: If you need to decode from multiple threads (read queues), you cannot simply have one static property. You need to make sure, each thread (read queue) has its decoding context (e.g. using DispatchSpecificKey). This is why the static var is private making it an implementation detail of decodingContext(at:).

Now that we've solved this problem, we can implement init(from:) in an extension:

extension CoreDataDecodable {
    init(from decoder: Decoder) throws {
         try self.init(with: DTO(from: decoder), in: .decodingContext(at: decoder.codingPath))
    }
}

If our protocol is implemented by an NSManagedObject (which is probably almost always the case), we can also give a default implementation of the initializer:

extension CoreDataDecodable where Self: NSManagedObject {
    init(with dto: DTO, in context: NSManagedObjectContext) throws {
        self.init(context: context)
        try update(from: dto)
    }
}

This leaves us with only two methods to implement: findOrCreate(for:in:) and update(from:).

Example

Lets put that now into our example:

extension Person: CoreDataDecodable {
    struct DTO: Decodable {
        enum CodingKeys: String, CodingKey {
            case id
            case firstName = "first_name"
            case lastName = "last_name"
            case birthday, dogs
        }
        let id: Int32
        let firstName: String
        let lastName: String
        let birthday: Date

        let dogs: [Dog.DTO]
    }

    func update(from dto: DTO) throws {
        id = dto.id
        firstName = dto.firstName
        lastName = dto.lastName
        birthday = dto.birthday
        guard let moc = managedObjectContext else { return }
        dogs = try Set(dto.dogs.map { try Dog.findOrCreate(for: $0, in: moc) })
    }
}

extension Dog: CoreDataDecodable {
    struct DTO: Decodable {
        let id: Int32
        let name: String
    }

    func update(from dto: DTO) throws {
        id = dto.id
        name = dto.name
    }
}

Note: You might have noticed the findOrCreate(for:in:) is missing in the example. This is simply, because the logic is pretty straightforward. Using a NSFetchRequest, we search for an object that (in our example) has the given id and update it if found. If not found, we create a new one using the dto.

FFCoreData

Since I already have a helper framework for CoreData usages, I've put this into the framework as well.
In that implementation, the findOrCreate(for:in:) also has a default implementation, since the framework already has findOrCreate logic.

You can find the repository on GitHub.

You might have noticed that the domain of my blog changed from blog.ffried.net to ffried.codes. I've been waiting to use one of the new gTLDs and now I finally made the switch.
It's not that easy to setup a root domain with Ghost, however. My domain registrar didn't support CNAME entries for root domains so I had to use CloudFlare to get it to work. Not the perfect solution but it works for now.

tl;dr; Swift is awesome, but there are still many bugs for Apple to fix.

What is Swift?

Swift is the new programming language Apple introduced at the last WWDC in 2014 alongside iOS 8. When I first heard about it a the keynote I was quite sceptic. Why? Because I just thought that I understood Objective-C. A new language means getting used to new features of the language. But that feeling was gone right after I played around with Swift for the first time. Despite the ton of bugs it had during the beta period (I'm getting back to that later) I really liked Swift. Many things I didn't like with Objective-C (and I already really liked Objective-C compared to other languages) were changed to the better. Like for example initializers. I've always hated it to write the same boilerplate code in Objective-C to create an initializer (or override one):

- (instancetype)init {
  self = [super init];
  if (self) {
      // Do your stuff
  }
  return self;
}

In Swift it's a lot easier:

init() {
	// Maybe call super.init()
	// Do your stuff
}

They saved a lot of overhead by removing repeating code.
But that's only a part of it. Generics, more powerful structs and enums, (pseudo-)namespaces, frameworks for iOS, etc... All these things make Swift great. But is it ready to use in a customer project yet?

Swift or Objective-C?

A client of mine was planning a new version of an app I wrote three years ago. It was my second app and the code is horrible. So for me it was clear that although it was "only an update" for the client, I would completely rewrite it. Now the question was, in which language am I going to rewrite it, Swift or Objective-C?
At first I thought that Objective-C would be the better choice at it's a stable language and I wouldn't have to deal with some strange bugs. But in the end I thought that in the future Apple will certainly push Swift and I don't know if I'm going to get the opportunity to rewrite the app again. So I decided to go with Swift.

The adventure begins

The choice was made and thus I started with a blank Swift project. Now, I've already had a lot of code I reused in almost any project, but all of it was Objective-C. Also, the code was spread over many repositories on GitHub. I used to add submodules for all of them. But since the new version of the app will be iOS 8+ only, I thought that this would be the perfect time to use frameworks. Hence I created FFFoundation and FFUIKit. For another app I had already created FFCoreData. I copied Objective-C code I didn't want to rewrite in Swift just yet into these frameworks. Some classes I rewrote in Swift as they were the perfect possibility to use some great features of Swift. And that's what I did the first two weeks, getting together a solid foundation for Swift apps.

First difficulties

Once the three frameworks were ready enough to work with I continued working on the actual app. I created the CoreData model and let Xcode generate the corresponding NSManagedObject subclasses. But what's that? The app crashes when I tried to create a new instance of one of the NSManagedObject subclasses. I've done that a thousand times and I was 100% sure I didn't do anything different this time. The only thing different was Swift. Now, Google is your friend and so I found that due to the "pseudo-namespaces" Swift has, you need to either add @objc($CLASSNAME) before every NSManagedObject subclass or to prefix the class names with $PROJECTNAME. in the CoreData model. First I tried the latter, but that didn't really work out as Xcode then also added it to the filename and the class name the next time I generated the model classes. So I went with adding @objc($CLASSNAME) before the class definition which would basically be Xcode's job. Also, due to Swifts much better support of primitive types (and String being one of them) I chose to use primitive types for CoreData, too. And I was surprised that it worked without any problems.

Optionals - A love-hate relationship

The first big change you're going to have to deal with in Swift (aside from the general language differences) are optionals. In Objective-C there was nil and in Swift there's still nil, but they're used quite differently. While in Objective-C every pointer could be nil, in Swift nil is wrapped into an optional (which is actually a protocol). If a variable/constant isn't an optional it's gotta have a value when you first try to access it. And the compiler makes sure that it does by not allowing you to access the value before there is one. In Objective-C it also wouldn't matter if you send a message to nil. It actually was kind of a feature and I've used it thousand of times. I used to laugh when my workmate was doing Android development and ran into a NullPointerException. But then I suddenly stopped laughing when I ran into fatal error: unexpectedly found nil while unwrapping an Optional value, which is basically the NullPointerException of Swift. At first I hated Swift for that. Did I really have to make sure that every value has something in it now? I've already hated that in Java. But then I realized that it wasn't like that. In fact, my coding style changed. Instead of not caring about nil, I tried to get rid of it as soon as possible. So instead of passing optionals all the way through my app, I began to handle them as soon as possible. For example when decoding JSON I tried not to use optionals for my properties wherever possible (except of course if null is a valid value for a JSON key).
Swift also has "implicitly unwrapped optionals". If you define a variable or property with an !, Swift will automatically try to unwrap any optionals you assigned when you access it (and fail if it contains nil). But automatically in this case mostly means dangerous. Because you have no compile-time control over it. You can assign anything to it. You can even assign nil and it'll work perfectly fine as long as you don't try to access it again. Hence I tried to use ! as little as possible (or only after checking that the optional doesn't contain nil). Instead I used the if let feature to safely unwrap optionals.
By now, there are still cases where I hate optionals. But mostly only because they come from "old" Objective-C frameworks from Apple which aren't really made for Swift (more about that in the next paragraph). In most cases I love being sure that there are actual values in my variables / properties.

Apple's Frameworks

While Swift is a great languages, it is currently held back from becoming even better by Apple's own frameworks. Sure, Apple is doing their best to make their frameworks compatible with Swift, but they're still in Objective-C. With Swift 1.2 they also released an update for Objective-C containg features to help developers making their Objective-C code integrate better with Swift. But it's still Objective-C code and you can't get all the cool features of Swift with Objective-C. If we could, Swift would be useless. What we need are Swift versions of Apple's frameworks. For example you can't have lightweight view controllers today because UIViewController inherits indirectly from NSObject. So every view controller you have will still use the old msg_send overhead for example. Also, the prefixes should be removed. UILabel should become Label, NSFetchedResultsController -> FetchedResultsController and so on.

Apple's Tools

Xcode is buggy as hell when it comes to Swift. I'd even go as far and say that Swift is way better than Xcode. Dear Apple, if you want us developers to use Swift, then please release an IDE with which we can actuall use Swift.

What's missing:

• Refactoring: Seriously Apple? Every developer refactors his code. Even if it's something simple as renaming a class or a property. That shouldn't be so hard.
• Interface Builder: Okay, you can create outlets to Swift classes and they'll be properly assigned at runtime. But why do we still have to use ! for outlets? Xcode knows if an outlet is connected and the compiler could just fail if there's no connection. Also, I want "Generic ViewControllers". For example being able to create a SearchViewController<T: Equatable>: TableViewController would be nice. Of course you can create such a view controller in code, but you can't use outlets. Why can't you just specify the type in the interface builder?
• SourceKitService: I don't care what it's used for. I don't even really care why it crashes. Just make it work again, Apple. Having it crash all the time and then having to wait until it's back is just nuts! (Update: With the latest versions of Xcode this seems to have improved a lot)

I'm sure there's more to add to this list...

Strange bugs, even stranger fixes

When you work with Swift you won't get around strange bugs, yet. You might just end up being unable to compile the project but not getting any help from Xcode why exactly it can't build. You will then go back step by step commenting out what you've done until you're back at a stage where your project compiles again. Then you reverse the process until you find the responsible line of code. Sometimes it's a reasonable error, which is just not reported back to the developer correctly. But most of the time you have such bugs, you'll most likely never understand why exactly it doesn't work. Let me give you an example.
This works:

override func preferredStatusBarStyle() -> UIStatusBarStyle {
	return UIStatusBarStyle.Default
}

But this does not (notice the missing enum name in the return):

override func preferredStatusBarStyle() -> UIStatusBarStyle {
	return .Default
}

Only in one particular view controller of course.

Another example:
This compiles:

import FFFoundation
import MapKit

class MapViewController: ViewController {
}

This does not:

import FFFoundation
import FFUIKit
import MapKit

class MapViewController: ViewController {
}

First you'll say: There's an error in FFUIKit! Sorry to disappoint you, but FFUIKit builds just fine and is used in many different places in the app. The truth is I haven't used anything from FFUIKit in that class hence it didn't compile. When I removed the import it compiled just fine. Once I used a function from FFUIKit and imported it again, it also compiled just fine.
If there was a rule that you shouldn't import frameworks you don't use, I'd be okay with that (only if Xcode gives an appropriate error message of course). But just having the compiler crashing around doesn't help much.

So Swift is crap?

Not at all! Although I wrote a lot about Swift's weaknesses, I still love Swift. And it's totally possible to write apps, even customer apps, in Swift. You'll have to deal with some difficulties, but it's nothing that would stop you completely. Also, all of the issues mentioned above aren't permanent. They're bugs and bugs can and most likely will be fixed by Apple. With Swift 1.2 they've already fixed many bugs. I haven't written any code at the time but I'm sure Objective-C wasn't perfect in the beginning, either.
The customer app I wrote is now finished and I'm still happy with my choice to write it in Swift. I've even updated everything to Swift 1.2 in the meantime and it's really another great update for the language as well as Xcode.

It's quite some time ago since I've last written something here. Sorry about that. I'm currenlty in the middle of moving from Switzerland to Germany which gives me quite a lot to do. But I plan to write some more posts again.
For example I'm gonna write about my experiences with Swift while writing a clients app with Swift.
Also, I will most likely write something about the Apple Watch as soon as I've got one.

So, expect more posts in the coming months.

tl;dr; There is a neat little library for handling NSUserDefaults over here.

For a project I've recently worked on, I've created a settings class which mirrors its properties into NSUserDefaults. I did so by overriding the getters and setters and directly forwarding the calls to NSUserDefaults. After adding one or the other property I've discovered that it's cumbersome to add new properties. You need to define a new static key variable for the NSUserDefaults, you need to override getter and setter and eventually add support for Key-Value Observing. So I had the idea write a little library for all this. Objective-C makes heavy use of those @dynamic properties in CoreData. Couchbase's iOS library does the same for their model objects. So I figured there is a way without using any private SDK to achieve the same.
-> The idea for FFUserDefaults was born.

What should it do

I haven't defined many requirements. What I wanted to create is a class which can be subclassed and which dynamically implements @dynamic properties at runtime. Also, those properties must be Key-Value observable. And one should be able to use properties of a primitive data type as well.

Dynamically resolve methods

First step was to find out how to resolve methods dynamically. This Question on StackOverflow showed me the basic approach. I then found this Site in Apple's documentation which shows how the attributes of a property look like and what different types there are. So the next step was to put the pieces together. I've created a model class which represents a property and the FFUserDefaults class which reads all its properties on demand. At the beginning I've only created IMPs for object properties. I've thought that I could get the primitive ones for free by dynamically creating the IMPs for them. But I was wrong and I ended up writing getter and setter for each and every primtive data type I wanted to support:

KVO support

One of the features I wanted to have in FFUserDefaults is to support KVO. The properties should be observable. For that I need to observe NSUserDefaults which isn't really a problem. The first tests without any observing options specified ran through without any problems. But once I created the example project, I added NSKeyValueObservingOptionNew and suddenly I got crashes caused by valueForUndefinedKey:. After a bit of playing around it turned out that I needed to implement valueForKey: and setValue:forKey: as well. And I couldn't even just call my methods as the valueForKey: and setValue:forKey: methods both have id as argument and the getters and setters need the primitive types. My next thought was to get rid of all my getters and setters and just use valueForKey: and setValue:forKey:. But then I couldn't use my @dynamic properties anymore.

KVO seems broken

While adding KVO support I've noticed that my tests eventually wouldn't run through. In fact, they just stopped without any reason. No crash, no (visible) endless loops, they just stopped. By setting breakpoints I've noticed that it seemed to have something to do with calling valueForKey: on an array (which then should call valueForKey: on all its objects and return an array with those values). I was able to step right to that call but no further. I couldn't even step into the method. The moment this specific line was executed nothing happened anymore. Now, the funny part about this was that this only happened when the KVO test was run. All other tests were just fine. With a bit of trial and error debugging I found out, that calling valueForKey: on an array while setting an observed property ends up in nirvana. I haven't tried calling valueForKey: on objects other than arrays. It seems like a bug to me but it would need further investigation before filing a radar with Apple.

It eventually works

In the end I got everything working. Dynamically adding methods to a class is something completely new to me. I haven't done this before. And I'm sure there's a lot to improve in FFUserDefaults. But it's working and that's what is important in the first place.

You can find the library on Github over here. Any feedback and/or pull requests are welcome!

Update 8/11/14:
This library isn't yet ported to Swift as I'm not aware of a way to achieve the same in Swift. But as soon as this is possible I'll be working on a Swift version.

About a month ago the App.net State of the Union blog post was published by the App.net crew. Like for many other App.net users it was surprising and sad at the same time for me, too. Surprising, because App.net just worked, without any severe problems so there was no reason to worry about it. Sad, because although it does continue to run it is a sign that something did not work out as planned.
There were many reactions to this post. Some of them really great. But others just made me angry. Especially some tweets from people who admittedly have an App.net account, but either just crossposted their tweets or aren't active on App.net at all. They think they know the community. And they pretend they knew right from the beginning that this would happen sooner or later.

But actually, for all the "normal" users nothing really changed. The changes mainly affect the App.net developers and the App.net crew itself. The DIP (Developer Incentive Program) was discontinued to reduce costs. I have an app registered for the DIP myself but I don't really miss the DIP. I do see the advantages of the DIP of course. The DIP should make it interesting for developers to create apps for App.net. But at the end of the day I don't need the DIP to develop for App.net. I'm working on Filapp together with Dennis (@thedan84) and we will continue to work on it! When I've received the email from App.net about the future of the DIP, I had the idea to donate the remaining earnings. After talking with Dennis about it and we agreed to make the offer and it was thankfully accepted by the ADN crew. It's not a big amount but we figured that App.net could make better use of the money right now.
I've seen other efforts to support App.net like gifting normal accounts to other people. For me this is what's so special about this community. If there's a problem there are always people willing to help out if they can.

Personally, I'll stay on App.net until the servers shut down. And not only will I stay, but also continue to develop for App.net.

[self writeCode:YES];

What is this?

I've decided to start a new experiment: a blog.
Why? I love reading blog posts of others, especially about development. But I've never really taken part in that "community". Now, this blog's aim is to change that.
You'll probably ask "What will this guy write here?". Well, here's the answer: As you might guess when looking at the banner and the logo, this blog will for sure contain many posts about development, mainly Objective-C development. But not only. There will be posts about things which don't have something to do with development in the first place.

Who is this guy?

I'm a young developer who mainly develops apps for iOS and OS X. This month I'm finishing my training as a developer which I've started in 2010. The first three years were school years and this last year I'm working as an intern in a mobile development company. I'm German but I'm currently living in Switzerland (and yes, English isn't my first language, so if you find mistakes please let me know so I can fix them. Thanks! 😊).

You can find me on App.net and Twitter.
If you want to see some code I've written here's my GitHub Profile.