When building the registration section of our apps, many of us will have heard that seductive whisper:

"These fields are so alike. You should generalise them. Let's build a form-engine"

It makes sense. The fields are alike. Why shouldn't you build a form-engine? You'd be praised by the rest of the team. You'd be easing the burden of any future developers. You'd be adding value 😇.

So you go away and build that form-engine that lets you assemble form pages in minutes. No sooner have you pushed this masterpiece in declarative programming to main than you're pulled onto another feature. Your colleague picks up registration and, moments after opening their Xcode, you hear their cry of frustration: trying to customise the password field, they discover your form-engine has buried the actual SecureField inside a view shared by five screens and surrounded by a mountain of if statements. An isolated, simple change has become a risky, regression-prone slog.

As you peek round from behind the coffee machine, you hear that whisper again, but this time it isn't seductive. This time, you hear the malice in it as it mocks you for falling into the form-engine trap 😈.

The instinct behind a form-engine is good: capture a common pattern in a reusable component. Where form-engines fall short is in being too ambitious. They go too far and end up creating a quasi-Domain-Specific Language (DSL) that doesn't complement SwiftUI — it tries to replace it. Creating your own language is hard. Creating one accidentally is harder. Maintaining it? Hardest of all. Especially when the creator doesn't control the ideas that form the bedrock of this DSL masquerading as a form-engine.

One day, a designer comes along - unaware that this form-engine exists - and asks for a small behaviour change on one screen: a password field should shake on error. The form-engine creator now must either add more conditionals to the abstraction or split it. Conditional wins at first, and they find a way to trickle that shake behaviour down to the input field. Then the designer changes the border colour. Then the font. Then adds a description label. And so on.

Before the form-engine creator knows it, that trickle has become a flood. The complexity has skyrocketed, and now screens are having to opt out of functionality that, without the form-engine, they wouldn't even know about. All those conditionals need tests, and each new conditional only makes testing harder - ending up with more risk, more work, and a codebase that's harder to teach to new developers.

You might be thinking:

"The form-engine creator should just create a new view type to decouple that generalisation"

But that's where they started before the abstraction spiral began. The issue isn't abstraction - it's the ambition to build on ideas that the developer doesn't control. What looks like "the same field" to the developer is "five different concepts" to the designer.

This post will explore how we can get up to the edge of a form-engine and reap the benefits that abstraction can give us without falling into that deep water. Each component we build will handle one concern. Each screen will then be free to use them as needed without becoming overly coupled to any other screen. We'll focus on a single example - a view that handles text entry and validation - but the pattern shown will work for other types of field too.

This post will gradually build up to a working example. But I get it, that whisper is demanding, so if you are unable to ignore it, then head on over to the completed example and take a look at ValidationInputField, Validator and ValidationViewModel to see how things end up.

Looking at What We Need to Build

Before jumping into building each component, let's take a moment to look at the overall architecture we're going to put in place and how each component fits into it.

• RegistrationView - the view that controls the screen we are going to add our input field to (we will only cover this briefly to see how ValidationInputField can be used).
• ValidationInputField - our input field view. It will allow us to share common functionality across different screens.
• ValidationViewModel- orchestrates validation, debouncing, and state exposure to the UI. There are a number of ways we could connect the outcome of validating to the view. In the example project for this post, MVVM is the architecture chosen, with the ViewModel acting as that connector. But as you will see, MVVM isn't intrinsic to this solution and you should be able to swap in whatever controller you want here.
• Debouncer - helps smooth out the validation feedback by waiting for the user to stop typing before triggering validation.
• Validator - performs validation on the user's input.
• ValidationState - represents the outcome of the validation in a declarative manner that the view can consume.

Don't worry if that doesn't all make sense yet; we will look into each component in greater depth below.

Now that we know where we are going, let's start by implementing password validation as a concrete solution, then we'll take that solution and abstract it to validate any input.

This project is built using Swift 6.2 with the Default Actor Isolation build setting set to MainActor. If you need to support an earlier version of Swift or don't want to set Default Actor Isolation to MainActor, then add @MainActor where needed.

Defining What Makes a Valid Password

Before we can validate anything, we need a validator. That validator will answer just one question:

"Is this input acceptable?"

At its most simple, the answer is either true or false.

Let's look at the steps involved in validating a password. First, we need to define the rules for what a valid password is:

1. Must have at least 8 characters.
2. Must have at most 24 characters.
3. Must have at least one lowercase character.
4. Must have at least one uppercase character.
5. Must have at least one number.

With this definition, let's implement a validator that enforces those rules:

struct PasswordValidator {

    // 1
    func validate(_ value: String) -> Bool {

        // 2

        // Must have at least 8 characters
        guard value.count >= 8 else {
            return false
        }

        // Must have at most 24 characters
        guard value.count <= 24 else {
            return false
        }

        // Must have at least one lowercase character
        guard value.contains(where: { $0.isLowercase }) else {
            return false
        }

        // Must have at least one uppercase character
        guard value.contains(where: { $0.isUppercase }) else {
            return false
        }

        // Must have at least one number
        guard value.contains(where: { $0.isNumber }) else {
            return false
        }

        return true
    }
}

Here's what we did:

1. PasswordValidator has one method validate(_:) -> Bool that takes a String argument and returns true if that argument constitutes a valid password or false for an invalid password.
2. value is checked against each rule. If a check fails, false is immediately returned. If all checks pass, true is returned.

In this post, all validation happens locally. If you need validation that involves asynchronous work, you could make this into an asynchronous method, e.g. func validate(_ value: String) async -> Bool.

PasswordValidator in its present form is a perfectly good validator, but notice how we lose details during the validation process - we know on which rule validation fails, but that information is lost by returning the same value for each validation failure. Let's tweak PasswordValidator to preserve that information by having PasswordValidator throw an error rather than return a boolean. Before PasswordValidator can be changed, we first need to define what errors can be thrown:

enum PasswordValidationError: Error {
    case tooShort
    case tooLong
    case missingLowercase
    case missingUppercase
    case missingNumber
}

PasswordValidationError is an enum that conforms to Error. Each case corresponds to one of the validation rules defined above. Let's update PasswordValidator to throw a PasswordValidationError error when the input is invalid:

struct PasswordValidator {

    // 1
    func validate(_ value: String) throws {

        // Must have at least 8 characters
        guard value.count >= 8 else {
            throw .tooShort
        }

        // Must have at most 24 characters
        guard value.count <= 24 else {
            throw .tooLong
        }

        // Must have at least one lowercase character
        guard value.contains(where: { $0.isLowercase }) else {
            throw .missingLowercase
        }

        // Must have at least one uppercase character
        guard value.contains(where: { $0.isUppercase }) else {
            throw .missingUppercase
        }

        // Must have at least one number
        guard value.contains(where: { $0.isNumber }) else {
            throw .missingNumber
        }
    }
}

Here's what we did:

1. Instead of using a Bool as the return type, we instead either throw an error for invalid input or return nothing at all for valid input. By throwing a unique error for each failure, any type making use of this validator will be able to tailor their response to what exactly went wrong.

PasswordValidator now throws specific errors for each rule it enforces. Which is good, but we can take it a step further. At present even though PasswordValidator only throws PasswordValidationError, the compiler requires us to handle unrelated error types as well because as far as the compiler is concerned, that plain throws that we added to validate(_:) effectively means "this method can throw any Error" so we need to handle any Error in our do-catch blocks. As validate(_:) will always throw PasswordValidationError, this error handling for any Error is effectively dead/unreachable code. We can give the compiler more information about the errors validate(_:) can throw, and so avoid that dead code by using: typed throws.

Typed throws allows us to declare exactly which error type a method throws. By eliminating error erasure, our validation chain can be fully type-safe. To take advantage of this, let's update PasswordValidator to specify that it only throws PasswordValidationError errors:

struct PasswordValidator {

    // 1
    func validate(_ value: String) throws(PasswordValidationError) {
        // Omitted method body as it is unchanged
    }
}

Here's what we did:

1. As validate(_:) throws already only ever throws PasswordValidationError, we only need to change the signature to use typed throws.

Notice what PasswordValidator doesn't know: it doesn't know about views, debouncing, or user feedback. It knows password rules - that's it. This single responsibility makes it reusable across any context that needs password validation: registration, settings, password reset, wherever. This is the first piece of our separation of concerns: validators validate, nothing more.

With password validation implemented, let's figure out how to share the outcome of that validation in a way that fits the needs of a declarative UI.

Transforming Outcomes

SwiftUI views excel when they react to state rather than handle control flow. Currently, PasswordValidator throws errors, which would force any consuming view into imperative error handling with do-catch blocks. Mixing imperative code in declarative spaces creates awkward code. Instead, we can introduce an enum for representing validation outcomes as observable state that a SwiftUI view can render declaratively. This enum will act as a clean boundary between the two styles:

// 1
enum ValidationState: Equatable {
    case valid
    case invalid(String)
}

Here's what we did:

1. ValidationState has two cases: .valid and .invalid, representing the successful and unsuccessful validation outcomes. .invalid has an associated value of type String that will be used to hold a user-friendly message for why their input failed validation.

Now that we can represent validation outcomes declaratively, let's look at how to transform the output of PasswordValidator into something our custom input field can consume.

Connecting the Validator and Input Field

We now have two pieces that don't quite fit together: PasswordValidator throws errors, but we've said that our custom input field needs ValidationState to render. We need something to bridge this gap.

That's the role of PasswordViewModel. It acts as the coordinator between these components, handling three key responsibilities:

1. Publishing - enabling PasswordInputField to react to state changes.
2. Triggering validation - when the user's input changes
3. Bridging the gap between our validator and the UI - catching and transforming validation errors into ValidationState

SwiftUI views automatically re-render when the state they observe changes. To enable this reactive behaviour, we need to mark PasswordViewModel with @Observable:

@Observable
final class PasswordViewModel {
}

@Observable automatically makes all properties observable, allowing SwiftUI to track changes and update the UI accordingly.

If your app needs to support pre-iOS 17, you can substitute in Combine to get the reactive behaviour provided by @Observable.

With the reactive foundation in place, we now need to decide what state to hold:

1. value - the current input.
2. validationState - the validation outcome.

Let's start with value:

@Observable
final class PasswordViewModel {
   // 1
   var value: String

   // 2
   init(initialValue: String) {
      self.value = initialValue
   }
}

Here's what we did:

1. Created a value property that will be bound to its associated view.
2. Implemented the initialiser to accept the initial value of value.

Before adding validationState, we need to decide its initial value. The current choices of valid and invalid don't feel right, as neither represents the nature of validation when no validation has occurred on value. We are missing a third ValidationState case - .unchanged to represent this unvalidated state. Let's add that:

enum ValidationState: Equatable {
    case unchanged
    case valid
    case invalid(String)
}

With .unchanged, we can now add a property to PasswordViewModel that can be used to drive any view changes based on the outcome of validation:

@Observable
final class PasswordViewModel {
    // Omitted unchanged properties

    // 1
    private(set) var validationState: ValidationState = .unchanged

    // Omitted unchanged methods
}

Here's what we did:

1. Added a validationState property to share the outcome of validation. Unlike value, we don't want anything external to PasswordViewModel to be able to set this property, so we mark it as private(set) to allow read but not write access.

Before we can trigger validation, we need to get from the PasswordValidationError that validator outputs to a String that the .invalid case of ValidationState expects. There are many ways to do this; here, we are going to extend PasswordValidationError to conform to LocalizedError and use errorDescription to do the conversion:

// 1
extension PasswordValidationError: LocalizedError {

    // 2
    var errorDescription: String? {
        switch self {
        case .tooShort:
            return "Password must be at least 8 characters long."
        case .tooLong:
            return "Password must be at most 24 characters long."
        case .missingLowercase:
            return "Password must contain at least one lowercase letter."
        case .missingUppercase:
            return "Password must contain at least one uppercase letter."
        case .missingNumber:
            return "Password must contain at least one number."
        }
    }
}

Here's what we did:

1. Created an extension that conforms PasswordValidationError to LocalizedError.
2. Implemented the property errorDescription of LocalizedError to provide a user-friendly description for each error.

LocalizedError defines several other properties; we don't need to make use of them here, so I've not implemented them.

Now that we have validationState and a way to convert PasswordValidationError to String, we have everything we need to validate:

@Observable
final class PasswordViewModel {
    // Omitted unchanged property

    // 1
    private let validator: PasswordValidator

    init(initialValue: String,
         validator: PasswordValidator) {
        // Omitted unchanged code

        self.validator = validator
    }

    // 2
    func validate(_ currentValue: String) {
        do {
            try validator.validate(currentValue)

            // 3
            validationState = .valid
        } catch {
          // 4
          let errorMessage = error.errorDescription ?? "Unknown error"

          validationState = .invalid(errorMessage)
        }
    }
}

Here's what we did:

1. Added a private validator property to hold the Validator instance that will be used during validation.
2. Implemented validate(_:) to trigger validation. As validate(_:) on Validator can throw, we wrap that call in a do-catch block.
3. When validation succeeds, we set validationState to .valid.
4. As PasswordValidationError now conforms to LocalizedError, we call errorDescription to get the user-friendly message we defined. As this is an optional property, we need to provide a default error message in case nil is returned. We set validationState to .invalid.

PasswordViewModel can validate, but we aren't triggering that validation. There are two options:

1. Property side-effect - trigger validation automatically whenever value changes.
2. Explicit method call - call validate(_:) from onChange(of:initial:_:) in the view.

We are going with the property side-effect approach here. This approach keeps decision-making out of the view layer, and while property side-effects are often discouraged because they hide behaviour, this case is different: we want validationState to always be in sync with value. These aren't separate concerns - they're two facets of the same state. By triggering validation in didSet, we maintain this invariant automatically rather than relying on the view to remember to call validate(_:). This keeps coordination logic where it belongs - in the view model, not scattered across views.

@Observable
final class PasswordViewModel {
    var value: String {
        // 1
        didSet {
            validate(value)
        }
    }

    // 2
    private func validate(_ currentValue: String) {
        // Omitted unchanged code
    }
}

Here's what we did:

1. Updated value so that any change will trigger validation to occur.
2. Updated validate(_:) to now be private.

While this is all that is required for our view model to trigger validation and convert the outcome of that validation into something PasswordInputField could consume, there is a subtle behavioural quirk: reverting value back to its initial state won't reset validationState back to .unchanged. This means that the same value might sometimes be treated as an error and sometimes not. Let's correct this:

@Observable
final class PasswordViewModel{
    var value: String {
        didSet {
            // 1
            guard value != initialValue else {
                validationState = .unchanged

                return
            }

            // Omitted unchanged code
        }
    }

    // Omitted other properties

    // 2
    private let initialValue: String

    // 3
    init(initialValue: String,
         validator: PasswordValidator) {
        // Omitted unchanged code

        self.initialValue = initialValue
    }

    // Omitted other methods
}

Here's what we did:

1. To avoid validating the initial value of value, we check that the current value is different from initialValue - we don't want to treat the initial value of value as an error even if it actually is.
2. Added an initialValue property to allow the current value of value to be compared against its initial value.
3. Updated the initialiser to set the initialValue property.

PasswordViewModel is now complete - it coordinates validation without knowing about text fields, and it manages state without knowing validation rules. Each piece stays focused. Let's build the next piece: PasswordInputField.

Creating a Password Input Field

Before we can begin writing a validating input field, we first need to know what our form field should look like:

Across the three different scenarios shown in the mock-up, the input view consists of:

1. Title Label - describes what information the user needs to enter.
2. Input Field - allows the user to enter the requested information.
3. Error Label - describes how the entered information is invalid.

Now we know what to build - we need to think about how we should design its API.

When creating reusable views, one goal is to give that view a clean, focused initialiser. SwiftUI's own SecureField demonstrates this beautifully: its initialiser accepts only the essential state needed to render the field, with styling and behaviour added through view modifiers.

Let's follow that pattern.

Looking at the mock-up, the title and input field are always present, so we want to pass the state used to set these via the initialiser, but the error label is conditional - we'll add that state via a modifier later. For now, the initialiser needs:

struct PasswordInputField: View {
    private let placeholder: String
    private let title: String
    @Binding private var value: String

    // 1
    init(title: String,
         placeholder: String,
         value: Binding) {
        self.placeholder = placeholder
        self.title = title
        self._value = value
    }

    // 2
    private var titleView: some View {
        Text(title)
            .font(.subheadline)
            .fontWeight(.medium)
            .foregroundColor(.primary)
    }

    // 3
    private var inputView: some View {
        SecureField(placeholder,
                    text: $value)
        .padding(12)
        .background(Color(.systemGray6))
        .cornerRadius(8)
    }

    // 4
    var body: some View {
        VStack(alignment: .leading,
               spacing: 0) {
            titleView
                .padding(.bottom, 6)

            inputView
                .padding(.bottom, 12)
        }
    }
}

Here's what we did:

1. PasswordInputField's initialiser takes three arguments: title (text for the title label), placeholder (text shown when empty), and value (stores user input). The first two are simple String values that PasswordInputField only reads. But value is different - the input field needs to both read the current value and write changes back. That's what Binding provides: a two-way connection that lets changes flow from the text field back to the view model, ensuring the user's input isn't lost.
2. Implemented the title label as the titleView property.
3. Implemented the input field as the inputView property.
4. Arranged the subviews in a vertical stack within body to match the mock-up.

Let's add our PasswordInputField to RegistrationView to capture the user's password:

struct RegistrationView: View {
    @Bindable var viewModel = PasswordViewModel(initialValue: "",
                                                validator: PasswordValidator())

    var body: some View {
        passwordFieldView
    }

    private var passwordFieldView: some View {
        PasswordInputField(title: "Enter your password",
                          placeholder: "Enter your password",
                          value: $viewModel.value)
        .textContentType(.newPassword)
    }
}

While PasswordInputField looks good, it's purely presentational - no validation yet. Let's layer validation in by connecting it to our view model's state.

SwiftUI uses environment values for configuration that should cascade down the view hierarchy - think .font(), .foregroundColor(), or .disabled(). Validation state fits this pattern perfectly: it's set at the parent level and consumed by nested input fields. Let's add validationState to the environment:

// 1
extension EnvironmentValues {
    @Entry var validationState: ValidationState = .unchanged
}

// 2
extension View {
    func validationState(_ value: ValidationState) -> some View {
        environment(\.validationState, value)
    }
}

Here's what we did:

1. Extended EnvironmentValues with a validationState property, making it available throughout the view hierarchy like built-in environment values.
2. Created a convenience modifier that lets us write .validationState(value) instead of the more verbose .environment(\.validationState, value).

With these changes, we can now pass validation state into the field:

struct RegistrationView: View {
    // Omitted unchanged code

    private var passwordFieldView: some View {
        PasswordInputField(title: "Enter your password",
                           placeholder: "Enter your password",
                           value: $viewModel.value)
        .textContentType(.newPassword)

        // 1
        .validationState(viewModel.validationState)
    }
}

Here's what we did:

1. Passed the validationState from the view model into the view.

Before updating PasswordInputField to respond to validationState changes, let's add a convenience property to ValidationState to make writing that code easier:

enum ValidationState: Equatable {
    // Omitted cases

    // 1
    var isInvalid: Bool {
        guard case .invalid(_) = self else {
            return false
        }

        return true
    }
}

Here's what we did:

1. Added an isInvalid helper to check if the current state is .invalid.

Armed with this convenience property, let's make PasswordInputField respond to validation changes:

struct PasswordInputField: View {
    // 1
    @Environment(\.validationState) private var validationState

    // Omitted unchanged methods and properties

    // 2
    @ViewBuilder
    private var validatedView: some View {
        if case let .invalid(errorMessage) = validationState {
            HStack(alignment: .top, spacing: 4) {
                Image(systemName: "exclamationmark.circle.fill")
                    .font(.caption)
                Text(errorMessage)
                    .font(.caption)
            }
            .foregroundColor(.red)
        }
    }

    @ViewBuilder
    private var inputView: some View {
        // Omitted unchanged code

        // 3
        .overlay(
            RoundedRectangle(cornerRadius: 8)
                .stroke(validationState.isInvalid ? Color.red : Color.clear, lineWidth: 2)
        )
    }

    var body: some View {
        VStack(alignment: .leading,
               spacing: 0) {
            titleView
                .padding(.bottom, 6)

            inputView
                .padding(.bottom, 12)

            // 4
            validatedView
        }
    }
}

Here's what we did:

1. Added a validationState property to read from SwiftUI's environment.
2. Created validatedView to display error messages when validation fails. This view only appears when the state is .invalid.
3. Added a red border to the input field when validationState is .invalid; otherwise, the border is clear.
4. Integrated validatedView into the layout below the existing views.

Before moving on, let's pause on an important design principle. PasswordInputField contains conditional logic - checking validation state, showing errors conditionally. So what separates healthy abstraction from the form-engine trap? The answer lies in shared behaviour vs screen-specific dialects.

The conditionals here - showing an error message, adding a red border - are shared: every screen using PasswordInputField gets the same error display. The field responds consistently to whatever ValidationState it receives. That's healthy abstraction.

The form-engine trap happens when conditionals create unshared behaviour: "if this is the registration screen, shake on error; if it's the settings screen, don't shake; if it's the profile screen, show a tooltip instead." Now we'd have three different dialects of the same component, each screen opting in or out of features. At that point, we'd be better off creating separate components rather than bloating one component with conditionals that only some screens use.

The line is simple: if conditional logic applies to all uses of a component, it belongs in the component. If it's specific to one or two screens, those screens should either decorate this component, use a different component or handle it themselves.

This distinction will guide us as we come to generalise this solution.

If you run the app and try entering an invalid password, you'll see the error message appear below the field as expected.

But you'll also notice something off: the error appears immediately as you start typing. That's not a great user experience 😔.

To smooth this out, we need a helper that waits for a pause in typing before triggering validation. That's where Debouncer comes in.

Debouncing Validation

Looking back at our solution, PasswordViewModel triggers validation in didSet whenever value changes - which means every keystroke. That's what causes the immediate error feedback we just saw.

We need to insert a delay between user-typed and validation-runs. This is debouncing: waiting for a pause in activity before taking action. Instead of validating on every keystroke, we'll wait until the user stops typing for a moment.

SwiftUI doesn't provide a debouncer out of the box, so we'll implement a simple version using Task:

final class Debouncer {

    // 1
    private let delay: Duration

    // 2
    private var task: Task?

    init(delay: Duration) {
        self.delay = delay
    }

    // 3
    func submit(_ action: @escaping () async -> Void) {
        task?.cancel()
        task = Task {
            try? await Task.sleep(for: delay)

            // 4
            guard !Task.isCancelled else {
                return
            }

            await action()
        }
    }
}

Here's what we did:

1. Added a delay property to store how long operations wait before executing.
2. Added a task property to store the currently delayed operation and enable cancellation.
3. Implemented submit(_:) to handle incoming operations. Each call cancels any existing task (preventing stale operations), waits for the specified delay, and then executes the operation - this "cancel and restart" behaviour is what makes debouncing work.
4. Added a cancellation check before executing the operation. Task cancellation in Swift is cooperative, so merely calling cancel() won't automatically stop its execution - we must explicitly check isCancelled to respect the cancellation.

Now that we have Debouncer, let's make use of it:

final class PasswordViewModel {
    var value: String {
        didSet {
            // Omitted unchanged code

            // 1
            debouncer.submit {
                self.validate(self.value)
            }
        }
    }

    // 2
    private let debouncer: Debouncer

    // 3
    init(initialValue: String,
         validator: PasswordValidator,
         debouncer: Debouncer = Debouncer(delay: .milliseconds(500))) {
        // Omitted unchanged code

        self.debouncer = debouncer
    }

    // Omitted other methods
}

Here's what we did:

1. Replaced immediate validation with debouncer.submit(_:) to delay validation until typing pauses.
2. Added a debouncer property to manage delayed validation execution.
3. Defaulted the debouncer parameter to a 500-millisecond delay for better user experience out of the box.

Notice how debouncing stays isolated within the view model, just like validation. PasswordInputField doesn't know anything about timing - it simply displays whatever validationState it receives. PasswordValidator doesn't know anything about timing - it receives input to validate. This preserves unidirectional data flow: the view model coordinates timing, the validator validates, and the view renders. This is a simple illustration of the separation of concerns that is driving our solution.

An alternative to writing a custom debouncer would have been to use the debouncer provided by the AsyncAlgorithms library. Using the debouncer from AsyncAlgorithms would involve using an AsyncChannel to drive the validation. I haven't used it here as doing so would distract from the overall theme of the post (that overabstracting is dangerous), but if you have a more complex use case, it would be worthwhile to investigate that approach.

Now, if you run the app, validation only happens once the user has paused long enough to show they mean it.

Generalising to Any Input

So far, we have a working solution for validating passwords - let's make it more reusable by generalising it to validate any input. To do this, we need to make three components generic:

1. The validator - currently tied to String input and PasswordValidationError.
2. The view model - currently tied to PasswordValidator.
3. The input field - currently tied to secure text entry.

Generalising the Validator

We'll define a protocol that any validator can conform to - whether validating passwords, emails, phone numbers, or anything else:

//1
protocol Validator {

    // 2
    associatedtype Value: Equatable

    // 3
    associatedtype ValidationError: Error

    // 4
    func validate(_ value: Value) throws(ValidationError)
}

Here's what we did:

1. Validator is a protocol, with two associated types, that all concrete validators will conform to.
2. Added Value as an associated type, allowing any conforming concrete validator to validate any input.
3. Added ValidationError as an associated type constrained to Error, allowing any conforming concrete validator to define its own specific error type.
4. validate(_:) makes use of the two associated types.

Value doesn't technically need to conform to Equatable, but allowing equality comparisons is so common in validation that having this constraint makes working with a Validator instance easier.

Now that we have a way of expressing any validator, we need to conform PasswordValidator to it:

struct PasswordValidator: Validator {
    // Omitted unchanged code
}

The only change needed is to conform PasswordValidator to Validator; the Swift compiler has enough contextual information to infer that Value is String and ValidationError is PasswordValidationError, so we don't need to define them explicitly.

Generalising the View Model

With the Validator protocol in place, we can make PasswordViewModel generic so it works with any validator:

@Observable

// 1
final class ValidationViewModel {

    // 2
    var value: V.Value {
        didSet {
            guard value != initialValue else {
                validationState = .unchanged

                return
            }

            debouncer.submit {
                self.validate(self.value)
            }
        }
    }

    private(set) var validationState: ValidationState = .unchanged

    // 3
    private let validator: V

    // 4
    private let errorMapper: ((V.ValidationError) -> (String))
    private let debouncer: Debouncer

    // 5
    private let initialValue: V.Value

    // MARK: - Init

    init(initialValue: V.Value,
         validator: V,
         errorMapper: @escaping ((V.ValidationError) -> (String)),
         debouncer: Debouncer = Debouncer(delay: .milliseconds(500))) {
        self.value = initialValue
        self.initialValue = initialValue
        self.validator = validator
        self.errorMapper = errorMapper
        self.debouncer = debouncer
    }

    // MARK: - Validate

    // 6
    private func validate(_ currentValue: V.Value) {
        do {
            try validator.validate(currentValue)

            validationState = .valid
        } catch {
            // 7
            let errorMessage = errorMapper(error)

            validationState = .invalid(errorMessage)
        }
    }
}

Here's what we did:

1. The name PasswordViewModel doesn't work for a generalised view model, so we've changed it to ValidationViewModel. ValidationViewModel now has a generic parameter clause V that is constrained to Validator. By specialising ViewModel, we are able to reuse this view model with any concrete validator.
2. Updated value so that it is no longer of type String; instead, it has the same type as the Value defined in its validator.
3. Updated validator to be of type V.
4. ValidationViewModel doesn't try to guess how a validator's errors should be turned into a String to be passed to ValidationState. Instead, we use a closure to transform the validator's error into what is needed. This closure keeps the view model generic while still letting each field decide how its errors should be handled.
5. Updated initialValue to be the same type as the Value type of its validator.
6. Updated currentValue to be the same type as the Value type of its validator.
7. Updated the errorMessage creation to make use of errorMapper.

Before moving on, let's add a convenience initialiser to simplify creating instances of ValidationViewModel that are specialised to use String for V.Value and LocalizedError for V.ValidationError:

extension ValidationViewModel {

    // MARK: - Convenience

    convenience init(validator: V) where V.Value == String, V.ValidationError: LocalizedError {
        self.init(initialValue: "",
                  validator: validator,
                  errorMapper: { $0.errorDescription ?? "Unknown error"})
    }
}

We now have a view model that can work with any validator. Let's update RegistrationView to use ValidationViewModel:

struct RegistrationView: View {
    @Bindable var viewModel = ValidationViewModel(validator: PasswordValidator())

    // Omitted unchanged code
}

Generalising the Input Field

With a generic view model, we need a generic input field to match. Let's rename PasswordInputField to ValidationInputField:

struct ValidationInputField: View {
    // Omitted unchanged code
}

Now that PasswordInputField has had a name change to ValidationInputField, it feels strange to have it limited to SecureField. Let's make it handle both secure and non-secure text input.

Following the pattern used for validationState, let's add a new environment value:

extension EnvironmentValues {
    @Entry var isSecure: Bool = false
}

extension View {
    func isSecure(_ value: Bool = true) -> some View {
        environment(\.isSecure, value)
    }
}

Let's update ValidationInputField to respond to isSecure changes:

struct ValidationInputField: View {
    // 1
    @Environment(\.isSecure) private var isSecure

    // Omitted unchanged properties and methods

    // 2
    @ViewBuilder
    private var inputView: some View {
        // 3
        Group {
            // 4
            if isSecure {
                SecureField(placeholder,
                            text: $value)
            } else {
                TextField(placeholder,
                          text: $value)
            }
        }
        .padding(12)
        .background(Color(.systemGray6))
        .cornerRadius(8)
    }
}

Here's what we did:

1. Added a isSecure property to read from SwiftUI's environment.
2. Marked inputView with @ViewBuilder since it can now return different view types.
3. Wrapped the possible subviews in a Group to allow common view modifiers to be applied to whichever one is selected.
4. Checking the isSecure property to determine whether to render a TextField or SecureField.

We can now make use of this change to include an email field in RegistrationView:

struct RegistrationView: View {
    @Bindable var passwordViewModel = ValidationViewModel(validator: PasswordValidator())
    @Bindable var emailAddressViewModel = ValidationViewModel(validator: EmailAddressValidator())

    var body: some View {
        VStack(spacing: 20) {
            emailAddressFieldView
            passwordFieldView
        }
    }

    private var emailAddressFieldView: some View {
        ValidationInputField(title: "Email Address",
                             placeholder: "Enter your email address",
                             value: $emailAddressViewModel.value)
        .textInputAutocapitalization(.never)
        .autocorrectionDisabled()
        .keyboardType(.emailAddress)
        .textContentType(.emailAddress)
        .validationState(emailAddressViewModel.validationState)
    }

    private var passwordFieldView: some View {
        ValidationInputField(title: "Enter your password",
                             placeholder: "Enter your password",
                             value: $passwordViewModel.value)
        .textContentType(.newPassword)
        .isSecure()
        .validationState(passwordViewModel.validationState)
    }
}

With generalisation complete, we now have reusable validation components: ValidationInputField handles presentation, ValidationViewModel coordinates state, and any Validator defines rules. Supporting new input types - like an email field as shown above - is straightforward. Our generalised components allow common functionality to be shared while remaining flexible enough to fit the exact needs of the screen they appear on.

You might be wondering if you missed the EmailAddressValidator implementation - you haven't. You can see it by checking out the example project associated with this post.

Resisting that Whisper

That whisper to build a form-engine never really goes away - it's tempting, and it feels clever. But clever isn't always helpful. What works better is keeping responsibilities small and honest: a field draws itself, a validator validates, and a view model joins the dots.

This approach isn't clever, but it's easy to customise as the UI evolves. And that's often what you need when you're trying to ship a real feature - not writing a DSL.

Well done on making it to the end! 🏆

To see the complete working example, visit the repository and clone the project.

Note that the code in the repository is unit tested, so it isn't 100% the same as the code snippets shown in this post. The changes mainly involve a greater use of protocols to allow for test-doubles to be injected into the various types - see Let Dependency Injection Lead You to a Better Design for how this works. Sometimes posts like these can present too rosy an image of adding this functionality, and I wanted instead to show what validation would truly look like.

As a project grows, modularisation becomes a valuable tool for managing that growth. In the projects I've worked on, the benefits of modularisation are almost exclusively felt in the production target, with little benefit being felt in the test target. This disparity in benefits doesn't need to exist - modularisation should benefit both production and test targets. All we need to do is realise that there are more than just tests in a test target.

This post will examine how to organise a project to ensure that modularisation reduces complexity in both production and test targets by introducing test-helper targets to each module.

Lining up the target 🏹

Once we start modularising a project, we quickly discover that not all modules are the same. Some modules are feature modules, designed to be consumed by the end user; others are utility modules, designed to be consumed by other modules. When a 3rd-party module uses a utility module, the developer will add that utility module as a dependency of the 3rd-party module. The functionality of the utility module is then consumed as-is. However, when that same developer adds tests to the 3rd-party module, they often have to write test doubles of the utility module's public functionality to test their module's functionality, as the utility module doesn't provide those test doubles. Suppose that the utility module is a dependency for multiple other modules - each of those other modules now has to either:

1. Rewrite those same test doubles leading to duplicated work.
2. Introduce a test double module for each utility module, leading to a lot more module management being required.
3. Introduce a common shared test double module leading to different domains in the project artificially living together (i.e. no production reason) so unintentionally coupling those otherwise independent modules together.

So, we either waste development time reinventing something that already exists, or we increase project complexity with unnecessary modules or we couple modules together when there is no production value in it.

🤮

A better approach is to have each utility module produce a test-helper target containing the test doubles of that module's public interface. Any 3rd-party test target can then consume that utility module's public test double functionality in the same way as its production target can consume that utility module's public production functionality.

A dedicated test-helper target for each utility module has the following advantages:

1. Eliminates duplication of test doubles - rather than each 3rd-party module having to implement its own test double, those test doubles are grouped and exposed via the test-helper target of the module.
2. Improves discoverability by creating a source-of-truth - having a single source-of-truth for each module's test doubles means that everyone knows where to get those test doubles. If the required test double does not exist, then they can fill in the gap, helping future consumers of that utility module.
3. Improves consistency of each test double type - grouping the test doubles makes it easier to spot inconsistencies.
4. Improves portability - the module provides a public test interface, allowing any 3rd-party module to more quickly integrate the utility module into both production and test targets, with production and test double functionality being consumed as-is.

Let's add a test-helper target to an example app.

Adding a test-helper target

Checkout the example app to follow along.

The example app has a workspace containing two projects:

1. TestHelpers-Example - the app project.
2. Networking - a utility framework project.

TestHelpers-Example depends on Networking.

Currently, Networking lacks a test-helper target to make writing unit tests in TestHelpers-Example easier; let's change that.

1. Open the Networking target list and add a new target by clicking on the + symbol:

2. On the Choose a template for your new target window, scroll down and select Framework:

3. On the Choose options for your new target window:
   3.1. Give the new target a product-name of NetworkingTestHelpers.
   3.2. Select None in the Testing System dropdown list.

I normally suffix TestHelpers to the project name to get the test-helper target name.

Following those steps should add a framework target to the project.

4. All that is left to do now is to add the production target as a dependency on the new test-helpers target:

Now that you have a test-helper target, let's populate it.

Adding test doubles

The test-helper target contains the public test doubles for that utility module. The public test doubles are only those types with access control levels of open or public.

Networking only has one protocol and one concrete type: NetworkingService and DefaultNetworkingService. DefaultNetworkingService is a concrete type and, as such, is the concern of the production code; therefore, we can ignore it. Instead, let's focus on the protocol - NetworkingService:

public protocol NetworkingService {
    func makeRequest(url: URL) async -> Data?
}

A stub test double of NetworkingService might look like:

import Networking // 1

public class StubNetworkingService: NetworkingService { // 2
    public enum Event: Equatable {
        case makeRequestCalled(URL)
    }

    private(set) public var events = [Event]()

    public var dataToBeReturned: Data?

    public init() { }

    public func makeRequest(url: URL) async -> Data? {
        events.append(.makeRequestCalled(url))

        return dataToBeReturned
    }
}

This isn't a post about how to write test doubles, so there is no need to get in touch about how you consider StubNetworkingService to be more of a Spy than a Stub.

Here's what we did:

1. Imported the Networking production target using a standard import. When adding a test double to the test-helpers target, avoid using @testable import, as the test-helpers target should have the same constraints as any other module when dealing with the production target. We don't want to accidentally break the encapsulation of Networking via its test doubles.
2. Created a public test double.

Now that we have a test-helper target and a test double, let's use them.

Using the test-helper target

In TestHelpers-Example, the type AwesomeFeature uses NetworkingService. To ensure that awesomeness continues, we need to write unit tests. Thankfully, TestHelpers-Example already has a unit test target. However, that target doesn't depend upon our new test-helpers target, so open TestHelpers-ExampleTests in the target list and add NetworkingTestHelpers as a dependency:

Open AwesomeFeatureTests and add NetworkingTestHelpers as an import:

import NetworkingTestHelpers

Again, note that we don't use @testable import here.

We can now use StubNetworkingService to write any unit tests as we would with any other imported type:

struct AwesomeFeatureTests {

    // MARK: - Tests

    @Test("Given gainAwesomeness is called, then networking service is called with the passed in URL")
    func checkNetworkRequestIsMade() async {
        let networkingService = StubNetworkingService() // 1
        let url = URL(string: "https://example.com")!
        let sut = AwesomeFeature(networkingService: networkingService)

        await sut.gainAwesomeness(from: url)

        #expect(networkingService.events == [.makeRequestCalled(url)])
    }
}

Here's what we did:

1. Used the StubNetworkingService declared in NetworkingTestHelpers to test whether the instance AwesomeFeature works as expected.

Nothing about the above test is unusual, except that we are using an imported test double.

If another module needs to verify that its networking calls are being made correctly. In that case, importing NetworkingTestHelpers will enable the other test module to access the functionality of StubNetworkingService in much the same way as its production target accesses NetworkingService.

Bullseye 🎯

This post provides a straightforward example of using a module's test target to share a test double. While the example might be simple, it illustrates how this approach can effectively scale to share any necessary test double while adhering to the principles of modularisation.

Writing effective unit tests is crucial for any app, but the process can be time-consuming. By utilising test-helper targets, we can maximise the efficiency of our unit testing efforts. This approach eliminates duplicated work, enhances discoverability, improves consistency, and increases the portability of each module.

To see the completed project with a test-helper target, checkout the NetworkingTestHelpers branch.

In the distant past, when dinosaurs still roamed the land, iOS focused solely on what the user was doing, ensuring a responsive user experience on performance-constrained devices. While beneficial for users, this focus meant that as soon as an app went into the background, iOS suspended it. However, as the years passed, devices have become more powerful and energy-efficient. Unlike our prehistoric predecessors 🦖, iOS adapted to meet its changing environment. iOS now allows apps to perform limited background work while maintaining foreground responsiveness.

One form of permitted background work is being able to continue download or upload network requests when an app has been suspended or even terminated. These network requests are collectively known as background-transfers - background being where the network request happens and transfer being the collective terms for download and upload requests. Support for background-transfers was introduced in iOS 7. While not a recent change, it remains a powerful tool.

This post will explore background-transfers and how we can enhance the overall experience of our apps by adding support for them.

This post will gradually build up to a working example. But I get it, this is exciting stuff 😎, so if you are unable to contain that excitement, then head on over to the completed example and take a look at BackgroundDownloadService and AppDelegate to see how things end up.

Different types of sessions

When Apple introduced the URLSession suite of classes, they addressed a number of the shortfalls that were present in the NSURLConnection networking stack. One particular pain point of an NSURLConnection networking stack was that there was no built-in way to group related types of requests together - each request in a group had to be individually configured. URLSession changed this by moving that group configuration to a session. A session represents a configurable object that handles the configuration and coordination of network requests. An app can have multiple sessions. Requests scheduled within a session automatically inherit its configuration. While useful for app developers to group requests, this session-level configuration also allowed iOS to offer more functionality as standard, enabling support for background-transfers on special background sessions.

There are three flavours of session:

1. Default - supports all URLSessionTask subtypes for making a network request. These network requests can only be performed with the app in the foreground.
2. Ephemeral - similar to default but does not persist caches, cookies, or credentials to disk.
3. Background - supports all URLSessionTask subtypes for making a network request. URLSessionDataTask can only be performed with the app in the foreground whereas URLSessionDownloadTask and URLSessionUploadTask can be performed with the app in the foreground, suspended or terminated.

Each session has its own use case; let's explore how we can use a background session to enable our transfers to continue when the user leaves the app.

So how do background-transfers work? 🕵️

When scheduling a transfer on a background session, that transfer is passed to the nsurlsessiond daemon to be processed. As nsurlsessiond lives outside of the lifecycle of any given app, any transfer scheduled on nsurlsessiond will continue even if the app that the transfer belongs to is suspended/terminated. Once a background-transfer is complete, if the app has been suspended/terminated, iOS will wake the app in the background and allow the app to perform any post-transfer processing (within a limited time frame); if the app is in the foreground, control will be passed back to the app as if the transfer has been scheduled on a default or ephemeral session (without the limited time frame for post-transfer processing).

You might be thinking:

"That sounds pretty amazing! Why isn't this the default behaviour for transfers?"

Well, there are a few reasons why this isn't the default behaviour:

• Resource Management - excessive background processing can drain battery life and consume unexpected bandwidth; Apple wants to ensure that as app developers, we use it responsibly and only where it's adding value.
• Programming Complexity - implementing background-transfers requires forgoing the convenient async/await and/or closure-based methods of URLSession and instead requires conforming to URLSessionDownloadDelegate to receive updates about the transfers, adding complexity when compared to foreground-only sessions.

You may have watched Efficiency awaits: Background tasks in SwiftUI from WWDC22 and be thinking that you can use the convenient async/await and/or closure-based methods however if you do so your app will crash. The functionality shown in that walkthrough sadly never made it into production as described in this Apple Developer forum thread.

Now that we know more about background-transfers let's see how we can add support for background-transfers to our project.

Setting up the project

The app must be granted additional permissions to enable background-transfers:

1. Open the Signing & Capabilities tab in the target.
2. Add the Background Modes capability.
3. In the newly added Background Modes section, select the Background fetch and Background processing checkboxes.

After completing these steps, your settings should look like this:

With the project correctly configured, let's look at how to perform a background download.

Let's get downloading

Our background-transfer layer has five primary responsibilities:

1. Configuring a background session.
2. Scheduling the download request.
3. Responding to download updates.
4. Processing any completed downloads by moving files to a permanent location on the file system or reporting an error.
5. Recovering from a terminated/suspended state to process completed downloads.

These responsibilities are handled inside BackgroundDownloadService. As URLSessionDownloadDelegate mostly doesn't differentiate between downloads that complete when the app is in the foregrounded, suspended or terminated state, all three possible states need to be handled together.

The steps involved in handling a download that completes with the app in the foreground state are slightly different than a download that completes with the app in the suspended/terminated state:

Don't worry if any part of the above steps doesn't make sense; each part will be explored in more depth as the solution is built up.

Before looking at BackgroundDownloadService, let's take a short detour and look at a supporting type that will used in BackgroundDownloadService:

enum BackgroundDownloadError: Error {
    case cancelled
    case unknownDownload
    case fileSystemError(_ underlyingError: Error)
    case clientError(_ underlyingError: Error)
    case serverError(_ underlyingResponse: URLResponse?)
}

BackgroundDownloadError is an enum that conforms to the Error protocol and will be used to provide details about the unhappy download paths through BackgroundDownloadService.

Detour over 🗺️.

Let's build the skeleton of BackgroundDownloadService:

actor BackgroundDownloadService: NSObject { // 1
    // 2
    private lazy var session: URLSession = {
        // 3
        let configuration = URLSessionConfiguration.background(withIdentifier: "com.williamboles.background.download.session")

        // 4
        configuration.isDiscretionary = false

        // 5
        configuration.sessionSendsLaunchEvents = true
        let session = URLSession(configuration: configuration,
                                 delegate: self,
                                 delegateQueue: nil)

        return session
    }()

    // MARK: - Singleton

    // 6
    static let shared = BackgroundDownloadService()

    // 7
    private override init() {
        super.init()
    }
}

Here's what we did:

1. As mentioned earlier, background session downloads don't support the async/await or closure methods on URLSession instead updates are provided via the sessions delegate. BackgroundDownloadService needs to be a subclass of NSObject to conform to URLSessionDownloadDelegate (we will add conformance to URLSessionDownloadDelegate later in an extension).
2. There exists a cyclic dependency between BackgroundDownloadService and URLSession where BackgroundDownloadService is responsible for not only creating the URLSession instance but also being its delegate. However, the delegate of URLSession can only be set at initialisation of the URLSession. Both these conditions cannot be satisfied. To break the cyclic dependency between BackgroundDownloadService and URLSession, session is lazy stored property so the URLSession instance won't be created until its first use which is after BackgroundDownloadService has been initialised.
3. A session's configuration is defined through a URLSessionConfiguration instance, which offers numerous configuration options. Every URLSessionConfiguration for a background session requires a unique identifier within that app's lifecycle. If an existing URLSession already has that identifier, iOS returns the previously created URLSession instance. The session identifier plays a vital role in ensuring that a download can survive app termination - for any download update, iOS will attempt to find a URLSession instance with the same identifier that the download was kicked off on to inform of the update; that found URLSession instance does not need to be the same instance that was used to kick off the download. To ensure the app always has a URLSession that can pick up any download updates, session is consistently configured with the same identifier.
4. Setting isDiscretionary to false tells iOS that any downloads scheduled on this session should be started as soon as possible rather than at the discretion of iOS.
5. Setting sessionSendsLaunchEvents to true tells iOS that any downloads scheduled on this session can launch the app if it is suspended/terminated.
6. BackgroundDownloadService is singleton to ensure that any background download updates are funnelled into the same type that holds the metadata of that download.
7. init() is private to ensure that another instance of BackgroundDownloadService cannot be created.

So far, we have configured a session for performing background downloads but can't yet download anything.

Two pieces of information are required for each download:

1. Remote URL - where the content to be downloaded is.
2. File path URL - where the downloaded content should be stored on the local file system.

Where we kick off a download and where we handle the outcome of that download are split over a delegate callback. So we need a way to cache the file path URL when that download is kicked off and then retrieve it when that download completes:

actor BackgroundDownloadService: NSObject {
    // Omitted other properties

    // 1
    private let persistentStore = UserDefaults.standard
}

Here's what we did:

1. As a download may complete after the app has been terminated and relaunched, details of where the downloaded content should end up on the local file system must be persisted outside of memory. UserDefaults is a simple and effective means of persisting that information. The remote URL will be the key, with the file path URL being the value.

As mentioned, background sessions don't support the async/await methods on URLSession. However, an async/await based interface is perfect for downloading content where the caller is really only interested in having the downloaded content or an error. We can recreate that convenient async/await interface by making use of a continuation to transform the delegate-based approach into an async/await approach:

actor BackgroundDownloadService: NSObject {
    // Omitted other properties

    // 1
    private var inMemoryStore = [String: CheckedContinuation<URL, Error>]()

    // MARK: - Download

    func download(from fromURL: URL,
                  to toURL: URL) async throws -> URL {
        // 2
        return try await withCheckedThrowingContinuation { continuation in
            // 3
            inMemoryStore[fromURL.absoluteString] = continuation
            persistentStore.set(toURL, forKey: fromURL.absoluteString)

            // 4
            let downloadTask = session.downloadTask(with: fromURL)
            downloadTask.resume()
        }
    }
}

Here's what we did:

1. Just like caching the file path URL, the continuation associated with the download needs to be cached to be resumed later. The continuation here is a communication path between the caller and callee. When the app is terminated the memory holding the caller and callee will be freed, so unlike with the file path URL storage, the storage of a continuation is only in memory as there is nothing useful to persist. The remote URL will act as the key.
2. To transform the delegate-based approach into an async/await approach, the download kickoff is wrapped in a withCheckedThrowingContinuation(isolation:function:_:) block.
3. The metadata associated with this download is stored.
4. A URLSessionDownloadTask instance is created using the background session, and the download request is kicked off.

To better see the downloads happening in the background, you may wish to add a slight delay before the download starts, making it easier to put the app into the background:

downloadTask.earliestBeginDate = Date().addingTimeInterval(5)

While it is currently possible to add breakpoints to determine what is happening in BackgroundDownloadService as we will see when implementing the app relaunch behaviour, breakpoints will only get us so far. Instead of waiting until then, let's add logging into BackgroundDownloadService just now to log what is happening:

actor BackgroundDownloadService: NSObject {
    // Omitted other properties

    // 1
    private let logger = Logger(subsystem: "com.williamboles",
                                category: "background.download")

    // MARK: - Download

    func download(from fromURL: URL,
                  to toURL: URL) async throws -> URL {
        return try await withCheckedThrowingContinuation { continuation in
            // 2
            logger.info("Scheduling download: \(fromURL.absoluteString)")

            inMemoryStore[fromURL.absoluteString] = continuation
            persistentStore.set(toURL, forKey: fromURL.absoluteString)

            let downloadTask = session.downloadTask(with: fromURL)
            downloadTask.resume()
        }
    }
}

Here's what we did:

1. Apple recommends using the unified logging system to log events over something like print or NSLog. Logger writes string messages to that unified logging system. Here, we are setting the subsystem and category values, allowing us to filter the logs to see only the messages that interest us.
2. Logging an event using the info log level.

If you are curious to know more, Antoine van der Lee has written an excellent post on the unified logging system.

Now that BackgroundDownloadService can kick off a download, it needs to conform to URLSessionDownloadDelegate to receive updates about that download. As BackgroundDownloadService is of type actor, conforming to URLSessionDownloadDelegate is slightly more complex than it would be for a class type. By default, an actor enforces mutual exclusion when accessing its state, resulting in those accesses needing to be performed asynchronously from outside the actor. However, URLSession expects synchronous access to its delegate. At first glance, it looks like BackgroundDownloadService can't conform to URLSessionDownloadDelegate; however, what appears impossible at first glance isn't. An actor can have nonisolated methods. A nonisolated method while defined within an actor, is outside its mutual exclusion zone, meaning it can be accessed synchronously - essentially, acting like a regular class method. By marking the URLSessionDownloadDelegate methods as nonisolated, BackgroundDownloadService can act as the delegate to its session - with one caveat: any work that needs to access the state of BackgroundDownloadService needs to be moved into the mutual exclusion zone rather than handled in the nonisolated method.

Let's implement the happy-path method of urlSession(_:downloadTask:didFinishDownloadingTo) from URLSessionDownloadDelegate:

extension BackgroundDownloadService: URLSessionDownloadDelegate { // 1
    // MARK: - URLSessionDownloadDelegate

    // 2
    nonisolated
    func urlSession(_ session: URLSession,
                    downloadTask: URLSessionDownloadTask,
                    didFinishDownloadingTo location: URL) {

        // 3
        let tempLocation = FileManager.default.temporaryDirectory.appendingPathComponent(location.lastPathComponent)
        try? FileManager.default.moveItem(at: location,
                                          to: tempLocation)

        // 4
        Task {
            await downloadFinished(task: downloadTask,
                                   downloadedTo: tempLocation)
        }
    }
}

Here's what we did:

1. BackgroundDownloadService conforms to URLSessionDownloadDelegate in an extension.
2. urlSession(_:downloadTask:didFinishDownloadingTo) is marked as a nonisolated method to allow synchronous access.
3. When URLSessionDownloadTask completes a download, it will store that downloaded content in a temporary location - location. iOS only guarantees until the end of this method that the downloaded content will be found at location. As the real work of completing a download will occur in downloadFinished(task:downloadedTo:) which is accessed asynchronously via a Task waiting on downloadFinished(task:downloadedTo:) being called before moving the downloaded content will mean that urlSession(_:downloadTask:didFinishDownloadingTo) will have exited and the downloaded content will have been deleted. So before making that asynchronous call, the downloaded content is moved from its current temporary location to a different temporary location that BackgroundDownloadService controls to ensure that it will still be there for downloadFinished(task:downloadedTo:) to work with.
4. The call to downloadFinished(task:downloadedTo:) is wrapped in a Task as it exists within BackgroundDownloadService mutual exclusion zone, which urlSession(_:downloadTask:didFinishDownloadingTo) is not in, so downloadFinished(task:downloadedTo:) treated as an async method.

Let's see what downloadFinished(task:downloadedTo:) does:

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    private func downloadFinished(task: URLSessionDownloadTask,
                                  downloadedTo location: URL) {

        // 1
        guard let fromURL = task.originalRequest?.url else {
            logger.error("Unexpected nil URL for download task.")
            return
        }

        logger.info("Download request completed for: \(fromURL.absoluteString)")

        // 2
        defer {
            cleanUpDownload(forURL: fromURL)
        }

        // 3
        guard let toURL = persistentStore.url(forKey: fromURL.absoluteString) else {
            logger.error("Unable to find existing download for: \(fromURL.absoluteString)")
            return
        }

        // 4
        let continuation = inMemoryStore[fromURL.absoluteString]

        logger.info("Download successful for: \(fromURL.absoluteString)")

        // 5
        do {
            try FileManager.default.moveItem(at: location,
                                             to: toURL)
            // 6
            continuation?.resume(returning: toURL)
        } catch {
            logger.error("File system error while moving file: \(error.localizedDescription)")
            // 7
            continuation?.resume(throwing: BackgroundDownloadError.fileSystemError(error))
        }
    }

    // MARK: - Cleanup

    // 8
    private func cleanUpDownload(forURL url: URL) {
        inMemoryStore.removeValue(forKey: url.absoluteString)
        persistentStore.removeObject(forKey: url.absoluteString)
    }
}

This method can be called when the app is foregrounded, suspended, or terminated, so all three states need to be handled in the same method:

1. In the download(from:to:) method on BackgroundDownloadService, the fromURL was used as the key for storing a download's metadata because that value is present when a download completes in the URLSessionDownloadTask instance. Here, we extract that URL from URLSessionDownloadTask. It's important to note that a URLSessionDownloadTask will automatically follow redirect instructions, so it is essential to ensure that the correct URL is used to connect the completed download with its metadata - specifically, the original URL stored in the originalRequest property. If originalRequest is nil, the method exits as no further action can be taken.
2. As there are multiple exits from downloadFinished(task:downloadedTo:), a defer block is used to clean up the download by deleting the download's associated metadata on any exit.
3. If toURL is nil, the download cannot be processed because there is no known location to move the downloaded content to. As a result, the method exits with no further action taken.
4. Unlike toURL, there is no hard requirement for a continuation to be present, so no check is made to ensure it is not nil.
5. Using FileManager, the downloaded content is moved from its temporary location to its permanent location.
6. The associated continuation is resumed with the file's permanent location returned.
7. The associated continuation is resumed with an error being thrown.
8. The metadata associated with this download is deleted. This deletion will occur in several methods, so it is best to extract it now.

That's the happy path completed; let's add in the unhappy-path.

The unhappy-path comes in two forms: server-side and client-side errors. Let's start with the server-side errors by updating downloadFinished(task:downloadedTo:):

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    private func downloadFinished(task: URLSessionDownloadTask,
                                  downloadedTo location: URL) {
        guard let fromURL = task.originalRequest?.url else {
            logger.error("Unexpected nil URL for download task.")
            return
        }

        logger.info("Download request completed for: \(fromURL.absoluteString)")

        defer {
            cleanUpDownload(forURL: fromURL)
        }

        guard let toURL = persistentStore.url(forKey: fromURL.absoluteString) else {
            logger.error("Unable to find existing download for: \(fromURL.absoluteString)")
            return
        }

        let continuation = inMemoryStore[fromURL.absoluteString]

        // 1
        guard let response = task.response as? HTTPURLResponse,
              response.statusCode == 200 else {
            logger.error("Unexpected response for: \(fromURL.absoluteString)")
            continuation?.resume(throwing: BackgroundDownloadError.serverError(task.response))
            return
        }

        // Omitted rest of method
    }
}

Here's what we changed:

1. To determine if a download was successful, a 200 HTTP status code is expected. Here, the response is checked. If the status code is anything other than a 200, then completionHandler is triggered with serverError.

If your server returns other status codes that should be treated as success, then it is simple enough to add those by refactoring the guard to accommodate those additional status codes.

Now that server-side errors are handled, let's handle the client-side errors. Let's implement the unhappy-path method of urlSession(_:task:didCompleteWithError:) from URLSessionDownloadDelegate:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    // Omitted other methods

    // 1
    nonisolated
    func urlSession(_ session: URLSession,
                    task: URLSessionTask,
                    didCompleteWithError error: Error?) {
        // 2
        Task {
            await downloadComplete(task: task,
                                   withError: error)
        }
    }
}

Here's what we changed:

1. Just like with urlSession(_:downloadTask:didFinishDownloadingTo), urlSession(_:task:didCompleteWithError:) is marked as a nonisolated method to allow synchronous access.
2. The call to downloadComplete(task:withError:) is wrapped in a Task as it exists within BackgroundDownloadService mutual exclusion zone, which urlSession(_:task:didCompleteWithError:) is not in, so downloadComplete(task:withError:) is treated as an async method.

Let's see what downloadComplete(task:withError:) does:

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    private func downloadComplete(task: URLSessionTask,
                                  withError error: Error?) {
        // 1
        guard let error = error else {
            return
        }

        // 2
        guard let fromURL = task.originalRequest?.url else {
            logger.error("Unexpected nil URL for task.")
            return
        }

        logger.info("Download failed for: \(fromURL.absoluteString), error: \(error.localizedDescription)")

        let continuation = inMemoryStore[fromURL.absoluteString]

        // 3
        continuation?.resume(throwing: BackgroundDownloadError.clientError(error))

        // 4
        cleanUpDownload(forURL: fromURL)
    }
}

Here's what we did:

1. downloadComplete(task:withError:) is called when any network request completes, meaning it will be triggered for both successful and unsuccessful network requests. If a network request fails, error will contain a value. To exclude successful downloads from the unhappy-path, error is checked for nil. If error is nil, the method exits.
2. Like in downloadFinished(task:downloadedTo:), the remote URL is extracted from the completed URLSessionTask instance as fromURL. If originalRequest is nil, the method exits as no further action can be taken.
3. The associated continuation is resumed with an error being thrown.
4. The metadata associated with this download is deleted.

If you've been putting together the code snippets, that's all you need to do to use a background session to download files in the foreground; however, we still have work to do to support relaunching the app to complete a download.

Keeping downloads going

When an app is in a suspended/terminated state and a download completes, iOS will wake the app up (in the background) by calling application(_:handleEventsForBackgroundURLSession:completionHandler:) on the AppDelegate. This method accepts a closure completionHandler that needs to be triggered once we have completed post-download processing for all background downloads. Triggering completionHandler instructs iOS to take a new snapshot of the app's UI for the app switcher preview. Failure to call completionHandler will result in iOS treating the app as a bad citizen, reducing future background processing opportunities. Regardless of how many downloads are in progress, application(_:handleEventsForBackgroundURLSession:completionHandler:) is only called once - when all downloads have been completed. If the user foregrounds the app before the above method is called, urlSession(_:downloadTask:didFinishDownloadingTo:) is called for each completed download.

Depending on how your app is set up, open the section relevant to your project:

class AppDelegate: NSObject, UIApplicationDelegate {
    // MARK: - UIApplicationDelegate

    func application(_ application: UIApplication,
                     handleEventsForBackgroundURLSession identifier: String,
                     completionHandler: @escaping () -> Void) {
        // 1
        Task {
            await BackgroundDownloadService.shared.saveAppPreviewCompletionHandler(completionHandler)
        }
    }
}

@main
struct BackgroundTransferRevised_ExampleApp: App {
    // 1
    @UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate

    // Omitted other properties
}

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    // MARK: - UIApplicationDelegate

    func application(_ application: UIApplication,
                     handleEventsForBackgroundURLSession identifier: String,
                     completionHandler: @escaping () -> Void) {
        // 1
        Task {
            await BackgroundDownloadService.shared.saveAppPreviewCompletionHandler(completionHandler)
        }
    }
}

Regardless of how we got here, the completionHandler closure has been passed to BackgroundDownloadService, which has then stored it as a property:

actor BackgroundDownloadService: NSObject {
    // Omitted other properties

    private var appPreviewCompletionHandler: (() -> Void)?

    // Omitted other methods

    func saveAppPreviewCompletionHandler(_ appPreviewCompletionHandler: @escaping (() -> Void)) {
        self.appPreviewCompletionHandler = appPreviewCompletionHandler
    }
}

Just like when the app is in the foreground, the URLSessionDownloadDelegate method: urlSession(_:downloadTask:didFinishDownloadingTo:) is called for each download however, unlike when the app is in the foreground, there is an additional background-transfer only delegate call made after the final urlSession(_:downloadTask:didFinishDownloadingTo:) call:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    // Omiited other methods

    nonisolated
    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        Task {
            await backgroundDownloadsComplete()
        }
    }
}

Here's what we changed:

1. Just like with the other URLSessionDownloadDelegate methods, urlSessionDidFinishEvents(forBackgroundURLSession:) is marked as nonisolated method to allow synchronous access.
2. The call to backgroundDownloadsComplete() is wrapped in a Task as it exists within BackgroundDownloadService mutual exclusion zone which urlSessionDidFinishEvents(forBackgroundURLSession:) is not in so backgroundDownloadsComplete() is treated as an async method.

Let's see what backgroundDownloadsComplete() does:

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    private func backgroundDownloadsComplete() {
        logger.info("All background downloads completed")

        // 1
        appPreviewCompletionHandler?()
        appPreviewCompletionHandler = nil
    }
}

Here's what we did:

1. appPreviewCompletionHandler is triggered and then set to nil to avoid it accidentally being triggered again.

While it's possible to make additional network requests with the app running in the background, they are subject to a rate limiter to prevent abuse of background sessions - this rate limiter delays the start of those network requests. With each separate network request, the delay increases - this delay is reset when the user brings the app to the foreground or if your app does not make any additional network requests during the delay.

The code has been added to handle the app being relaunched when suspended/terminated, but it is important to verify that the implementation works as intended. When an app is manually force-quit, iOS interprets this as the user explicitly indicating a desire to halt all activity related to the app. Consequently, all scheduled background transfers are cancelled, which prevents testing of the restoration functionality. To overcome this gotcha, app termination needs to happen programmatically using exit(0), which shuts down the app without user intervention.

Depending on how your app is set up, open the section relevant to your project:

@main
struct BackgroundTransferRevised_ExampleApp: App {
    // Omitted other properties

    @Environment(\.scenePhase) private var scenePhase

    private let logger = Logger(subsystem: "com.williamboles",
                                category: "app")

    // MARK: - Scene

    var body: some Scene {
        WindowGroup {
            // Omitted statements
        }
        .onChange(of: scenePhase) { (_, newPhase) in
            guard newPhase == .background else {
                return
            }

            //Exit app to test restoring app from a terminated state.
            Task {
                logger.info("Simulating app termination by exit(0)")

                exit(0)
            }
        }
    }
}

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
    private let logger = Logger(subsystem: "com.williamboles",
                                category: "appDelegate")

    // Omitted other methods

    func applicationDidEnterBackground(_ application: UIApplication) {
        //Exit app to test restoring app from a terminated state.
        Task {
            logger.info("Simulating app termination by exit(0)")

            exit(0)
        }
    }
}

With the above changes, the app will be terminated when it goes into the background. However, once the app is terminated, the debugger will lose its link with the app, meaning any set breakpoints will be made invalid 😟. But don't be too sad, as we've built up this solution, we've been using Logger to output what the app has been doing. These logs will keep being produced even when the debugger isn't linked. These logs can be seen in the Console app. It is possible to filter the logs in the Console app - as logged events have been using com.williamboles as the subsystem value, this can be used to filter:

Running the app with the Console open should allow us to see that any in-progress downloads complete beyond the app termination.

So, we've implemented the ability to start downloading when the app is in the foreground and have that continue even if the app is terminated. However, there is one more common scenario to implement - handling duplicate requests. At the moment, if a duplicate download is made before the first one has completed, then the following error is thrown:

SWIFT TASK CONTINUATION MISUSE: download(from:to:) leaked its continuation!

This error is due to the metadata associated with the first download being replaced without the continuation being resumed.

There are two approaches we could implement to handle this scenario:

1. Coalescing - piggybacking any subsequent request on the in-progress request and so calling multiple continuations when the download completes.
2. Cancellation - cancelling the in-progress request, throwing an error and starting a new request.

Both of these approaches are valid, and which one you go for depends on the requirements of your app. Cancellation is more straightforward, so let's implement it.

To cancel a download, we need access to its URLSessionDownloadTask instance, so let's track the currently active downloads:

actor BackgroundDownloadService: NSObject {
    // Omitted other properties

    // 1
    private var activeDownloads = [String: URLSessionDownloadTask]()

    // Omitted other methods

    func download(from fromURL: URL,
                  to toURL: URL) async throws -> URL {
        return try await withCheckedThrowingContinuation { continuation in
            // Omitted other statements

            let downloadTask = session.downloadTask(with: fromURL)
            // 2
            activeDownloads[fromURL.absoluteString] = downloadTask
            downloadTask.resume()
        }
    }

    private func cleanUpDownload(forURL url: URL) {
        //Omitted other statements

        // 3
        activeDownloads.removeValue(forKey: url.absoluteString)
    }
}

Here's what we did:

1. Similar to the continuation caching, the URLSessionDownloadTask instance is in a dictionary. The remote URL will act as the key.
2. The URLSessionDownloadTask instance is stored.
3. When a download is completed, the URLSessionDownloadTask instance is removed from activeDownloads.

Now that we have access to the URLSessionDownloadTask instance associated with a download, we can cancel it:

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    func cancelDownload(forURL url: URL) {
        logger.info("Cancelling download for: \(url.absoluteString)")

        // 1
        inMemoryStore[url.absoluteString]?.resume(throwing: BackgroundDownloadError.cancelled)

        // 2
        activeDownloads[url.absoluteString]?.cancel()

        // 3
        cleanUpDownload(forURL: url)
    }
}

Here's what we did:

1. The continuation associated with url is resumed with a cancelled error being thrown.
2. The URLSessionDownloadTask instance associated with url is cancelled.
3. The metadata associated with url is deleted.

The cancellation shown is a simple throw-everything-away cancellation; if you want a more sophisticated approach, check out my post on pausable downloads to see how a download can be resumed from where it was cancelled.

When a download cancels the downloadComplete(task:withError:) method is called with a URLError.cancelled error, let's update that method so we don't double clean up cancelled downloads:

actor BackgroundDownloadService: NSObject {
   // Omitted properties and other methods

    private func downloadComplete(task: URLSessionTask,
                                  withError error: Error?) {
        guard let error = error else {
            return
        }

        // 1
        if let error = error as? URLError,
           error.code == .cancelled {
            return
        }

        // Omitted rest of method
    }
}

Here's what we did:

1. error is to see if it is an URLError.cancelled error, if so the method is exited.

All that is left to do is to cancel the active download when a new download request is made:

actor BackgroundDownloadService: NSObject {
    // Omitted properties and other methods

    func download(from fromURL: URL,
                  to toURL: URL) async throws -> URL {
        // 1
        if activeDownloads[fromURL.absoluteString] != nil {
            cancelDownload(forURL: fromURL)
        }

        // Omitted rest of method
    }
}

Here's what we did:

1. activeDownloads is checked for an entry with fromURL as the key. If that entry exists, the download is cancelled; if that entry does not exist, cancellation is skipped.

With those changes, no continuation misuse error will be thrown when a duplicate download request is made.

Congratulations, that's all the code needed to support background-transfers.

💃 🕺

Downloads keep going 🤯

Background-transfers provide a powerful tool to meet your users' ever-increasing expectations about what your app can do for them, even when it isn't in the foreground. The enhanced user experience we gain from supporting background-transfers, I believe, outweighs any additional complexity added by supporting background-transfers.

To see the complete working example, visit the repository and clone the project.

Running the example project

I've used TheCatAPI in the example project to populate the app with downloaded images. TheCatAPI has an extensive library of freely available cat photos which it shares via a JSON-based API. TheCatAPI does require you to register to get full access to it's API (limited access is provided without an API key). Once registered you will be given an x-api-key token which you can paste as the APIKey value in NetworkService to get that full access.

Dealing with a JSON network response in iOS projects used to be a pain - you would have to manually parse the response, extract the required values, ignore those that weren't needed, and build your model instances 🤮. At best, it was tedious work; at worst, it was a source of bugs. It wasn't long before a whole host of 3rd party solutions were developed to automate away the process of matching a JSON response to a Swift type. As sometimes happens when a 3rd party solution gains traction in the developer community, this functionality was pulled into Swift itself. In Swift 4, we got native support for encoding and decoding JSON in the form of JSONEncoder and JSONDecoder that worked side-by-side with two protocols Encodable and Decodable to make converting between Swift types and JSON easy-peasy.

Encodable and Decodable protocols are often combined as the Codable protocol.

For encoding, take the type you want to encode, conform it to Encodable and pass it to an instance of JSONEncoder:

struct Example: Encodable {
    let title: String
}

let example = Example(title: "Stay wonderful!")

let encoded = try JSONEncoder().encode(example)

encoded holds a JSON representation of an instance of Example.

Decoding is equally as simple. Take the below JSON response:

{
    "title":"Will do!"
}

The above JSON can be decoded into an Example instance by conforming Example to Decodable and passing it, along with the JSON response in Data form, to an instance of JSONDecoder:

struct Example: Encodable, Decodable {
    let title: String
}

let decoded = try JSONDecoder().decode(Example.self, from: data)

decoded now holds an Example instance with a title value of: Will do!.

We get a lot of functionality for the amount of code written above. We get that functionality because Encodable and Decodable are more than just protocols. Conforming a type to either protocol triggers the compiler to synthesise conformance to those protocols (notice how Example doesn't implement any of the methods defined in either Encodable or Decodable). In order to synthesise conformance, the compiler makes several assumptions about how our Swift type matches its JSON counterpart.

In this post, I want to explore what happens when one of those assumptions proves false. When automatic decoding/encoding isn't possible because the structure of the JSON representation can't be directly converted into a Swift representation due to differences between how JSON treats data and how Swift treats data.

Overcoming differences 🤝

The array type in Swift is homogeneous, i.e. each element is of the same type; the array type in JSON is heterogeneous, i.e. elements can be of different types. This can present a tricky issue for us as consumers of a JSON endpoint that returns different types in the same array.

Let's take the below JSON response as an example:

{
   "media":[
      {
         "media_type":"text",
         "id":12,
         "text":"This is an example of text media"
      },
      {
         "media_type":"image",
         "id":2785,
         "caption":"This is an example of image media",
         "url":"https://example.com/images/2785.jpg"
      }
   ]
}

Here the array, media, is heterogeneous as it contains 2 different JSON objects: text and image. Directly converting the JSON media array into a Swift array isn't possible as there is no way to declare a Swift array that holds multiple types.

However, it is possible to indirectly hold Swift representations of text and image in an array if those two types are grouped under a common type. In Swift, an enum is the perfect data structure for grouping a suite of distinct but related types.

Using an enum, it is possible to customise the decoding process to extract elements from the above JSON response as distinct objects and still keep them grouped in the same array.

Let's start by looking at how we determine what type each element in the media array is:

//1
struct Content: Decodable {
    let media: [Media]
}

//2
enum Media: Decodable {
    case text
    case image

    //3
    enum CodingKeys: String, CodingKey {
        case mediaType = "media_type"
    }

    // MARK: - Init

    //4
    init(from decoder: Decoder) throws {
        //5
        let container = try decoder.container(keyedBy: CodingKeys.self)
        let type = try container.decode(String.self, forKey: .mediaType)

        //6
        switch type {
        case "text":
            self = .text
        case "image":
            self = .image
        default:
            fatalError("Unexpected media type encountered")
        }
    }
}

Let's walk through the above code:

1. Content, which conforms to Decodable, holds the array of all Media instances and is used to mirror the JSON structure.
2. Media, which conforms to Decodable, is an artificial type that expresses each known media type as a case.
3. As Media doesn't have any properties, no coding keys are synthesised. So Media has to define its own - CodingKeys. CodingKeys conforms to CodingKey, which the JSONDecoder instance expects its keys to be. The CodingKeys enum only contains one case as the only information from the JSON that Media needs to know about to determine what case to be is - media_type.
4. In order to customise the decoding process, Media needs to implement it's own init(from decoder: Decoder) throws method rather than depend on the synthesised version.
5. A container is created using the keys declared in the CodingKeys enum, with the media_type value extracted as a String instance.
6. type is switched over to compare against the 2 supported media types. If type is a match for the string representation, self is set to that case; if there is no match, a fatal error is thrown.

The fatalError could be replaced with an unknown/unsupported case if crashing the app here is undesired.

While Media can determine which type each element in media is, it's not that useful on its own. Let's extend Media to capture the details of each element in media:

struct Text: Decodable {
    let id: Int
    let text: String
}

struct Image: Decodable {
    let id: Int
    let caption: String
    let url: URL
}

Text and Image each conform to Decodable and mirror their respective JSON object. Text and Image will be used as associated values to the cases in Media.

Note that we didn't need to implement init(from decoder: Decoder) throws here as the synthesised implementation is perfect for our needs.

Let's alter Media to make use of Text and Image:

enum Media: Decodable {
    //1
    case text(Text)
    case image(Image)

    //Omitting unchanged code

    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        let type = try container.decode(String.self, forKey: .mediaType)

        //2
        switch type {
        case "text":
            let text = try Text(from: decoder)
            self = .text(text)
        case "image":
            let image = try Image(from: decoder)
            self = .image(image)
        default:
            fatalError("Unexpected media type encountered")
        }
    }
}

1. Each case now has a dedicated type for that media type as an associated value.
2. For each media type, the Decoder instance is passed into Text or Image as needed to continue the decoding process at the next level.

We can test our decoding implementation by:

let json = """
{
   "media":[
      {
         "media_type":"text",
         "id":12,
         "text":"This is an example of text media"
      },
      {
         "media_type":"image",
         "id":2785,
         "caption":"This is an example of image media",
         "url":"https://example.com/images/2785.jpg"
      }
   ]
}
"""

let content = try JSONDecoder().decode(Content.self, from: json.data(using: .utf8)!)

If everything went well, Content should contain the same data as media does in the JSON representations.

You can see this in action by running the ContentTests in the linked project.

Now that the decoding side has been explored let's look at how to encode Content:

//1
struct Content: Decodable, Encodable {
    let media: [Media]
}

//2
enum Media: Decodable, Encodable {
    //Omitted unchanged properties and methods

    // MARK: - Encode

    //3
    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)

        let object: Codable
        let type: String

        switch self {
        case .text(let text):
            type = "text"
            object = text
        case .image(let image):
            type = "image"
            object = image
        }

        try container.encode(type, forKey: .mediaType)
        try object.encode(to: encoder)
    }
}

//4
struct Text: Decodable, Encodable {
    //Omitted unchanged properties
}

//5
struct Image: Decodable, Encodable {
    //Omitted unchanged properties
}

I could have used Codable, which combines Decodable and Encodable, but I kept them separate in this article for clarity.

1. To support encoding, Content now needs to conform to Encodable.
2. To support encoding, Media now needs to conform to Encodable.
3. As the Swift implementation of a JSON media object is two types, a custom func encode(to encoder: Encoder) throws needs to be implemented to combine those two types back into one. Here a container is made from CodingKeys so that the media type can be encoded before the encoder is passed to the enum cases associated value type instance to add the stored properties of that instance to the data that's already been encoded.
4. To support encoding, Text now needs to conform to Encodable.
5. To support encoding, Image now needs to conform to Encodable.

Using an enum to bridge the gap between JSON and Swift isn't only limited to elements in an array. Another common use case is where the structure of the JSON object is the same, but the type of a field changes, e.g. sometimes an Int, sometimes a String, etc. - in this case, we use an enum to represent the type in pretty much the same way as shown above.

And that's it! 🥳

Looking back

As we have seen, Decodable and Encodable are easy-to-use, powerful tools in the Swift toolkit. While on the surface, Decodable and Encodable are just protocols, when combined with the synthesised functionality we get from the compiler, we get the ability in most cases to convert from JSON into Swift types and vice versa with either no customisation or at least very little. Even in our tricky JSON example that proved too complex for the synthesised functionality to automatically convert to Swift, the amount of code we had to write wasn't much.

To see the above code snippets in a working example alongside unit tests, head over to the repo and clone the project.

UIAlertController alerts form the backbone of many interactions between our users and our apps. Alerts are often shown at those critical points in our apps where we request confirmation of an action or permission to access a resource. While there have been some changes to alerts over the years, their appearance has remained largely unchanged. This lack of customisation presents significant difficulties for app designers 😱. Having an UIAlertController alert pop up at the most critical points of our apps with its semi-transparent rectangular layout and standard fonts, breaking the app's theme is enough to make any app designer spill their flat white coffee down their checkered shirt and throw their Caran d'Ache coloured pencils across the table with an angst that few could ever comprehend, never mind experience. At this point, as you gaze into their tear-filled eyes, you offer a few comforting words:

"We can always write our own custom alerts instead"

Slowly, as your words start to cut through their anguish, they start to nod, and with strength returning to their voice, they say:

"Sounds good man, I'll get started on those right away"

And with that, you turn away and start thinking about how you can present those custom alerts.

This post will look at how to build a custom alert presenter that adheres to most of the conventions used by UIAlertController alerts. Along the way, we will even overcome a particularly tricky and challenging-to-reproduce navigation bug 💪.

This post will gradually build up to a working example; however, if you are too excited and want to jump ahead, then head on over to the completed example and take a look at AlertPresenter and AlertWindow to see how things end up.

Thinking about being standard

A standard UIAlertController alert has four parts:

1. Foreground view.
2. Background view.
3. Presentation animation.
4. Dismissal animation.

The first thing to decide is:

"What is part of the custom alert, and what is part of the mechanism to show an alert?"

Let's look at how UIAlertController handles the four parts of an alert:

1. Foreground view - UIAlertController allows some customisation.
2. Background view - UIAlertController handles this for us.
3. Presentation animation - UIAlertController handles this for us.
4. Dismissal animation - UIAlertController handles this for us.

The only customisable part of an UIAlertController alert is the foreground view. This lack of control over the other parts may initially feel limiting, but by preventing customisation of three of the four parts, iOS forces us to focus on the most critical part - the message. The message is contained in the foreground view.

Just like UIAlertController, the alert presentation layer we will build below will only allow the foreground view to be customisable. This limitation will ensure that presenting and dismissing alerts will happen consistently across the app. Instead of the foreground view being a UIView instance, it will be a UIViewController instance to provide greater control to our custom alerts. This UIViewController instance will be added to the view hierarchy as a child view-controller. This functionality will come together in the following class structure:

• AlertPresenter is the entry point for presenting and dismissing alerts.
• AlertWindow, as we will see shortly, each alert is presented in its own UIWindow instance. Used to overcome that tricky navigation issue we spoke about above.
• HoldingViewController, the window's root view-controller that is responsible for presenting the AlertContainerViewController that will hold the alert view-controller as a child view-controller.
• AlertContainerViewController, the parent view-controller that the alert view-controller is embedded in as a child view-controller.
• CustomAlertPresentAnimationController is responsible for presenting the AlertContainerViewController instance with the same animation as a standard UIAlertController.
• CustomAlertDismissAnimationController is responsible for dismissing the AlertContainerViewController instance with the same animation as a standard UIAlertController.

HoldingViewController, AlertContainerViewController, CustomAlertPresentAnimationController and CustomAlertDismissAnimationController are private and only known to AlertWindow.

Don't worry if that doesn't all make sense yet; we will look into each class in greater depth below.

Let's start with AlertPresenter:

class AlertPresenter {
    static let shared = AlertPresenter()

    // MARK: - Present

    func presentAlert(_ viewController: UIViewController) {
        os_log(.info, "Alert being presented")

        //TODO: Present
    }
}

AlertPresenter is a singleton, so the same instance will be used to present (and dismiss) all alerts. As AlertPresenter isn't a UIViewController subclass, it's not possible to directly present the alert. Instead, we are going to use a dedicated UIWindow instance to present alerts from. Using a dedicated UIWindow instance should avoid the situation where multiple simultaneous navigation events (presentation/dismissal) occur at the same time, resulting in one of those events being cancelled, and the following error is generated:

The navigation stack in one window is independent of the navigation stack of any other windows. An alert in the process of being presented on window A will not cause a navigation collision with a view-controller being pushed on window B 🥳.

Before delving into how to use a dedicated window to present alerts, let's get to know windows better.

If you are comfortable with how UIWindow works, feel free to skip ahead.

Getting to know windows 💭

UIWindow is a subclass of UIView that acts as the container for an app's visible content - it is the top of the view hierarchy. All views that are displayed to the user need to be added to a window. An app can have multiple windows, but only windows that are visible can have their content displayed to the user - by default, windows are not visible. Multiple windows can be visible at once. Each window's UI stack is independent of other windows UI stacks. Where a window is displayed in relation to other visible windows is controlled by setting that window's windowLevel property. The higher the windowLevel, the nearer to the user that window is. UIWindow has three default levels:

1. .normal
2. .statusBar
3. .alert

With .alert > .statusBar > .normal. If a more fine-grain level of control is needed, it's possible to use a custom level:

window.windowLevel = .normal + 25

As of iOS 13: .alert has a raw value of 2000, .statusBar has a raw value of 1000 and .normal has a raw value of 0.

Where two or more windows have the same level, their ordering is determined by the order they were made visible in - the last window made visible is nearest one to the user.

It's unusual to add subviews to a window directly; instead, each window should have a root view-controller whose view is used as the window's initial subview.

As well as displaying content, UIWindow is also responsible for forwarding any events (touch, motion, remote-control or press) to interested parties in its responder chain. While all touch events are forwarded to the responder chain of the window that the event occurred on, events that are outside of the app's UI, such as motion events or keyboard entry, are forwarded to the key window. Only one window can be key at any one given time (which window is key can change throughout the lifetime of the app).

An iOS app needs at least one window; a reference to this window can be found in the app delegate:

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?

    //Omitted methods
}

If you're using storyboards to lay out your interface, then most of the work of setting up this window is happening under the hood. When the app is launched, a UIWindow instance is created that fills the screen. This window is then assigned to the window property (declared in the UIApplicationDelegate protocol), configured with the view-controller declared as the storyboard entry point from the project's main storyboard and made key and visible.

If you are not using storyboards, you can better see this setup:

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?

    // MARK - AppLifecycle

    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        window = UIWindow()
        window?.rootViewController = YourViewController()
        window?.makeKeyAndVisible()

        return true
    }
}

Using a dedicated window

As this new window will only display alerts, let's subclass UIWindow for this single purpose:

class AlertWindow: UIWindow {

    // MARK: - Init

    init(withViewController viewController: UIViewController) {
        super.init(frame: UIScreen.main.bounds)

        // 1
        rootViewController = viewController

        // 2
        windowLevel = .alert
    }

    @available(*, unavailable)
    required init?(coder aDecoder: NSCoder) {
        fatalError("Unavailable")
    }

    // MARK: - Present

    // 3
    func present() {
        makeKeyAndVisible()
    }
}

In AlertWindow we:

1. Set the alert as the window's rootViewController.
2. Set the window level value to .alert. The .alert level will put this window above the app's main window, ensuring that it can be seen.
3. present() is a bit of syntactic sugar around making the window key and visible; it will act as a mirror to the soon-to-be-seen dismiss method.

An AlertWindow instance is only meant to show one alert and then be disposed of.

Let's use AlertWindow to present alerts:

class AlertPresenter {
    // 1
    private var alertWindows = Set()

    //Omitted other properties

    // MARK: - Present

    // 2
    func presentAlert(_ viewController: UIViewController) {
        os_log(.info, "Alert being presented")

        let alertWindow = AlertWindow(withViewController: viewController)
        alertWindow.present()

        alertWindows.insert(alertWindow)
    }
}

With the above changes, we:

1. Store all windows that are being presented in a Set. Storing the window inside the set will keep that window alive by increasing its retain count (as making a window key-and-visible doesn't increase the retain count).
2. Create an AlertWindow instance using the alert to be presented and then instruct that window to present itself.

If you were to create an instance of AlertWindow and make it visible, you would notice that the alert is presented without an animation - it just appears. A window's root view-controller cannot be animated on-screen, so an intermediate view-controller is needed, which can be the window's root view-controller, and then the alert can be presented from that view-controller:

class HoldingViewController: UIViewController {
    private let viewController: UIViewController

    // MARK: - Init

    init(withViewController viewController: UIViewController) {
        self.viewController = viewController
        super.init(nibName: nil, bundle: nil)
    }

    required init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    // MARK: - ViewLifecycle

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)

        present(viewController, animated: true, completion: nil)
    }
}

HoldingViewController only has one responsibility - presenting an alert once viewDidAppear(_:) has been called.

Trying to present an alert earlier will cause a navigation collision due to the HoldingViewController instance having not finished its own "animation" on screen. While (at the time of writing - iOS 13) this doesn't seem to affect the actual presentation of the alert, waiting for HoldingViewController to be presented before attempting another presentation ensures that the error isn't produced.

Let's go back to AlertWindow and make use of HoldingViewController:

class AlertWindow: UIWindow {
    private let holdingViewController: HoldingViewController

    // MARK: - Init

    init(withViewController viewController: UIViewController) {
        holdingViewController = HoldingViewController(withViewController: viewController)
        super.init(frame: UIScreen.main.bounds)

        rootViewController = holdingViewController

        windowLevel = .alert
    }


    // MARK: - Present

    func present() {
        makeKeyAndVisible()
    }
}

If you run the project with the above changes and hook in your own alert, you would notice two issues:

1. Sizing - the alert occupies the full-screen.
2. Presentation - the alert is animated from bottom to top.

Sizing alerts

An alert should be shown at the smallest size possible - we can't do this if that alert is the modal. Instead, we need to embed that alert into another view controller. HoldingViewController can't be used for this as it is being used as the window's root view-controller so it can't be animated on-screen. We need to introduce a new view-controller that will act as a container so that the container can be animated on-screen from the HoldingViewController instance:

class AlertContainerViewController: UIViewController {
    let childViewController: UIViewController

    // MARK: - Init

    // 1
    init(withChildViewController childViewController: UIViewController) {
        self.childViewController = childViewController
        super.init(nibName: nil, bundle: nil)
    }

    required init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    // MARK: - ViewLifecycle

    override func viewDidLoad() {
        super.viewDidLoad()

        // 2
        addChild(childViewController)
        view.addSubview(childViewController.view)
        childViewController.didMove(toParent: self)

        // 3
        childViewController.view.translatesAutoresizingMaskIntoConstraints = false
        NSLayoutConstraint.activate([
            childViewController.view.centerXAnchor.constraint(equalTo: view.centerXAnchor),
            childViewController.view.centerYAnchor.constraint(equalTo: view.centerYAnchor),
            childViewController.view.widthAnchor.constraint(lessThanOrEqualTo: view.widthAnchor, multiplier: 1),
            childViewController.view.heightAnchor.constraint(lessThanOrEqualTo: view.heightAnchor, multiplier: 1)
        ])
    }
}

In AlertContainerViewController, we:

1. Store the alert passed in via the init'er as a property.
2. Embed the alert as a child view-controller.
3. Centre the alert and constrain it to (at maximum) the width and height of the view.

I don't view an alert that is too large for the container as being the presentation layer's problem to solve - if an alert is too large for a single screen, I'd question if it really is an alert.

Let's update HoldingViewController to use AlertContainerViewController:

class HoldingViewController: UIViewController {
    let containerViewController: AlertContainerViewController

    // MARK: - Init

    init(withViewController viewController: UIViewController) {
        // 1
        containerViewController = AlertContainerViewController(withChildViewController: UIViewController)
        super.init(nibName: nil, bundle: nil)
    }

    //Omitted other methods

    // MARK: - ViewLifecycle

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)

        // 2
        present(containerViewController, animated: true, completion: nil)
    }
}

With the changes to HoldingViewController we:

1. Create an AlertContainerViewController instance.
2. Modally present that AlertContainerViewController instance rather than the alert.

Now, if you run the above code with your UIViewController subclass, you would notice that your alert is compressed to be as small as it can be and centred on the screen.

Time to address the second point - Presentation.

Presenting alerts

Let's change the presentation animation to make it feel more like the UIAlertController.

As we move from one view-controller to another, we are used to seeing different types of animation, the most common being:

• Push: the new view slides in from the side on top of the current view.
• Pop: the current view slides out to the side to reveal another view underneath.
• Present: the new view slides up from the bottom on top of the current view.
• Dismiss: the current view slides down to reveal another view underneath.

These view transactions are provided free by iOS. However, it is possible to specify our own view transitions by using the Transitions API. The Transitions API is a suite of protocols that determine how custom transitions should behave. There are quite a few protocols in the suite, however only two of them are of interest to us:

1. UIViewControllerTransitioningDelegate - is a set of methods that vend objects used to manage a fixed-length or interactive transition between view controllers. Every view-controller has a transitioningDelegate which has a type of UIViewControllerTransitioningDelegate. When a transaction is about to happen, iOS asks the transitioning-delegate for an animator to use. If the transitioningDelegate is nil or the necessary UIViewControllerTransitioningDelegate method hasn't been implemented, then iOS will fall back to using the default animation for that type of transition.
2. UIViewControllerAnimatedTransitioning - is a set of methods for implementing the animations for a custom view controller transition. A class that conforms to UIViewControllerAnimatedTransitioning is known as the animator. An animator controls the duration of the transition and allows for manipulating the from-view-controller's view and to-view-controller's view on the transition canvas.

I briefly covered the Transitions API above - if you want to read more, I'd recommend this post.

Let's create an animator to handle presenting the alert:

class CustomAlertPresentAnimationController: NSObject, UIViewControllerAnimatedTransitioning // 1 {

    // MARK: - UIViewControllerAnimatedTransitioning

    // 2
    func transitionDuration(using transitionContext: UIViewControllerContextTransitioning?) -> TimeInterval {
        return 0.2
    }

    // 3
    func animateTransition(using transitionContext: UIViewControllerContextTransitioning) {
        guard let toViewController = transitionContext.viewController(forKey: .to),
            let snapshot = toViewController.view.snapshotView(afterScreenUpdates: true)
            else {
                return
        }

        let containerView = transitionContext.containerView
        let finalFrame = transitionContext.finalFrame(for: toViewController)

        snapshot.frame = finalFrame
        snapshot.transform = CGAffineTransform(scaleX: 1.1, y: 1.1)
        snapshot.alpha = 0.0

        containerView.addSubview(toViewController.view)
        containerView.addSubview(snapshot)
        toViewController.view.isHidden = true

        let duration = transitionDuration(using: transitionContext)

        UIView.animate(withDuration: duration, animations: {
            snapshot.alpha = 1.0
            snapshot.transform = CGAffineTransform(scaleX: 1.0, y: 1.0)
        }) { _ in
            toViewController.view.isHidden = false
            snapshot.removeFromSuperview()
            transitionContext.completeTransition(!transitionContext.transitionWasCancelled)
        }
    }
}

CustomAlertPresentAnimationController can at first glance look a little intimidating, so let's break it down:

1. CustomAlertPresentAnimationController conforms to UIViewControllerAnimatedTransitioning and needs to be a subclass of NSObject as UIViewControllerAnimatedTransitioning extends NSObjectProtocol.
2. In transitionDuration(using:), the duration of the animation is specified as 0.2 seconds - we've copied the timing of an UIAlertController alert here.
3. UIAlertController instances when animated onto the screen, gradually fade in with a slight bounce effect before settling to their actual size. In animateTransition(using:) we use the UIViewControllerContextTransitioning instance (transitionContext) to get the offscreen view of the to-view-controller (i.e. the AlertContainerViewController instance holding our alert). We take a snapshot (screenshot) of the AlertContainerViewController instance's view, add that snapshot to the animator's container view (think of this as a temporary transaction view that is present during the animation) and animate the snapshot on the screen to mimic a UIAlertController animation. Taking a snapshot means we avoid having to deal with any constraint issues that may arise from manipulating the actual AlertContainerViewController instance's view. Once the animation is finished, we remove the snapshot from the view hierarchy and reveal the AlertContainerViewController instance's view occupying the same position.

As HoldingViewController is the view-controller that presents the AlertContainerViewController instance, it needs to conform to UIViewControllerTransitioningDelegate:

class HoldingViewController: UIViewController, UIViewControllerTransitioningDelegate {
    //Omitted properties

    init(withViewController viewController: UIViewController) {
        //Omitted start of method

        // 1
        containerViewController.modalPresentationStyle = .custom

        // 2
        containerViewController.transitioningDelegate = self
    }

    //Omitted other methods

    // MARK: - UIViewControllerTransitioningDelegate

    // 3
    func animationController(forPresented presented: UIViewController, presenting: UIViewController, source: UIViewController) -> UIViewControllerAnimatedTransitioning? {
        return CustomAlertPresentAnimationController()
    }
}

With the changes to HoldingViewController, we:

1. Set the modalPresentationStyle to custom as we will be providing the modal presentation animation.
2. Set this HoldingViewController instance as the transitioningDelegate of the AlertContainerViewController instance.
3. Return an instance of CustomAlertPresentAnimationController when presenting an AlertContainerViewController instance.

If you add in the above code changes to your project and run it, you would now see your alert being animated onto the screen in the same way as an UIAlertController alert.

So we just need to add in a semi-transparent background, and our alert presentation will be complete:

class AlertContainerViewController: UIViewController {
    //Omitted properties and other methods

    override func viewDidLoad() {
        super.viewDidLoad()

        let backgroundView = UIView()
        backgroundView.backgroundColor = UIColor.darkGray.withAlphaComponent(0.75)
        backgroundView.translatesAutoresizingMaskIntoConstraints = false

        view.addSubview(backgroundView)

        //Omitted child view-controller setup

        NSLayoutConstraint.activate([
            //Omitted child view-controller layout setup

            backgroundView.leadingAnchor.constraint(equalTo: view.leadingAnchor),
            backgroundView.topAnchor.constraint(equalTo: view.topAnchor),
            backgroundView.trailingAnchor.constraint(equalTo: view.trailingAnchor),
            backgroundView.bottomAnchor.constraint(equalTo: view.bottomAnchor)
        ])
    }
}

Now our alert presentation is complete; let's turn our attention to dismissing that alert.

Dismissing alerts

Just like presentation, dismissal will happen inside the AlertPresenter:

class AlertPresenter {
    //Omitted properties and other methods

    func dismissAlert(_ viewController: UIViewController) {
        // 1
        guard let alertWindow = alertWindows.first(where: { $0.viewController == viewController } )  else {
            return
        }

        os_log(.info, "Alert being dismissed")

        // 2
        alertWindow.dismiss { [weak self] in
            // 3
            self?.alertWindows.remove(alertWindow)
        }
    }
}

With the changes to AlertPresenter, we:

1. Find the window that is showing the alert.
2. Call dismiss(completion:) on that window to begin the dismissal process.
3. Remove the window from alertWindows once the dismissal has completed.

There are a few ways we could have implemented the dismissal logic. In the end, I decided to require all custom alerts to call dismissAlert(_:) when they are ready to be dismissed. This direct calling approach has symmetry with how the alert is presented and is very simple.

If you try to run the above method, you will get an error because AlertWindow doesn't yet have an viewController property or dismissAlert(_:) method, so let's add them in:

class AlertWindow: UIWindow {
    // 1
    var viewController: UIViewController {
        return holdingViewController.containerViewController.childViewController
    }

    //Omitted other methods and properties

    // MARK: - Dismiss

    func dismiss(completion: @escaping (() -> Void)) {
        // 2
        holdingViewController.dismissAlert { [weak self] in
            // 3
            self?.resignKeyAndHide()
            completion()
        }
    }

    // MARK: - Resign

    private func resignKeyAndHide() {
        resignKey()
        isHidden = true
    }
}

With the above changes to AlertWindow, we:

1. Added a new viewController property that gets the alert being displayed and returns it.
2. Pass the dismissal command onto the HoldingViewController instance.
3. Resign the window and hide it once the dismissal has completed.

Normally, I don't like the level of method chaining shown inside the viewController property. However, I feel comfortable doing it here as HoldingViewController and AlertContainerViewController are private implementation details of AlertWindow.

Let's add a dismissAlert(_:) method into HoldingViewController:

class HoldingViewController: UIViewController, UIViewControllerTransitioningDelegate {
    //Omitted properties and other methods

    // MARK: - Dismiss

    func dismissAlert(completion: @escaping (() -> Void)) {
        containerViewController.dismiss(animated: true, completion: {
            completion()
        })
    }
}

With the above change, we call the standard UIKit dismiss(animation:completion:) on the containerViewController and trigger the completion closure when that dismiss action completes.

If you add the above code changes to your project and run it by calling AlertPresenter.shared.dismiss(completion:) when your alert's dismissal button is pressed, then your alert should be dismissed. However, it will still be using the standard modal dismissal animation. Just like with presenting, dismissing will need an animator:

class CustomAlertDismissAnimationController: NSObject, UIViewControllerAnimatedTransitioning {

    // MARK: - UIViewControllerAnimatedTransitioning

    func transitionDuration(using transitionContext: UIViewControllerContextTransitioning?) -> TimeInterval {
        return 0.2
    }

    func animateTransition(using transitionContext: UIViewControllerContextTransitioning) {
        guard let fromViewController = transitionContext.viewController(forKey: .from),
            let snapshot = fromViewController.view.snapshotView(afterScreenUpdates: true)
            else {
                return
        }

        let containerView = transitionContext.containerView
        let finalFrame = transitionContext.finalFrame(for: fromViewController)

        snapshot.frame = finalFrame

        containerView.addSubview(snapshot)
        fromViewController.view.isHidden = true

        let duration = transitionDuration(using: transitionContext)

        UIView.animate(withDuration: duration, animations: {
            snapshot.alpha = 0.0
        }) { _ in
            snapshot.removeFromSuperview()
            transitionContext.completeTransition(!transitionContext.transitionWasCancelled)
        }
    }
}

CustomAlertDismissAnimationController is mostly the opposite of CustomAlertPresentAnimationController but without the bounce effect.

Now HoldingViewController just has to return an CustomAlertDismissAnimationController instance at the appropriate moment:

class HoldingViewController: UIViewController, UIViewControllerTransitioningDelegate {
    //Omitted properties and other methods

    func animationController(forDismissed dismissed: UIViewController) -> UIViewControllerAnimatedTransitioning? {
        return CustomAlertDismissAnimationController()
    }
}

And that completes our custom alert presenter.

🍾💃🏻🕺🏻

Feeling hopeful? 🌞

To recap, we built a simple custom alert presentation system that takes an alert view-controller, places it in the centre of a semi-transparent container and presents it in a dedicated window, all the while doing so with the same feel as presenting a UIAlertController instance.

I spent quite a long time toying with the idea of having AlertPresenter accept view-models rather than view-controllers (with the view-model then being transformed into view-controllers inside the presenter). However, the view-model solution always ended up feeling very confused - was AlertPresenter part of the app's infrastructure or part of the app's UI? By using view-models AlertPresenter had to be both 😧. Each time someone created a new type of alert, AlertPresenter would need to be modified to know about the new view-model - breaking the open/closed principle and opening the door to unexpected consequences rippling through the app (you can see this approach on this branch) via the AlertPresenter. By moving the alert view-controller creation from AlertPresenter to the component that wanted to use AlertPresenter, I was able to give AlertPresenter a clear purpose: AlertPresenter takes a view-controller and presents it in the same manner as an UIAlertController alert. This clear division of responsibility between alert creator and alert presenter, I believe, has meant that AlertPresenter is a simple, easy-to-understand component that should very rarely need to be modified.

To see the above code snippets together in a working example, head over to the repository and clone the project.

Being British, queuing is essential to me. I take every opportunity I can get to queue: posting a package ✅, paying for my groceries ✅, fleeing a burning building ✅, etc. So every time I need to present a sequence of alerts, I come up against the uncomfortable truth that UIAlertController doesn't care as much about queuing as I do 😔.

This post is a look at how UIAlertController can be made a little more British through embracing a strong queuing etiquette.

Let's get queuing 🇬🇧

A queue is a first-in, first-out data structure so that the oldest element is the first to be removed. Swift doesn't come with a built-in queue type, so let's build one:

struct Queue<Element> {
    private var elements = [Element]()

    // MARK: - Operations

    mutating func enqueue(_ element: Element) {
        elements.append(element)
    }

    mutating func dequeue() -> Element? {
        guard !elements.isEmpty else {
            return nil
        }

        return elements.removeFirst()
    }
}

The above class is a relatively trivial generic queue implementation that allows only two operations - enqueue and dequeue.

When queuing alerts, it's important to ensure that all alerts are queued on the same instance. To achieve this, we need a specialised singleton class that will control access to the queue and be the central component for all alert presentation logic:

class AlertPresenter {
    private var alertQueue = Queue<UIAlertController>()

    static let shared = AlertPresenter()

    // MARK: - Present

    func enqueueAlertForPresentation(_ alertController: UIAlertController) {
        alertQueue.enqueue(alertController)

        //TODO: Present
    }
}

In the above class, AlertPresenter holds a private queue specialised for UIAlertController elements and a method for enqueuing alerts. But as of yet, no way to present the queued alert. Before we can implement a presentation method, let's look at how alerts are normally presented:

let alertController = ...

present(alertController, animated: true, completion: nil)

UIAlertController (which is a subclass of UIViewController) is presented modally like any other view-controller by calling present(_:animated:completion:) on the presenting view-controller. A consequence of queuing alerts in AlertPresenter is that it breaks the usual alert presentation flow. As AlertPresenter isn't a subclass of UIViewController, we can't use it directly for presenting. Instead, we need to get a view-controller to present from. As well as requiring a view-controller to be injected into AlertPresenter, our indirect alert presentation also exacerbates an existing subtle issue with presenting alerts - simultaneous navigation events (presentation/dismissal) occurring at the same time, resulting in one of those events being cancelled and the following error being generated:

Without AlertPresenter being involved, the alert would be presented directly from the view-controller, allowing that view-controller to prevent other navigation events from occurring until after the alert is dismissed. However, by queuing the alert, the view-controller has no way of knowing when it will be shown, so no way of knowing when it should or shouldn't prevent other navigation events.

While it's possible for AlertPresenter to query the topmost view-controller and determine if it is in the process of presenting or being dismissed, doing so requires logic that gets messy quickly 🤢. Instead of having to accommodate events happening on the navigation stack that AlertPresenter doesn't control, we can raise AlertPresenter above all navigation concerns by using a dedicated UIWindow instance (with a dedicated view-controller) to present the queued alerts from. As the navigation stack in one window is independent of any other window's navigation stack, an alert can be presented on its window at the same time as a view controller is being pushed on its window without navigation collisions 🥳.

Before delving into how to use a dedicated window to present alerts, let's get to know windows better.

If you are comfortable with how UIWindow works, feel free to skip ahead.

Getting to know windows 💭

UIWindow is a subclass of UIView that acts as the container for an app's visible content - it is the top of the view hierarchy. All views that are displayed to the user need to be added to a window. An app can have multiple windows, but only windows that are visible can have their content displayed to the user - by default, windows are not visible. Multiple windows can be visible at once. Each window's UI stack is independent of other windows UI stacks. Where a window is displayed in relation to other visible windows is controlled by setting that window's windowLevel property. The higher the windowLevel, the nearer to the user that window is. UIWindow has three default levels:

1. .normal
2. .statusBar
3. .alert

With .alert > .statusBar > .normal. If a more fine-grain level of control is needed, it's possible to use a custom level:

window.windowLevel = .normal + 25

As of iOS 13: .alert has a raw value of 2000, .statusBar has a raw value of 1000 and .normal has a raw value of 0.

Where two or more windows have the same level, their ordering is determined by the order they were made visible in - the last window made visible is nearest one to the user.

It's unusual to add subviews to a window directly; instead, each window should have a root view-controller whose view is used as the window's initial subview.

As well as displaying content, UIWindow is also responsible for forwarding any events (touch, motion, remote-control or press) to interested parties in its responder chain. While all touch events are forwarded to the responder chain of the window that the event occurred on, events that are outside of the app's UI, such as motion events or keyboard entry, are forwarded to the key window. Only one window can be key at any one given time (which window is key can change throughout the lifetime of the app).

An iOS app needs at least one window; a reference to this window can be found in the app delegate:

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?

    //Omitted methods
}

If you're using storyboards to lay out your interface, then most of the work of setting up this window is happening under the hood. When the app is launched, a UIWindow instance is created that fills the screen. This window is then assigned to the window property (declared in the UIApplicationDelegate protocol), configured with the view-controller declared as the storyboard entry point from the project's main storyboard as its rootViewController, and finally, the window is made key and visible.

If you are not using storyboards, you can better see this setup:

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?

    // MARK - AppLifecycle

    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        window = UIWindow()
        window?.rootViewController = YourViewController()
        window?.makeKeyAndVisible()

        return true
    }
}

Using windows

As this new window will only display alerts, let's subclass UIWindow for this single purpose:

class AlertWindow: UIWindow {
    // MARK: - Init

    init(withAlertController alertController: UIAlertController) {
        super.init(frame: UIScreen.main.bounds)

        rootViewController = alertController

        windowLevel = .alert
    }

    @available(*, unavailable)
    required init?(coder aDecoder: NSCoder) {
        fatalError("Unavailable")
    }

    // MARK: - Present

    func present() {
        makeKeyAndVisible()
    }
}

In the above AlertWindow class, an alert is passed into the init'er and set to the window's rootViewController, the window is then configured to have a .alert window level - this will put it above the app's main window (which by default has a .normal window level).

If you create an instance of AlertWindow and make it visible, you will notice that it's not being presented with an animation - it just appears, which feels weird. A window's root view-controller can not be animated on-screen, so to keep the alert's animation, we need that alert to be presented from an intermediate view-controller, which can be the window's root view-controller:

class HoldingViewController: UIViewController {
    private let alertController: UIAlertController

    // MARK: - Init

    init(withAlertController alertController: UIAlertController) {
        self.alertController = alertController
        super.init(nibName: nil, bundle: nil)
    }

    required init?(coder aDecoder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    // MARK: - ViewLifecycle

    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = .clear
    }

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)

        present(alertController, animated: true, completion: nil)
    }
}

The above HoldingViewController class only has one responsibility - presenting an alert once viewDidAppear(_:) has been called.

Trying to present an alert earlier will cause an animation error due to the HoldingViewController instance not having finished its own "animation" on screen. While (at the time of writing - iOS 13) this doesn't seem to affect the actual presentation of the alert, waiting for HoldingViewController to be presented before attempting another presentation ensures that the error isn't produced.

Let's use HoldingViewController:

class AlertWindow: UIWindow {

    // MARK: - Init

    init(withAlertController alertController: UIAlertController) {
        super.init(frame: UIScreen.main.bounds)

        rootViewController = HoldingViewController(withAlertController: alertController)

        windowLevel = .alert
    }

    //Omitted methods
}

In the above class, instead of the UIAlertController instance being set directly as the rootViewController it is used to create an HoldingViewController instance, which is set as the window's rootViewController.

Each AlertWindow instance is intended to be single-use - to present one alert. This single-use nature allows for its mere presence to be the determining factor in whether a queued alert should be presented or not:

class AlertPresenter {
    //Omitted properties

    private var alertWindow: AlertWindow?

    // MARK: - Present

    func enqueueAlertForPresentation(_ alertController: UIAlertController) {
        alertQueue.enqueue(alertController)

        showNextAlertIfPresent()
    }

    private func showNextAlertIfPresent() {
        guard alertWindow == nil,
            let alertController = alertQueue.dequeue() else {
                return
        }

        let alertWindow = AlertWindow(withAlertController: alertController)
        alertWindow.present()

        self.alertWindow = alertWindow
    }
}

With the above changes, alertWindow holds a reference to the window that is being used to present the alert. Inside showNextAlertIfPresent() if alertWindow is nil and there is a queued alert, then a new AlertWindow instance is created, presented and assigned to alertWindow. Making the window key and visible sets off the chain of activity that results in the alert being animated on screen.

The example so far, while functional, is limited - it can present only one alert. AlertPresenter needs to be informed when an alert has been dismissed so it can move on to the next alert.

It turns out knowing when an alert has been dismissed is one of the trickier things to know about in an iOS app 😕. I had to work through a number of different solutions before I got somewhere I was happy:

1. My first attempt involved subclassing UIAlertController and overriding viewDidDisappear(_:) with a call to AlertPresenter - everyone would then use this subclass instead of UIAlertController directly. However, this approach is explicitly warned against in the documentation for UIAlertController:

👎

2. My second attempt involved requiring each developer to explicitly call AlertPresenter from inside each UIAlertAction closure. However, this approach puts too much of a burden on each developer to remember to include those calls in each and every action. A missed call from any action closure (that was substantially triggered) would cause the queue to be jammed from that alert until the app was killed. This is too easy a requirement to forget when writing or reviewing 👎.

3. My third attempt involved using a view model to abstract the creation of UIAlertController instances to a factory method where a call to AlertPresenter could then be injected into each UIAlertAction closure (during the conversion from view model to UIAlertController instance). However, this approach would require a lot of custom code to be written, maintained and tested - using UIAlertController directly would avoid all effort 👎.

Tricky, tricky 🤔.

Instead of thinking about how to get UIAlertController to tell us about what was happening to it, I decided to start thinking about what impact UIAlertController had on its surroundings. UIViewController has a great suite of methods for when it will appear and disappear - these appearance events can tell us what has happened to the presented alert. When an alert is dismissed, it calls func dismiss(animated:completion:) on the view-controller it was presented from. We can hook into this behaviour to inform AlertWindow about the dismissal:

protocol HoldingDelegate: class {
    func viewController(_ viewController: HoldingViewController, didDismissAlert alertController: UIAlertController)
}

class HoldingViewController: UIViewController {
    //Omitted properties

    weak var delegate: HoldingDelegate?

    // Omitted methods

    override func dismiss(animated flag: Bool, completion: (() -> Void)? = nil) {
        //Called when a UIAlertController instance is dismissed
        super.dismiss(animated: flag) {
            completion?()

            self.delegate?.viewController(self, didDismissAlert: self.alertController)
        }
    }
}

👍

You may be thinking: "Why is func dismiss(animated:completion:) being called on a view-controller that isn't being dismissed instead of viewDidAppear(_:) as it is actually reappearing"?. The same thought occurred to me, and this mismatch between expectations vs reality left me a little uncomfortable 😒. However, in the end, I decided that being able to use UIAlertController without having to care about the queue outweighed my discomfort. If Apple changes this to better match the behaviour of other UIViewController instances, modifying this solution to handle both cases will be trivial.

When an alert has been dismissed, the presenting AlertWindow instance has served its purpose and can itself be dismissed:

class AlertWindow: UIWindow, HoldingDelegate {
      //Omitted properties

      // MARK: - Init

      init(withAlertController alertController: UIAlertController) {
          super.init(frame: UIScreen.main.bounds)

          let holdingViewController = HoldingViewController(withAlertController: alertController)
          holdingViewController.delegate = self

          rootViewController = holdingViewController

          windowLevel = .alert
      }

      //Omitted other methods

      // MARK: - HoldingDelegate

      func viewController(_ viewController: HoldingViewController, didDismissAlert alertController: UIAlertController) {
          resignKeyAndHide()
      }

      // MARK: - Resign

      private func resignKeyAndHide() {
          resignKey()
          isHidden = true
      }

}

All that is left to do is inform AlertPresenter that the alert has been dismissed, again the delegation pattern can be used here:

protocol AlertWindowDelegate: class {
    func alertWindow(_ alertWindow: AlertWindow, didDismissAlert alertController: UIAlertController)
}

class AlertWindow: UIWindow, HoldingDelegate {
    weak var delegate: AlertWindowDelegate?

    //Omitted properties and methods

    func viewController(_ viewController: HoldingViewController, didDismissAlert alertController: UIAlertController) {
        resignKeyAndHide()
        delegate?.alertWindow(self, didDismissAlert: alertController)
    }

    //Omitted methods
}

AlertPresenter then just needs to present the next alert (if present):

class AlertPresenter: AlertWindowDelegate {
    //Omitted properties and methods

    private func showNextAlertIfPresent() {
        guard alertWindow == nil,
            let alertController = alertQueue.dequeue() else {
                return
        }

        let alertWindow = AlertWindow(withAlertController: alertController)
        alertWindow.delegate = self

        alertWindow.present()

        self.alertWindow = alertWindow
    }

    // MARK: - AlertWindowDelegate

    func alertWindow(_ alertWindow: AlertWindow, didDismissAlert alertController: UIAlertController) {
        self.alertWindow = nil
        showNextAlertIfPresent()
    }
}

All queued out 🍾

Congratulations on having made it to the end of a post about queuing. You've shown a level of patience that even many Brits would struggle to achieve.

To recap, in this post, we implemented a minimally intrusive alert queuing mechanism that allows us to continue using UIAlertController without having to ask the creators of those alerts to do anything more complicated than calling AlertPresenter.shared.enqueueAlertForPresentation(_:).

To see the above code snippets together in a working example, head over to the repository and clone the project.

Many of us feel nervous when meeting a group of people for the first time. What are the dynamics of the group, what are the in-jokes, will I find common ground with someone - are just a few questions that can plague you. A lot of your hard work at suppressing your crippling social self-doubt can unravel with a shaky introduction or misplaced comment. You may even find yourself wondering:

"Are they laughing with me because I'm funny or at me because of the overly pointy brogues that I'm wearing?"

Of course, that's nonsense. Pointy brogues are the height of fashion 👞.

It's not just you who can be anxious about new introductions; your app can be too. Computers are notoriously fickle about who they speak to and how they expect to be addressed. A barrage of errors awaits any app that breaks these social norms - it can be enough to make your app want to stay quietly in the corner rather than jump in and face potential rejection.

Thankfully, this social anxiety can be eased if your app already knows someone who understands the dynamics of the group - someone cool and suave like SSDP.

This post will explore how to build a solution that allows us to send an SSDP message onto the network to discover if the device we want to connect with is present. And importantly, why we shouldn't trust every device that responds.

This post will gradually build up to a working example however, if you want to jump ahead, then head on over to the completed example and take a look at SSDPSearchSessionConfiguration, SSDPSearchSession, UDPSocketController, UDPSocket, SSDPServiceParser and SSDPService to see how things end up.

Getting to know SSDP

SSDP (Simple Service Discovery Protocol) is a discovery protocol used to determine what services are available on a network. It is defined as part of the UPnP spec. SSDP is a zero-configuration networking protocol designed to allow nodes to be added and removed from a network without any involvement from a central service such as DNS or by assigning static IP addresses to specific nodes. This decentralised, dynamic approach is possible because SSDP uses UDP as its underlying transportation protocol, which allows for multicast communication.

Multicasting allows a node to transmit one message onto the network and for that message to be forwarded onto all interested nodes on the network without the sender node having to know what other network nodes are available (forwarding happens at the IP routing level). SSDP takes advantage of this forwarding functionality to allow any node to ask other nodes if they support a particular service, or conversely, for a node which offers services to tell other nodes about those services.

For multicast messages, IANA has reserved the IPv4 address 239.255.255.250 and port 1900 for SSDP.

SSDP messages conform to the header field format of HTTP 1.1. It's important to note that SSDP does not allow any message to contain a body; everything is shared via those header fields.

An SSDP node is either a root device or control point. A root device offers one or more SSDP services; a control point uses SSDP services.

When a root device responds to a discovery message, it does so by sending a unicast (to a single specific node) message directly to the control point.

That's a lot of information to take in 😥. If it doesn't all make sense, that's ok, most of the details will come up again later, and when seen in context, those details are easier to understand.

SSDP messages fall into two categories:

1. Discovery
2. Advertisement

This post is mainly concerned with Discovery messages, but to ensure that we have the fullest possible understanding of SSDP, I will also cover Advertisement messages (feel free to skip the Advertisement section).

Discovery

Discovery involves two message types:

1. Request
2. Response

A request message is when a control point transmits an M-SEARCH message onto the network looking for a particular service (or as we shall see soon, any service), e.g.

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 1
ST: urn:dial-multiscreen-org:service:dial:1

An M-SEARCH request message contains:

• The host and port (HOST) the message will be sent to. Typically, an M-Search message is multicast (like the example above), but it can also be unicast.
• The message type (MAN), for an M-Search, is always ssdp:discover.
• The search target (ST) of the service the search request is attempting to discover.
• The maximum wait response time (MX) in seconds that a root device can take before responding. The MX field is an attempt to overcome a scaling issue implicit with SSDP. SSDP is a chatty protocol, in a network with a significant number of nodes that host SSDP services, sending an M-SEARCH message could result in accidentally DDOS-ing the questing node due to too many services responding at once. The MX field instructs the root device to wait a random time between 0 and MX before attempting to respond - this should allow the responses to be spaced out enough to ease the processing strain on the control point. The MX value should be between 1 and 5. Even with the MX workaround, SSDP is recommended for use only in home or small office networks.

A root device should only respond with services that match the search target field of the request, e.g.

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=3600
ST: urn:dial-multiscreen-org:service:dial:1
USN: uuid:0175c106-5400-10f8-802d-b0a7374360b7::urn:dial-multiscreen-org:service:dial:1
EXT:
SERVER: Roku UPnP/1.0 MiniUPnPd/1.4
LOCATION: http://192.168.1.104:8060/dial/dd.xml

An M-Search response message contains:

• The cache-control (CACHE-CONTROL) value to determine for how long the message is valid.
• The search target (ST) of the service that is responding. ST should be common across all devices of this type.
• The unique service name (USN) to identify the service.
• The server system information (SERVER) value providing information in the following format: [OS-Name] UPnP/[Version] [Product-Name]/[Product-Version].
• The location URL (LOCATION) to allow the control point to gain more information about this service.

The EXT field is required for backwards compatibility with UPnP 1.0, but can otherwise be ignored.

Advertisement

An advertisement message is when a root device shares the status of each service it offers with the other nodes on the network.

There are three types of advertisement:

1. Alive
2. Update
3. ByeBye

An alive message allows interested devices to know that a service is available. An alive message is a multicast NOTIFY message e.g.

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
CACHE-CONTROL: max-age=3600
NT: urn:dial-multiscreen-org:service:dial:1
NTS: ssdp:alive
LOCATION: http://192.168.1.104:8060/dial/dd.xml
USN: uuid:0175c106-5400-10f8-802d-b0a7374360b7::urn:dial-multiscreen-org:service:dial:1

An alive message contains:

• The host and port (HOST) the message will be sent to.
• The cache-control (CACHE-CONTROL) value to determine for how long the message is valid.
• The notification type (NT) that defines the service it offers (the equivalent of ST in an M-Search message).
• The notification subtype (NTS), for an alive message this will always be ssdp:alive (the equivalent of MAN in an M-Search message).
• The location URL (LOCATION) to allow a receiving control point to gain more information about this service.
• The unique service name (USN) to identify the service.

An update message allows changes to a service to be shared. An update message is also a multicast NOTIFY message like the alive message e.g.

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
NT: urn:dial-multiscreen-org:service:dial:1
NTS: ssdp:update
LOCATION: http://192.168.1.160:8060/dial/dd.xml
USN: uuid:0175c106-5400-10f8-802d-b0a7374360b7::urn:dial-multiscreen-org:service:dial:1

An update message has the same header fields as an alive message, with only the NTS value differing between them.

A byebye message allows any interested nodes to know when a service is about to be removed from the network. A byebye message should be sent for each valid (non-expired) alive message that was sent. A byebye message is a multicast NOTIFY message, e.g.

NOTIFY * HTTP/1.1
HOST: 239.255.255.250:1900
NT: urn:dial-multiscreen-org:service:dial:1
NTS: ssdp:byebye
USN: uuid:0175c106-5400-10f8-802d-b0a7374360b7::urn:dial-multiscreen-org:service:dial:1

Again, a byebye message has a very similar structure to both an alive and update message, except it omits the LOCATION header field and has a different NTS value.

Now that we have an understanding of what SSDP is, let's get back to the solution.

Getting to know who's there 🔭

As SSDP communication is built on top of UDP, it isn't possible to use URLSession to send an M-Search message; instead, we need to read and write from a socket manually. I was tempted to dive in and write a simple socket layer to handle this, but the more I read up about the various network setups I would have to support, the more writing a socket layer started to look like an unforgiving task. Instead, I decided to introduce a 3rd-party dependency into the project: BlueSocket. BlueSocket will handle the nitty-gritty socket communication, freeing us up to focus on sending and receiving SSDP messages.

In the example project, CocoaPods is used to manage this dependency.

At the time of writing (April 2019), the new(ish) Network framework doesn't support UDP multicasting.

Since I wrote this article, Apple has released iOS 14, which contained a number of user privacy improvements. One area of focus was on how an app accesses the local network. There are two changes that affect the below example. Firstly, before accessing the local network, iOS will request permission from the user as it does for, e.g. accessing the camera. Secondly, any app that wants to multicast on a network must also have the com.apple.developer.networking.multicast entitlement enabled - this entitlement needs to be requested from Apple. Without both the user permission and the entitlement, the below example won't be able to send multicast messages from a device (everything will still work on the simulator) - see this note for more details.

Before we get started, let's look at what we are going to build:

• SSDPSearchSessionConfiguration represents the configuration of an SSDP search session.
• SSDPSearchSession is responsible for triggering a search request, parsing and filtering received responses and passing back those services that match our search terms to whoever requested the search. A typical search session will write multiple search requests to the socket to try to overcome the unreliable nature of UDP.
• SSDPSearchSessionDelegate is a protocol that informs the delegator of any found services, any errors encountered and ultimately that the search session has ended.
• SocketControllerFactory is responsible for producing socket controllers configured to use sockets that use a specific transport protocol.
• UDPSocketController is a wrapper around a Socket instance configured to use UDP. UDPSocketController helps to hide the messiness of socket communication.
• UDPSocketControllerDelegate is a protocol that informs the delegator of any responses received and any errors encountered.
• SocketFactory is responsible for producing fully configured sockets that are ready to be written/read to/from.
• UDPSocket is a Socket instance configured to use UDP. As Socket is from the BlueSocket dependency, Socket is wrapped in a protocol to limit the spread of BlueSocket in the project.
• SSDPServiceParser is responsible for parsing an SSDP response into Service instance. If the SSDP response is invalid, no service is created.
• SSDPService represents an SSDP service.

Don't worry if that doesn't all make sense yet; we will look into each class in greater depth below.

Now that we know where we are going, let's start at the bottom with UDPSocket and work our way up.

Ideally, when adding a 3rd-party dependency to our projects, we want to hide that dependency behind a facade, which should make removing that dependency easier by limiting its spread in the project. In Swift, we can use a protocol and an extension to wrap that 3rd-party dependency inside our facade:

enum UDPSocketError: Error {
    case addressCreationFailure
    case writeError(underlayingError: Error)
    case readError(underlayingError: Error)
}

protocol UDPSocketProtocol {
    func write(_ string: String, to host: String, on port: UInt) throws
    func readDatagram(into data: inout Data) throws
    func close()
}

extension Socket: UDPSocketProtocol {
    // 1
    func write(_ string: String, to host: String, on port: UInt) throws {
        guard let signature = self.signature, signature.socketType == .datagram, signature.proto == .udp else {
            fatalError("Only UDP sockets can use this method")
        }

        guard let address = Socket.createAddress(for: host, on: Int32(port)) else {
            throw(UDPSocketError.addressCreationFailure)
        }
        do {
            try write(from: string, to: address)
        } catch {
            throw(UDPSocketError.writeError(underlayingError: error))
        }
    }

    // 2
    func readDatagram(into data: inout Data) throws {
        guard let signature = self.signature, signature.socketType == .datagram, signature.proto == .udp else {
            fatalError("Only UDP sockets can use this method")
        }

        do {
            let (_,_) = try readDatagram(into: &data)
        } catch {
            throw(UDPSocketError.readError(underlayingError: error))
        }
    }
}

Socket does a lot more than what we need for our SSDP solution. UDPSocketProtocol reduces the range of tasks that can be performed on a Socket instance to only the three that we need: write, read and close. As well as reducing scope, UDPSocketProtocol also simplifies the interface of Socket by wrapping the Socket methods in our own methods.

1. The Socket type in BlueSocket can be used to create sockets with different configurations. In UDPSocketProtocol, we only want to cater for sockets configured that using UDP to send datagram messages, so in write(_:to:on:) a check is made to determine if this Socket instance has been configured for UDP and datagram - if it hasn't then a fatal error is thrown as this is a developer error. The write(from:to:) method on Socket takes an Address type which is defined in Socket - as mentioned above we want to limit the spread of BlueSocket in the project, so the write method defined in UDPSocketProtocol doesn't use Address but rather sticks with two String parameters host and port. In the extension, these parameters are used to create an Address instance, which is then forwarded to the write(from:to:) method on Socket. If, for some reason, an Address instance can't be created, an exception is thrown. It's also possible for an exception to be thrown when writing to the socket; if an exception is thrown, that exception is caught, wrapped inside an UDPSocketError case, before a new exception is thrown.
2. Just like with write(_:to:on:), in readDatagram(into:), a check is made to ensure that we are dealing with a socket configured for UDP and datagrams. The readDatagram(into:) method on Socket returns the number of bytes read as an Int and the address that sent those bytes as an Address. We aren't interested in either of those return values, so the readDatagram(into:) defined in SocketProtocol doesn't have a return type; our readDatagram(into:) simply ignores those details. If, for some reason, an exception is thrown when reading from the socket, this exception is caught, wrapped inside a UDPSocketError case, and then a new exception is thrown.

The close() on Socket fits our needs perfectly, so we don't need to wrap that method.

UDPSocketProtocol does a good job of hiding Socket, but it can be called on with all configurations of Socket when we really only intend for it to be called on UDP sockets sending datagram messages, so let's make making that socket configuration as easy as possible:

extension Socket {
    static func createUDPSocket() throws -> UDPSocketProtocol {
        return try Socket.create(type: .datagram, proto: .udp)
    }
}

You will need to import the Socket module into any class that uses Socket.

Now that we have UDPSocketProtocol, let's build Socket instances wrapped in it:

protocol SocketFactoryProtocol {
    func createUDPSocket() -> UDPSocketProtocol?
}

class SocketFactory: SocketFactoryProtocol {

    // MARK: - UDP

    func createUDPSocket() -> UDPSocketProtocol? {
        guard let socket = try? Socket.createUDPSocket() else {
            return nil
        }

        return socket
    }
}

The above factory attempts to create a UDP socket; if successful, that Socket instance is returned hidden behind the UDPSocketProtocol protocol; if unsuccessful, nil is returned.

SocketFactory conforms to SocketFactoryProtocol so that when testing UDPSocketController, we can replace the SocketFactory instance with a mock factory that conforms to that protocol. I use this technique throughout this example, so any other protocols named after a concrete type serve this purpose.

Now that we have our socket, let's try writing to it:

protocol UDPSocketControllerProtocol: AnyObject {
    var state: UDPSocketControllerState { get }

    func write(message: String)
    func close()
}

// 1
enum UDPSocketControllerState {
    case ready
    case active
    case closed

    var isReady: Bool {
        self == .ready
    }

    var isActive: Bool {
        self == .active
    }

    var isClosed: Bool {
        self == .closed
    }
}

class UDPSocketController: UDPSocketControllerProtocol {
    private(set) var state: UDPSocketControllerState = .ready

    private let socket: UDPSocketProtocol

    private let host: String
    private let port: UInt

    private let callbackQueue: OperationQueue
    private let socketWriterQueue = DispatchQueue(label: "com.williamboles.udpsocket.writer.queue",  attributes: .concurrent)

    // MARK: - Init

    // 2
    init?(host: String, port: UInt, socketFactory: SocketFactoryProtocol) {
        guard let socket = socketFactory.createUDPSocket() else {
            return nil
        }

        self.host = host
        self.port = port
        self.socket = socket
    }

    // MARK: - Write

    // 3
    func write(message: String) {
        guard !state.isClosed else {
            os_log(.info, "Attempting to write to a closed socket")
            return
        }

        state = .active

        write(message: message, on: socketWriterQueue)
    }

    // 4
    private func write(message: String, on queue: DispatchQueue) {
        queue.async {
            do {
                try self.socket.write(message, to: self.host, on: self.port)
            } catch {
                self.closeAndReportError(error)
            }
        }
    }

    // MARK: - Close

    private func closeAndReportError(_ error: Error) {
        close()
        os_log(.info, "Error received: \r%{public}@", error)
        //TODO: Implement reporting error
    }

    func close() {
        state = .closed
        socket.close()
    }
}

Let's look at what we did above:

1. A socket can be in 3 states: ready, active and closed - UDPSocketControllerState represents these states. As UDPSocketController interacts with its socket, it will move through states. This will ensure that we don't try to write to a socket that can't be written to.
2. To write a message to a socket, we need the host and port on the remote machine that we want to communicate with. The init'er of UDPSocketController accepts this host and port. It also accepts a socket factory instance, which is used to create a UDP socket.
3. When writing to a socket, we only want to allow communication with a socket that hasn't been closed, so the first action of the write(message:) is to check if state is in a closed state.
4. Writing to a socket can be a time-consuming operation, so the operation is pushed onto a background queue: socketWriterQueue. As a write operation can throw an exception, we wrap that operation in a do...catch block. If an exception is caught, the socket is closed, and any error is reported (we will implement this shortly).

UDPSocketController is designed to be tied to one host and port. If the host and port need to be changed, then a new instance of UDPSocketController must be created.

Let's build a factory to produce socket controllers:

protocol SocketControllerFactoryProtocol {
    func createUDPSocketController(host: String, port: UInt, socketFactory: SocketFactoryProtocol) -> UDPSocketControllerProtocol?
}

class SocketControllerFactory: SocketControllerFactoryProtocol {

    // MARK: - UDP

    func createUDPSocketController(host: String, port: UInt, socketFactory: SocketFactoryProtocol) -> UDPSocketControllerProtocol? {
        UDPSocketController(host: host, port: port, socketFactory: socketFactory)
    }
}

Now that we have a socket controller and a socket to write to let's look at how to give it an M-Search message to send.

3 pieces of information are required to send an M-Search message:

1. Search target (ST).
2. The IP address and port (HOST).
3. Maximum wait response time (MX).

These pieces of information can be represented as:

struct SSDPSearchSessionConfiguration {
    let searchTarget: String
    let host: String
    let port: UInt
    let maximumWaitResponseTime: TimeInterval

    // MARK: - Init

    init(searchTarget: String, host: String, port: UInt, maximumWaitResponseTime: TimeInterval) {
        assert(maximumWaitResponseTime >= 1 && maximumWaitResponseTime <= 5, "maximumWaitResponseTime should be between 1 and 5 (inclusive)")

        self.searchTarget = searchTarget
        self.host = host
        self.port = port
        self.maximumWaitResponseTime = maximumWaitResponseTime
    }
}

SSDPSearchSessionConfiguration has a custom initialiser to allow for an assertion to be performed on the value of maximumWaitResponseTime (which needs to be between 1 and 5 inclusive). Having to write this custom initialiser manually is a price worth paying to allow for quicker feedback during development if an invalid value is passed in (by causing the app to crash).

While it's possible to send unicast M-Search messages, here we are only interested in sending multicast messages, so to make things easier, let's add a small factory method to return a preconfigured multicast SSDPSearchSessionConfiguration instance:

extension SSDPSearchSessionConfiguration {

    static func createMulticastConfiguration(forSearchTarget searchTarget: String, maximumWaitResponseTime: TimeInterval = 3) -> SSDPSearchSessionConfiguration {
        let configuration = SSDPSearchSessionConfiguration(searchTarget: searchTarget, host: "239.255.255.250", port: 1900, maximumWaitResponseTime: maximumWaitResponseTime)

        return configuration
    }
}

Setting the searchTarget to ssdp:all should cause all root devices to respond with their full range of SSDP services.

With this configuration it is possible to build a simple searcher class to drive any UDPSocketController instance:

protocol SSDPSearchSessionProtocol {
    func startSearch()
    func stopSearch()
}

class SSDPSearchSession: SSDPSearchSessionProtocol {
    private let socketController: UDPSocketControllerProtocol
    private let configuration: SSDPSearchSessionConfiguration

    // 1
    private lazy var mSearchMessage = {
        // Each line must end in `\r\n`
        return "M-SEARCH * HTTP/1.1\r\n" +
            "HOST: \(configuration.host):\(configuration.port)\r\n" +
            "MAN: \"ssdp:discover\"\r\n" +
            "ST: \(configuration.searchTarget)\r\n" +
            "MX: \(Int(configuration.maximumWaitResponseTime))\r\n" +
        "\r\n"
    }()

    // MARK: - Init

    // 2
    init?(configuration: SSDPSearchSessionConfiguration, socketControllerFactory: SocketControllerFactoryProtocol = SocketControllerFactory()) {
        guard let socketController = socketControllerFactory.createUDPSocketController(host: configuration.host, port: configuration.port, socketFactory: SocketFactory()) else {
            return nil
        }
        self.socketController = socketController
        self.configuration = configuration
    }

    deinit {
        stopSearch()
    }

    // MARK: - Search

    // 3
    func startSearch() {
        os_log(.info, "SSDP search session starting")
        writeMessageToSocket(mSearchMessage)
    }

    func stopSearch() {
        os_log(.info, "SSDP search session stopping")
        close()
    }

    // MARK: - Close

    // 4
    private func close() {
        if socketController.state.isActive {
            socketController.close()
        }
    }

    // MARK: - Write

    private func writeMessageToSocket(_ message: String) {
        os_log(.info, "Writing to socket: \r%{public}@", message)
        socketController.write(message: message)
    }
}

SSDPSearchSession above:

1. Configures the M-Search message using the SSDPSearchSessionConfiguration instance.
2. Creates an UDPSocketController instance.
3. Writes the M-Search message to the socket controller.
4. Closes the socket controller if it's active.

With the M-Search message, you may have noticed the \r\n character sequence at the end of each line - don't be tempted to remove this from the M-Search message, as the \r\n sequence is part of the protocol spec.

SSDPSearchSession is designed to be a single-use instance - once stopSearch() is called and the socket closed, a new instance of SSDPSearchSession is needed to perform another search.

Don't forget to add import os to get the os_log statements to compile.

Calling startSearch() should cause an M-Search message to be written to the network; however, as grizzly, well-travelled developers, we know that trusting our code to do something without testing it is a recipe for disappointment 😞. To test that this message is being written to the network, we can snoop on our network traffic using tcpdump.

To test on the simulator, open your terminal and run:

sudo tcpdump -vv -A -s 0 'port 1900 and host 239.255.255.250 and udp'

-vv: verbose output.
-A: print each packet in ASCII.
-s 0: sets the amount of captured data for each frame. Using 0 sets the amount to the default: 65535 bytes - we explicitly set this here for backwards compatibility with recent older versions of tcpdump.
'port 1900 and host 239.255.255.250 and udp' is using Berkeley Packet Filter (BPF) syntax to filter what traffic we see.

The above command will capture all the UDP traffic using host 239.255.255.250 on port 1900 (i.e. SSDP traffic) on your local machine.

If you are using a VPN, you may need to disable it to see anything in the terminal.

Testing on a device is slightly more difficult. We need to tell tcpdump which device to snoop on by creating a remote virtual interface using rvictl:

Connect your device and grab its UDID, then open the terminal and run:

rvictl -s {UDID} && sudo tcpdump -vv -A -s 0 -i rvi0 'port 1900 and host 239.255.255.250 and udp'

Replacing {UDID} with the UDID of the device.

You should see similar traffic to what you would if testing on the simulator.

Run rvictl -x {UDID} to stop the remote virtual interface and Ctrl-C to kill tcpdump.

Now that we have confirmation that M-Search messages are being sent, let's build the functionality to parse any responses we may receive. First, UDPSocketController needs to read anything sent to the socket:

class UDPSocketController: UDPSocketControllerProtocol {
    //Omitted other properties

    // 1
    private let socketListeningQueue = DispatchQueue(label: "com.williamboles.udpsocket.listen.queue",  attributes: .concurrent)

    //Omitted other methods

    // MARK: - Write

    func write(message: String) {
        //Omitted code

        // 2
        let shouldStartListening = state.isReady
        state = .active

        if shouldStartListening {
           startListening(on: socketListeningQueue)
        }

        //Omitted code
    }

    // MARK: - Listen

    // 3
    private func startListening(on queue: DispatchQueue) {
        queue.async {
            do {
                repeat {
                    var data = Data()
                    try self.socket.readDatagram(into: &data) //blocking call
                    self.reportResponseReceived(data)
                } while self.state.isActive
            } catch {
                if self.state.isActive { // ignore any errors for non-active sockets
                    self.closeAndReportError(error)
                }
            }
        }
    }

    private func reportResponseReceived(_ data: Data) {
        os_log(.info, "Response received: \r%{public}@", response)
        //TODO: Implement reporting response received
    }

    //Omitted other methods
}

With the above changes, UDPSocketController is now able to read from its socket.

1. Reading from a BlueSocket socket configured to read datagram messages using UDP is a blocking call - any thread that readDatagram(into:) is called on will be blocked at that line until there is data to be read. To avoid the app from freezing, reading from the socket must be pushed off the caller queue and onto a background queue: socketWriterQueue.
2. We only need to configure the socket to listen once - on the first write.
3. Once a response is received, that response is converted into a string and (for the moment) logged. Finally, if the controller is still listening for responses, the socket is polled again. As a read operation can throw an exception, we wrap that operation in a do...catch block. If an exception is caught, the socket is closed, and any error is reported (we will implement this shortly). An interesting point to note is that closing a socket that is being polled will throw an exception. So when an exception is thrown during polling, we only care about that exception if the session is listening.

If you have devices on your network that support SSDP, you should start to see responses in the Console when running the above code. However, if you don't, it's possible to fake a response using netcat. You will need to extract the host and port from the M-Search request via tcpdump and run the following command:

echo "HTTP/1.1 200 OK\r\nCache-Control: max-age=3600\r\nST: urn:dial-multiscreen-org:service:dial:1\r\nUSN: uuid:0175c106-5400-10f8-802d-b0a7374360b7::urn:dial-multiscreen-org:service:dial:1\r\nExt: \r\nServer: Roku UPnP/1.0 MiniUPnPd/1.4\r\nLOCATION: http://192.168.1.104:8060/\r\n\r\n" | nc -u {host} {port}

Replacing {host} {port} with the extracted values.

The above command will send a response pretending to be a Roku set-top box.

Now that it is possible to read and write from a socket, lets pass any responses (and any errors) out of our socket controller:

protocol UDPSocketControllerDelegate: AnyObject {
    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data)
    func controller(_ controller: UDPSocketControllerProtocol, didEncounterError error: Error)
}
protocol UDPSocketControllerProtocol: AnyObject {
    //Omitted other properties
    var delegate: UDPSocketControllerDelegate? { get set }

    //Omitted methods
}

class UDPSocketController: UDPSocketControllerProtocol {
    //Omitted other properties

    weak var delegate: UDPSocketControllerDelegate?

    private let callbackQueue: OperationQueue

    init?(host: String, port: UInt, socketFactory: SocketFactoryProtocol, callbackQueue: OperationQueue) {
        //Omitted rest of method

        self.callbackQueue = callbackQueue
    }

    //Omitted other methods

    private func reportResponseReceived(_ data: Data) {
        callbackQueue.addOperation {
           self.delegate?.controller(self, didReceiveResponse: data)
        }
    }

    private func closeAndReportError(_ error: Error) {
        close()
        callbackQueue.addOperation {
            self.delegate?.controller(self, didEncounterError: error)
        }
    }
}

With the above changes, SSDPSearchSession can now add itself as the delegate of UDPSocketControllerDelegate. To make the communication via that delegate more predictable when creating that socket, the thread that the communication will happen on is passed in.

Let's update the SocketControllerFactory to support the callbackQueue:

protocol SocketControllerFactoryProtocol {
    func createUDPSocketController(host: String, port: UInt, socketFactory: SocketFactoryProtocol, callbackQueue: OperationQueue) -> UDPSocketControllerProtocol?
}

class SocketControllerFactory: SocketControllerFactoryProtocol {

    // MARK: - UDP

    func createUDPSocketController(host: String, port: UInt, socketFactory: SocketFactoryProtocol, callbackQueue: OperationQueue) -> UDPSocketControllerProtocol? {
        UDPSocketController(host: host, port: port, socketFactory: socketFactory, callbackQueue: callbackQueue)
    }
}

Now, let's update SSDPSearchSession to be the delegate of UDPSocketControllerDelegate:

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted properties

    init?(configuration: SSDPSearchSessionConfiguration, socketControllerFactory: SocketControllerFactoryProtocol = SocketControllerFactory()) {
        guard let socketController = socketControllerFactory.createUDPSocketController(host: configuration.host, port: configuration.port, socketFactory: SocketFactory(), callbackQueue: .main) else {
            return nil
        }

        //Omitted other assignments

        self.socketController.delegate = self
    }

    //Omitted other methods

    // MARK: - UDPSocketControllerDelegate

    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data) {
        os_log(.info, "Received response: \r%{public}@", response)
        //TODO: Implement
    }

    func controller(_ controller: UDPSocketControllerProtocol, didEncounterError error: Error) {
        os_log(.info, "Encountered socket error: \r%{public}@", error.localizedDescription)
        close()
        //TODO: Implement
    }
}

SSDPSearchSession can now receive responses from its UDPSocketController, let's turn those responses into something useful:

struct SSDPService {
    let cacheControl: Date
    let location: URL
    let server: String
    let searchTarget: String
    let uniqueServiceName: String
    let otherHeaders: [String: String]
}

An M-Search response has mandatory and optional/custom header fields. In SSDPService, the mandatory header fields are mapped to named properties, and the optional/custom header fields are mapped to the otherHeaders property. Each optional/custom header field is represented as a dictionary, where the field name serves as the dictionary key and the field's value is the dictionary value.

To get an SSDPService instance, it needs to be parsed:

private enum SSDPServiceResponseKey: String {
    case cacheControl = "CACHE-CONTROL"
    case location = "LOCATION"
    case server = "SERVER"
    case searchTarget = "ST"
    case uniqueServiceName = "USN"
}

protocol SSDPServiceParserProtocol {
    func parse(_ data: Data) -> SSDPService?
}

class SSDPServiceParser: SSDPServiceParserProtocol {
    private let dateFactory: DateFactoryProtocol

    // Init

    init(dateFactory: DateFactoryProtocol =  DateFactory()) {
        self.dateFactory = dateFactory
    }

    // MARK: - Parse

    func parse(_ data: Data) -> SSDPService? {
        guard let responseString = String(data: data, encoding: .utf8) else {
            return nil
        }

        os_log(.info, "Received SSDP response: \r%{public}@", responseString)

        // 1
        var responseDict = parseResponseIntoDictionary(responseString)

        // 2
        guard let cacheControl = parseCacheControl(responseDict[SSDPServiceResponseKey.cacheControl.rawValue]),
            let location = parseLocation(responseDict[SSDPServiceResponseKey.location.rawValue]),
            let server = responseDict[SSDPServiceResponseKey.server.rawValue],
            let searchTarget = responseDict[SSDPServiceResponseKey.searchTarget.rawValue],
            let uniqueServiceName = responseDict[SSDPServiceResponseKey.uniqueServiceName.rawValue] else {
                return nil
        }

        // 3
        responseDict.removeValue(forKey: SSDPServiceResponseKey.cacheControl.rawValue)
        responseDict.removeValue(forKey: SSDPServiceResponseKey.location.rawValue)
        responseDict.removeValue(forKey: SSDPServiceResponseKey.server.rawValue)
        responseDict.removeValue(forKey: SSDPServiceResponseKey.searchTarget.rawValue)
        responseDict.removeValue(forKey: SSDPServiceResponseKey.uniqueServiceName.rawValue)

        // 4
        return SSDPService(cacheControl: cacheControl, location: location, server: server, searchTarget: searchTarget, uniqueServiceName: uniqueServiceName, otherHeaders: responseDict)
    }

    private func parseResponseIntoDictionary(_ response: String) -> [String: String] {
        var elements = [String: String]()
        for element in response.split(separator: "\r\n") {
            let keyValuePair = element.split(separator: ":", maxSplits: 1)
            guard keyValuePair.count == 2 else {
                continue
            }

            let key = String(keyValuePair[0]).uppercased().trimmingCharacters(in: CharacterSet.whitespacesAndNewlines)
            let value = String(keyValuePair[1]).trimmingCharacters(in: CharacterSet.whitespacesAndNewlines)

            elements[key] = value
        }

        return elements
    }

    private func parseCacheControl(_ value: String?) -> Date? {
        guard let cacheControlRange = value?.range(of: "[0-9]+$", options: .regularExpression),
            let cacheControlString = value?[cacheControlRange],
            let cacheControlTimeInterval = TimeInterval(cacheControlString) else {
                return nil
        }

        let currentDate = dateFactory.currentDate()
        return currentDate.addingTimeInterval(cacheControlTimeInterval)
    }

    private func parseLocation(_ value: String?) -> URL? {
        guard let urlString = value,
            let url = URL(string: urlString) else {
                return nil
        }

        return url
    }
}

SSDPServiceParser above takes the string response and attempts to parse it into an SSDPService instance by:

1. Splitting the string into a dictionary using \r\n to determine fields and : to determine key and value pairs.
2. The response dictionary is checked to ensure that all mandatory fields are present. If any of the mandatory fields are missing, the SSDP response is considered invalid, and nil is returned.
3. As an SSDP response can contain non-mandatory fields, the mandatory fields are stripped from the response dictionary, leaving only the non-mandatory fields present.
4. An SSDPService is created using the mandatory and non-mandatory fields.

I could have combined SSDPService and SSDPServiceParser into a single type with the initialiser accepting the string response, but I think having an independent parser makes the code easier to read.

The DateFactory is used to make unit-testing this parser possible:

protocol DateFactoryProtocol {
    func currentDate() -> Date
}

class DateFactory: DateFactoryProtocol {

    // MARK: - Current

    func currentDate() -> Date {
        return Date()
    }
}

Let's update SSDPSearchSession to use SSDPServiceParser:

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted other properties

    private let parser: SSDPServiceParserProtocol

    // MARK: - Init

    init?(configuration: SSDPSearchSessionConfiguration, socketControllerFactory: SocketControllerFactoryProtocol = SocketControllerFactory(), parser: SSDPServiceParserProtocol = SSDPServiceParser()) {
        //Omitted other assignments
        self.parser = parser
    }

    //Omitted methods

    // MARK: - UDPSocketControllerDelegate

    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data) {
        guard !response.isEmpty,
            let service = parser.parse(response) else {
                return
        }

        os_log(.info, "Received service \r%{public}@", service)
    }

    //Omitted methods
}

Once you start parsing responses, you will notice that some root devices respond to any M-Search message they receive, rather than just those discovery requests that match one of their services. To counter these chatty root devices we need to ensure that the parsed SSDPService instance is the searched-for-service:

class SSDPSearchSession {
    //Omitted properties & methods

    // MARK: - UDPSocketControllerDelegate

    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data) {
       guard !response.isEmpty,
           let service = parser.parse(response),
           searchedForService(service) else {
               return
       }

       os_log(.info, "Received service \r%{public}@", service)
    }

    private func searchedForService(_ service: SSDPService) -> Bool {
       return service.searchTarget.contains(configuration.searchTarget) || configuration.searchTarget == "ssdp:all"
    }

    //Omitted methods
}

If the search target is set to the special ssdp:all value, all services that respond are treated as valid.

SSDPSearchSession is doing good work but isn't able to share the fruits of its labour with anyone. Let's add in a delegate to tell interested parties how things are going with the search:

protocol SSDPSearchSessionDelegate: AnyObject {
    func searchSession(_ searchSession: SSDPSearchSession, didFindService service: SSDPService)
    func searchSession(_ searchSession: SSDPSearchSession, didEncounterError error: SSDPSearchSessionError)
    func searchSessionDidStopSearch(_ searchSession: SSDPSearchSession, foundServices: [SSDPService])
}

enum SSDPSearchSessionError: Error {
    case searchAborted(Error)
}

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted properties

    // 1
    private var servicesFoundDuringSearch = [SSDPService]()

    weak var delegate: SSDPSearchSessionDelegate?

    //Omitted methods

    func stopSearch() {
        os_log(.info, "SSDP search session stopping")
        close()

        // 2
        delegate?.searchSessionDidStopSearch(self, foundServices:servicesFoundDuringSearch)
    }

    //Omitted methods

    // MARK: - UDPSocketControllerDelegate

    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data) {
        guard !response.isEmpty,
            let service = parser.parse(response),
            searchedForService(service) else {
                return
        }

        os_log(.info, "Received a valid service response")

        servicesFoundDuringSearch.append(service)

        // 3
        delegate?.searchSession(self, didFindService: service)
    }

    func controller(_ controller: UDPSocketControllerProtocol, didEncounterError error: Error) {
        os_log(.info, "Encountered socket error: \r%{public}@", error.localizedDescription)

        // 4
        let wrappedError = SSDPSearchSessionError.searchAborted(error)
        delegate?.searchSession(self, didEncounterError: wrappedError)
        close()
    }

    //Omitted methods
}

With the above changes, we now:

1. Store all valid services that have been received during that search session.
2. Inform the delegate when the search has been stopped, returning all valid services found.
3. Inform the delegate when a valid service has been found.
4. Wrap any received error in an SSDPSearchSessionError error and inform the delegate of that error.

It's interesting to note that searchSession(_:, didFindService:) is called as soon as a valid SSDPService instance is parsed, rather than waiting for all services to be parsed. This will allow the app to respond immediately to any found services.

An alternative to delegation would have been to pass a closure into startSearch(). In fact, using a closure was my preferred option to begin with. However, after experimenting, I found that having one closure handle three possible states resulted in code that was very busy, and readability suffered as a consequence.

Every M-Search message contains an MX value that represents the maximum time a service can wait before responding. When this time has elapsed, it can be confidently assumed that all services that can respond have responded. Meaning that MX can be used as a timeout for the search session:

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted other properties

    private let searchTimeout: TimeInterval

    private var timeoutTimer: Timer?

    // MARK: - Init

    init?(configuration: SSDPSearchSessionConfiguration, socketControllerFactory: SocketControllerFactoryProtocol = SocketControllerFactory(), parser: SSDPServiceParserProtocol = SSDPServiceParser()) {
        //Omitted other assignments

        self.searchTimeout = configuration.maximumWaitResponseTime + 0.1
    }

    //Omitted methods

    // MARK: - Search

    func startSearch() {
        //Omitted code

        timeoutTimer = Timer.scheduledTimer(withTimeInterval: searchTimeout, repeats: false, block: { [weak self] (timer) in
            self?.searchTimedOut()
        })
    }

    private func searchTimedOut() {
        os_log(.info, "SSDP search timed out")
        stopSearch()
    }

    //Omitted methods

    // MARK: - Close

    private func close() {
        timeoutTimer?.invalidate()
        timeoutTimer = nil

        //Omitted code
    }

    //Omitted methods
}

With the above changes, the search session ends after maximumWaitResponseTime seconds, causing the socket to be closed. The more eagle-eyed reader may have spotted that timeoutTimer has a trigger time that is 0.1 seconds longer than maximumWaitResponseTime - this is to allow any responses from root devices that waited the full maximumWaitResponseTime seconds before responding to reach the app and be processed before the search session is ended.

Some people are harder to talk to than others

If you have been combining the above code snippets into a working project, you will now be able to search for SSDP services and parse any response received. However, every so often, you may notice that an SSDP service that you know exists on the network does not respond.

🤔

As described above, SSDP uses the unreliable UDP transportation protocol (because UDP supports multicasting). UDP is unreliable because there is no acknowledgement if a message reaches its destination. This means there is no way of knowing if a service hasn't responded because the message was dropped along the way, or that the service is no longer available. Unreliability isn't a great characteristic for a discovery service to have. While not foolproof, it is possible to increase the reliability of an SSDP-based discovery service by sending multiple M-Search messages over the lifecycle of an SSDPSearchService instance. Sending multiple M-Search messages will increase the chances that at least one message reaches each root device on the network. For this to be possible, our SSDPSearchService instance must exist longer than the MX value before timing out:

struct SSDPSearchSessionConfiguration {
    //Omitted properties

    let maximumSearchRequestsBeforeClosing: UInt

    // MARK: - Init

    init(searchTarget: String, host: String, port: UInt, maximumWaitResponseTime: TimeInterval, maximumSearchRequestsBeforeClosing: UInt) {
        //Omitted other assignments

        self.maximumSearchRequestsBeforeClosing = maximumSearchRequestsBeforeClosing
    }
}

extension SSDPSearchSessionConfiguration {

    static func createMulticastConfiguration(forSearchTarget searchTarget: String, maximumWaitResponseTime: TimeInterval = 3, maximumSearchRequestsBeforeClosing: UInt = 3) -> SSDPSearchSessionConfiguration {
        let configuration = SSDPSearchSessionConfiguration(searchTarget: searchTarget, host: "239.255.255.250", port: 1900, maximumWaitResponseTime: maximumWaitResponseTime, maximumSearchRequestsBeforeClosing: maximumSearchRequestsBeforeClosing)

        return configuration
    }
}

maximumSearchRequestsBeforeClosing will control how many M-Search messages are sent before the search session is closed. maximumSearchRequestsBeforeClosing needs to be at least 1, or no M-Search will be sent.

An alternative to using a count property would have been to use a total-duration property. However, the total-duration approach would have required the config-maintainer to ensure that this timeout property was always a multiple of the maximumWaitResponseTime. Having a value that isn't a multiple would result in the situation where the session was stopped before the last M-Search iteration's maximumWaitResponseTime had expired, potentially resulting in ignored responses because the root devices waited until the maximumWaitResponseTime value before responding. It's easy to imagine this mistake happening. By expressing the timeout as the value to multiply maximumWaitResponseTime by, we ensure that this error scenario can never happen.

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted properties

    private var searchRequestTimer: Timer?

    //Omitted methods

    // MARK: - Init

    init?(configuration: SSDPSearchSessionConfiguration, socketControllerFactory: SocketControllerFactoryProtocol = SocketControllerFactory(), parser: SSDPServiceParserProtocol = SSDPServiceParser()) {
        //Omitted other assignments

        // 1
        self.searchTimeout = (TimeInterval(configuration.maximumSearchRequestsBeforeClosing) * configuration.maximumWaitResponseTime) + 0.1
    }

    // MARK: - Search

    func startSearch() {
        // 2
        guard configuration.maximumSearchRequestsBeforeClosing > 0 else {
            delegate?.searchSessionDidStopSearch(self, foundServices: servicesFoundDuringSearch)
            return
        }

        os_log(.info, "SSDP search session starting")
        sendMSearchMessages()

        //Omitted code
    }

    //Omitted methods

    // MARK: - Close

    private func close() {
        //Omitted code

        // 3
        searchRequestTimer?.invalidate()
        searchRequestTimer = nil

        //Omitted code
    }

    // MARK: Write

    private func sendMSearchMessages() {
        let message = mSearchMessage

        // 4
        if configuration.maximumSearchRequestsBeforeClosing > 1 {
            let window = searchTimeout - configuration.maximumWaitResponseTime
            let interval = window / TimeInterval((configuration.maximumSearchRequestsBeforeClosing - 1))

            searchRequestTimer = Timer.scheduledTimer(withTimeInterval: interval, repeats: true, block: { [weak self] (timer) in
                self?.writeMessageToSocket(message)
            })
        }
        writeMessageToSocket(message)
    }

    //Omitted methods
}

With the above changes, a new M-Search message is sent every maximumWaitResponseTime seconds until the desired number of messages have been sent.

1. Increased the total search time for multiple M-Search message writes.
2. Check that maximumSearchRequestsBeforeClosing is greater than 0 and return if it isn't.
3. Invalidate the new search request timer.
4. Send multiple M-Search messages inside the maximum send window.

However, while making SSDPSearchSession a more reliable discovery service, sending multiple M-Search messages creates a new problem. In ideal network conditions, the same SSDP service would respond to each sent M-Search message. If these responses were just blindly passed to the delegate, the SSDPSearchService instance would, in effect, be spamming that delegate with the same service multiple times. Thankfully, each service parsed is already stored in the servicesFoundDuringSearch array (to be used when searchSessionDidStopSearch(_:, foundServices:) is called), so to prevent becoming a spammer, a check can be made to determine if an SSDPService instance representing the same service has already been passed to the delegate:

class SSDPSearchSession: SSDPSearchSessionProtocol, UDPSocketControllerDelegate {
    //Omitted properties and methods

    // MARK: - UDPSocketControllerDelegate

    func controller(_ controller: UDPSocketControllerProtocol, didReceiveResponse response: Data) {
        guard !response.isEmpty,
            let service = parser.parse(response),
            searchedForService(service),
            // 1
            !servicesFoundDuringSearch.contains(service) else {
                return
        }

        //Omitted code
    }

    //Omitted methods
}

With the above change:

1. A check is made to see if servicesFoundDuringSearch already contains the service that has been parsed.

To get the above code to work, SSDPService needs to conform to Equatable:

struct SSDPService: Equatable {
    //Omitted properties

    // MARK: - Equatable

    static func == (lhs: SSDPService, rhs: SSDPService) -> Bool {
        return lhs.location == rhs.location &&
            lhs.server == rhs.server &&
            lhs.searchTarget == rhs.searchTarget &&
            lhs.uniqueServiceName == rhs.uniqueServiceName
    }
}

The above custom equality check excludes cacheControl because it changes with each response, as it's based on the date when the service was parsed.

🎉🎉🎉

And that's everything you need for discovering what SSDP services are available on a network - congratulations.

Happy to get to know everyone

Social situations can be tricky. It's easy to think that you don't have anything of value to add and to allow that thought to leave you alone in the corner. However, with a little bit of effort (and bravery), you can reach out and get to know new people.

This is just as true for your app.

SSDP is a lightweight, widely supported protocol that makes it straightforward to discover services on a network. It has a few gotchas, but provided that we treat it with care and don't 100% trust any root devices to behave as expected, SSDP can be a useful tool to have in the toolbox.

To see the above code snippets together in a working example, head over to the repo and clone the project.

There are very few certainties in app development, but one is that once your app is released, it will change in unexpected ways. And no matter how flexible your architecture is, inevitably one of those changes will be a breaking change. Perhaps the most important breaking changes involve the user's data. If your app loses or corrupts user data, you can expect at least some reputational damage, and if the loss is severe enough, you can end up doing your competitors' marketing for them by turning your users into their users. If you have any doubt about the impact of data loss, imagine how you would feel if a game you had been playing was updated and deleted your recently hard-earned thingamabob - all that time and effort lost through no fault of yours. And that's just a game, now imagine how your users would feel when your far-more-important app starts losing their data.

In our iOS apps, we often store these thingamabobs in Core Data. The structure of which is defined by a model/schema - a set of entities with attributes and relationships. To allow changes to be made to this model to meet your app's changing needs, Core Data has a built-in mechanism to migrate data from one model structure to another model structure.

In this post, we are going to build a simple system to manipulate the built-in Core Data migration mechanism to make migrations simpler and so reduce the risk of losing your users' data.

This post will gradually build up to a working example however if you're on a tight deadline and/or there is murderous look creeping into your manager's eyes 😡, then head on over to the completed example and take a look at CoreDataManager, CoreDataMigrator, CoreDataMigrationStep and CoreDataMigrationVersion to see how things end up.

The Migration Process

As mentioned above, Core Data allows the model to evolve through model versions. Typically, a model version's changeable lifecycle (when it can be changed) is from when that version is created until it's released as an app update. Once released, a version is effectively "frozen" - any further changes made to that version would result in an app crash upon launch. To change an already released model, you need to create a new version of that model and migrate users from the old version to the new version. Thankfully, Core Data has a built-in migration system.

Migrations can be handled using one of two techniques:

1. Lightweight Migration - when Core Data can automatically infer how the migration should happen and creates the mapping model on the fly.
2. Standard Migration - when Core Data cannot infer how the migration should happen and so we must write a custom migration by providing a mapping model (xcmappingmodel) and/or a migration policy (NSEntityMigrationPolicy).

By default, Core Data will attempt to perform a migration automatically when it detects a mismatch between the model used in the persistent store and the bundle's current model. When this happens, Core Data will first attempt to perform a Standard migration by searching in the app's bundle for a mapping model that maps from the persistent store model to the current bundle model. If a custom mapping model isn't found, Core Data will then attempt to perform a Lightweight migration. If neither form of migration is possible, an exception is thrown.

If you are using NSPersistentContainer, Lightweight migrations are enabled by default; however, if you are still directly setting up the NSPersistentStoreCoordinator, then you need to enable Lightweight migrations by passing in an options dictionary with both NSMigratePersistentStoresAutomaticallyOption and NSInferMappingModelAutomaticallyOption set to true when loading the persistent store.

These automatic migrations are performed as one-step migrations; directly from the source to the destination model. So if we support 4 model versions, mapping models would exist for 1 to 4, 2 to 4 and 3 to 4. While this is the most efficient migration approach from a device performance point-of-view, it can actually be quite wasteful from a development point-of-view. For example if we added a new model version (5) we would need to create 4 new mapping models from 1 to 5, 2 to 5, 3 to 5 and 4 to 5 which as you can see doesn't reuse any of the mapping models for migrating to version 4. With a one-step migration approach, each newly added model version requires n-1 mapping models (where n is the number of supported model versions) to be created.

It's possible to reduce the amount of work required to perform a Core Data migration by disabling automatic migrations, and so break the requirement to perform migrations in one-step. With a manual migration approach, we can perform the full migration by chaining multiple smaller migrations together. As the full migration is split into smaller migrations, when adding a new model version, we only need to handle migrating to the new model version from its direct predecessor rather than all it's predecessors e.g. 4 to 5 because we can reuse the existing 1 to 2, 2 to 3 and 3 to 4 mapping models. Not only do manual migrations reduce the amount of work involved they also help to reduce the complexity of the migration as the conceptional distance between the source and destination version is reduced when compared to one-step migrations i.e. version 4 is much nearer to the structure of version 5 than version 1 is - this should make it easier spot any issues with the migration.

Progressive migrations

To support progressive migrations, we'll need to answer a few questions:

1. Which model version comes after version X?
2. What is a migration step?
3. How can we combine the migration steps into a migration path?
4. How do we trigger a migration?

These questions will be answered with the help of 4 separate types:

1. CoreDataMigrationVersion
2. CoreDataMigrationStep
3. CoreDataMigrator
4. CoreDataManager

These types will come together in the following class structure (along with several helper extensions):

Don't worry if that doesn't all make sense yet; we will look into each type in greater depth below.

Which model version comes after version X?

Each CoreDataMigrationVersion instance will represent a Core Data model version. As each Core Data model version is unique and known at compile time, they can be perfectly represented as enum cases, with the raw value of each case being the Core Data model name:

enum CoreDataMigrationVersion: String, CaseIterable {
    case version1 = "CoreDataMigration_Example"

    // MARK: - Current

    static var current: CoreDataMigrationVersion {
        guard let current = allCases.last else {
            fatalError("no model versions found")
        }

        return current
    }

    // MARK: - Migration

    func nextVersion() -> CoreDataMigrationVersion? {
        switch self {
        case .version1:
            return nil
        }
    }
}

Migrations are often concerned with what the latest model version is - the static current property allows easy access to this version. Before Swift 4.2, we would probably have had to hardcode this property to one case, which would then lead to bugs if we forgot to update that property when adding a new version. However, in Swift 4.2, we got the CaseIterable protocol, which makes it possible to get an array of the cases in an enum in the order they were defined in via the allCases property. This means that to get the latest model version should be as simple as calling last on the allCases array - no need to hardcode anything.

In CoreDataMigrationVersion, the nextVersion() method is where the real work happens, as it determines which (if any) version comes after self.

You may be thinking:

"Why bother with nextVersion() when we can just always choose the next enum case?"

If you are reading this post before performing your first migration I congratulate you on your:

1. Excellent taste in selecting blog posts.
2. Organisational ability.

However, it's more likely that you've found this post having already performed several migrations and been hit by the inherent scaling issue with the default one-step migration approach. If you are in the latter camp, then you will have already implemented one-step migrations, having configured various mapping models and maybe even written a migration policy or two. Instead of throwing all that work away, we can use it and tie it into the new progressive approach. In a hypothetical project that had 6 model versions, which until model version 4 used the one-step migration approach before switching over to the progressive migration approach, then nextVersion would look like:

func nextVersion() -> CoreDataMigrationVersion? {
    switch self {
    case .version1, .version2, .version3:
        return .version4
    case .version4:
        return .version5
    case .version5:
        return .version6
    case .version6:
        return nil
    }
}

In the above code snippet, version1, version2 and version3 migrate directly to version4, and then version4 and version5 migrate to their direct successor. As you can see, both these migration approaches can co-exist very happily with each other.

Even if you don't have any existing migrations, it's possible that at some point in the future, a broken model version is released that corrupts your users ' data upon migration. To minimise the impact of this mistake, nextVersion could be configured to bypass that broken model version so that any currently unaffected users are never impacted:

func nextVersion() -> CoreDataMigrationVersion? {
    switch self {
    case .version1:
        return .version2
    case .version2:
        return .version4 // skipping corrupted .version3
    case .version3:
        return .version4
    case .version4:
        return nil
    }
}

Both these issues are easily bypassed using nextVersion() without adding too much complexity to the overall solution.

What is a migration step?

A migration happens between 2 model versions by having a mapping from the entities, attributes and relationships of the source model and their counterpoints in the destination model. As such, CoreDataMigrationStep needs to contain 3 properties:

1. Source version model.
2. Destination version model.
3. Mapping model.

struct CoreDataMigrationStep {

    let sourceModel: NSManagedObjectModel
    let destinationModel: NSManagedObjectModel
    let mappingModel: NSMappingModel

    // MARK: Init

    init(sourceVersion: CoreDataMigrationVersion, destinationVersion: CoreDataMigrationVersion) {
        let sourceModel = NSManagedObjectModel.managedObjectModel(forResource: sourceVersion.rawValue)
        let destinationModel = NSManagedObjectModel.managedObjectModel(forResource: destinationVersion.rawValue)

        guard let mappingModel = CoreDataMigrationStep.mappingModel(fromSourceModel: sourceModel, toDestinationModel: destinationModel) else {
            fatalError("Expected modal mapping not present")
        }

        self.sourceModel = sourceModel
        self.destinationModel = destinationModel
        self.mappingModel = mappingModel
    }

    // MARK: - Mapping

    private static func mappingModel(fromSourceModel sourceModel: NSManagedObjectModel, toDestinationModel destinationModel: NSManagedObjectModel) -> NSMappingModel? {
        guard let customMapping = customMappingModel(fromSourceModel: sourceModel, toDestinationModel: destinationModel) else {
            return inferredMappingModel(fromSourceModel:sourceModel, toDestinationModel: destinationModel)
        }

        return customMapping
    }

    private static func inferredMappingModel(fromSourceModel sourceModel: NSManagedObjectModel, toDestinationModel destinationModel: NSManagedObjectModel) -> NSMappingModel? {
        return try? NSMappingModel.inferredMappingModel(forSourceModel: sourceModel, destinationModel: destinationModel)
    }

    private static func customMappingModel(fromSourceModel sourceModel: NSManagedObjectModel, toDestinationModel destinationModel: NSManagedObjectModel) -> NSMappingModel? {
        return NSMappingModel(from: [Bundle.main], forSourceModel: sourceModel, destinationModel: destinationModel)
    }
}

It's possible to have multiple mapping models between versions (this can be especially useful when migrating large data sets). In this post, in an attempt to keep things simple, I assume only one mapping model.

CoreDataMigrationStep takes the source model and destination model and attempts to find a way to map between them. As we know, there are two types of migrations: Lightweight and Standard - both of which use a NSMappingModel instance to hold the mapping path between the versions. Because of this shared output type, mappingModel(fromSourceModel:toDestinationModel) handles searching for a mapping model using either Lightweight or Standard migration. First, a search is made for a custom migration mapping existing in the bundle (Standard migration), and then if no custom mapping model is found, Core Data is asked to try and infer a mapping model (Lightweight migration). If a mapping model can't be found using either approach, a fatal error is thrown as this migration path isn't supported.

How can we combine the migration steps into a migration path?

CoreDataMigrator is at the heart of our migration solution and has 3 tasks:

1. Determining if there needs to be a migration.
2. Ensuring the persistent store is ready to be migrated.
3. Performing the migration.

As CoreDataManager (we will see this later) holds a reference to CoreDataMigrator, we can make our lives easier by wrapping CoreDataMigrator in a protocol so that it's easier to mock when writing tests for CoreDataManager:

protocol CoreDataMigratorProtocol {
    func requiresMigration(at storeURL: URL, toVersion version: CoreDataMigrationVersion) -> Bool
    func migrateStore(at storeURL: URL, toVersion version: CoreDataMigrationVersion)
}

Now that we have that protocol, let's look at how CoreDataMigrator implements the first of those protocol methods:

class CoreDataMigrator: CoreDataMigratorProtocol {

    // MARK: - Check

    func requiresMigration(at storeURL: URL, toVersion version: CoreDataMigrationVersion) -> Bool {
        guard let metadata = NSPersistentStoreCoordinator.metadata(at: storeURL) else {
            return false
        }

        return (CoreDataMigrationVersion.compatibleVersionForStoreMetadata(metadata) != version)
    }

    //Omitted other methods
}

In the above method, the persistent store's metadata is loaded and checked to see if it's compatible with the current bundle model's metadata. To support this, we need to extend CoreDataMigrationVersion to include:

private extension CoreDataMigrationVersion {

    // MARK: - Compatible

    static func compatibleVersionForStoreMetadata(_ metadata: [String : Any]) -> CoreDataMigrationVersion? {
        let compatibleVersion = CoreDataMigrationVersion.allCases.first {
            let model = NSManagedObjectModel.managedObjectModel(forResource: $0.rawValue)

            return model.isConfiguration(withName: nil, compatibleWithStoreMetadata: metadata)
        }

        return compatibleVersion
    }
}

The above method attempts to find a compatible model for the metadata by iterating through the model associated with a case of CoreDataMigrationVersion. If a compatible model is found, the associated version is returned; else, nil is returned.

Now that we know if a migration is required or not, let's look at how that migration happens by implementing the next protocol method:

class CoreDataMigrator: CoreDataMigratorProtocol {

    //Omitted other methods

    // MARK: - Migration

    func migrateStore(at storeURL: URL, toVersion version: CoreDataMigrationVersion) {
        forceWALCheckpointingForStore(at: storeURL)

        var currentURL = storeURL
        let migrationSteps = self.migrationStepsForStore(at: storeURL, toVersion: version)

        for migrationStep in migrationSteps {
            let manager = NSMigrationManager(sourceModel: migrationStep.sourceModel, destinationModel: migrationStep.destinationModel)
            let destinationURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(UUID().uuidString)

            do {
                try manager.migrateStore(from: currentURL, sourceType: NSSQLiteStoreType, options: nil, with: migrationStep.mappingModel, toDestinationURL: destinationURL, destinationType: NSSQLiteStoreType, destinationOptions: nil)
            } catch let error {
                fatalError("failed attempting to migrate from \(migrationStep.sourceModel) to \(migrationStep.destinationModel), error: \(error)")
            }

            if currentURL != storeURL {
                //Destroy intermediate step's store
                NSPersistentStoreCoordinator.destroyStore(at: currentURL)
            }

            currentURL = destinationURL
        }

        NSPersistentStoreCoordinator.replaceStore(at: storeURL, withStoreAt: currentURL)

        if (currentURL != storeURL) {
            NSPersistentStoreCoordinator.destroyStore(at: currentURL)
        }
    }

    private func migrationStepsForStore(at storeURL: URL, toVersion destinationVersion: CoreDataMigrationVersion) -> [CoreDataMigrationStep] {
        guard let metadata = NSPersistentStoreCoordinator.metadata(at: storeURL), let sourceVersion = CoreDataMigrationVersion.compatibleVersionForStoreMetadata(metadata) else {
            fatalError("unknown store version at URL \(storeURL)")
        }

        return migrationSteps(fromSourceVersion: sourceVersion, toDestinationVersion: destinationVersion)
    }

    private func migrationSteps(fromSourceVersion sourceVersion: CoreDataMigrationVersion, toDestinationVersion destinationVersion: CoreDataMigrationVersion) -> [CoreDataMigrationStep] {
        var sourceVersion = sourceVersion
        var migrationSteps = [CoreDataMigrationStep]()

        while sourceVersion != destinationVersion, let nextVersion = sourceVersion.nextVersion() {
            let migrationStep = CoreDataMigrationStep(sourceVersion: sourceVersion, destinationVersion: nextVersion)
            migrationSteps.append(migrationStep)

            sourceVersion = nextVersion
        }

        return migrationSteps
    }

    // MARK: - WAL

    func forceWALCheckpointingForStore(at storeURL: URL) {
        guard let metadata = NSPersistentStoreCoordinator.metadata(at: storeURL), let currentModel = NSManagedObjectModel.compatibleModelForStoreMetadata(metadata) else {
            return
        }

        do {
            let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: currentModel)

            let options = [NSSQLitePragmasOption: ["journal_mode": "DELETE"]]
            let store = persistentStoreCoordinator.addPersistentStore(at: storeURL, options: options)
            try persistentStoreCoordinator.remove(store)
        } catch let error {
            fatalError("failed to force WAL checkpointing, error: \(error)")
        }
    }
}

There is quite a bit of code there, let's break it down into smaller pieces and explore each separately, building the migration process up from bottom-to-top.

Before attempting a migration, we need to undertake some housekeeping on our persistent store.

Since iOS 7, Core Data has used the Write-Ahead Logging (WAL) option on SQLite stores to provide the ability to recover from crashes by allowing changes to be rolled back until the database is stable. If you have ever had to perform a rollback before, the WAL approach may work a little differently from what you are expecting. Rather than directly writing changes to the sqlite file and having a pre-write copy of the changes to rollback to, in WAL mode, the changes are first written to the sqlite-wal file, and at some future date, those changes are transferred to the sqlite file. The sqlite-wal file is in effect an up-to-date copy of some of the data stored in the main sqlite file.

The sqlite-wal and sqlite files store their data using the same structure to allow data to be transferred easily between them. However, this shared structure causes issues during migration as Core Data only migrates the data stored in the sqlite file to the new structure, leaving the data in the sqlite-wal file in the old structure. The resulting mismatch in structure will lead to a crash when Core Data attempts to update/use data stored in the sqlite-wal file 😞 . To avoid this crash, we need to force any data in the sqlite-wal file into the sqlite file before we perform a migration - a process known as checkpointing:

func forceWALCheckpointingForStore(at storeURL: URL) {
    guard let metadata = NSPersistentStoreCoordinator.metadata(at: storeURL), let currentModel = NSManagedObjectModel.compatibleModelForStoreMetadata(metadata) else {
        return
    }

    do {
        let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: currentModel)

        let options = [NSSQLitePragmasOption: ["journal_mode": "DELETE"]]
        let store = persistentStoreCoordinator.addPersistentStore(at: storeURL, options: options)
        try persistentStoreCoordinator.remove(store)
    } catch let error {
        fatalError("failed to force WAL checkpointing, error: \(error)")
    }
}

The above method forces checkpointing to occur. A side effect of checkpointing is that the empty sqlite-wal file is deleted for us, so removing the store from the persistentStoreCoordinator is all the cleanup that we need to perform.

An easy mistake to make when checkpointing is using the bundle's model rather than the store's model - remember we want to perform checkpointing on the live (store) model before attempting to migrate to the latest (bundle) model.

Before a migration can be performed, Core Data must first construct the individual migration steps into a migration path:

private func migrationStepsForStore(at storeURL: URL, toVersion destinationVersion: CoreDataMigrationVersion) -> [CoreDataMigrationStep] {
    guard let metadata = NSPersistentStoreCoordinator.metadata(at: storeURL), let sourceVersion = CoreDataMigrationVersion.compatibleVersionForStoreMetadata(metadata) else {
        fatalError("unknown store version at URL \(storeURL)")
    }

    return migrationSteps(fromSourceVersion: sourceVersion, toDestinationVersion: destinationVersion)
}

private func migrationSteps(fromSourceVersion sourceVersion: CoreDataMigrationVersion, toDestinationVersion destinationVersion: CoreDataMigrationVersion) -> [CoreDataMigrationStep] {
    var sourceVersion = sourceVersion
    var migrationSteps = [CoreDataMigrationStep]()

    while sourceVersion != destinationVersion, let nextVersion = sourceVersion.nextVersion() {
        let migrationStep = CoreDataMigrationStep(sourceVersion: sourceVersion, destinationVersion: nextVersion)
        migrationSteps.append(migrationStep)

        sourceVersion = nextVersion
    }

    return migrationSteps
}

In the above methods, the migration path is built by looping through the appropriate model versions until the destination model version is reached. This migration path will take the user's data from the persistent store's model version to the bundle model version in a progressive migration:

func migrateStore(at storeURL: URL, toVersion version: CoreDataMigrationVersion) {
    forceWALCheckpointingForStore(at: storeURL)

    var currentURL = storeURL
    let migrationSteps = self.migrationStepsForStore(at: storeURL, toVersion: version)

    for migrationStep in migrationSteps {
        let manager = NSMigrationManager(sourceModel: migrationStep.sourceModel, destinationModel: migrationStep.destinationModel)
        let destinationURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(UUID().uuidString)

        do {
            try manager.migrateStore(from: currentURL, sourceType: NSSQLiteStoreType, options: nil, with: migrationStep.mappingModel, toDestinationURL: destinationURL, destinationType: NSSQLiteStoreType, destinationOptions: nil)
        } catch let error {
            fatalError("failed attempting to migrate from \(migrationStep.sourceModel) to \(migrationStep.destinationModel), error: \(error)")
        }

        if currentURL != storeURL {
            //Destroy intermediate step's store
            NSPersistentStoreCoordinator.destroyStore(at: currentURL)
        }

        currentURL = destinationURL
    }

    NSPersistentStoreCoordinator.replaceStore(at: storeURL, withStoreAt: currentURL)

    if (currentURL != storeURL) {
        NSPersistentStoreCoordinator.destroyStore(at: currentURL)
    }
}

In the above method, we iterate through each migration step and attempt to perform a migration using NSMigrationManager. The result of each completed migration step is saved to a temporary persistent store; only once the migration is complete is the original persistent store overwritten. If there is a failure during any individual migration step, a fatal error is thrown - this is especially useful during the development of a custom migration path.

In the above code snippets, we've seen a number of methods used that are not part of the standard API, so I've included the extensions that contain these methods below. As with most extensions, the methods are used to reduce boilerplate code:

extension NSPersistentStoreCoordinator {

    // MARK: - Destroy

    static func destroyStore(at storeURL: URL) {
        do {
            let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: NSManagedObjectModel())
            try persistentStoreCoordinator.destroyPersistentStore(at: storeURL, ofType: NSSQLiteStoreType, options: nil)
        } catch let error {
            fatalError("failed to destroy persistent store at \(storeURL), error: \(error)")
        }
    }

    // MARK: - Replace

    static func replaceStore(at targetURL: URL, withStoreAt sourceURL: URL) {
        do {
            let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: NSManagedObjectModel())
            try persistentStoreCoordinator.replacePersistentStore(at: targetURL, destinationOptions: nil, withPersistentStoreFrom: sourceURL, sourceOptions: nil, ofType: NSSQLiteStoreType)
        } catch let error {
            fatalError("failed to replace persistent store at \(targetURL) with \(sourceURL), error: \(error)")
        }
    }

    // MARK: - Meta

    static func metadata(at storeURL: URL) -> [String : Any]?  {
        return try? NSPersistentStoreCoordinator.metadataForPersistentStore(ofType: NSSQLiteStoreType, at: storeURL, options: nil)
    }

    // MARK: - Add

    func addPersistentStore(at storeURL: URL, options: [AnyHashable : Any]) -> NSPersistentStore {
        do {
            return try addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: storeURL, options: options)
        } catch let error {
            fatalError("failed to add persistent store to coordinator, error: \(error)")
        }
    }
}

extension NSManagedObjectModel {

    // MARK: - Resource

    static func managedObjectModel(forResource resource: String) -> NSManagedObjectModel {
        let mainBundle = Bundle.main
        let subdirectory = "CoreDataMigration_Example.momd"
        let omoURL = mainBundle.url(forResource: resource, withExtension: "omo", subdirectory: subdirectory) // optimised model file
        let momURL = mainBundle.url(forResource: resource, withExtension: "mom", subdirectory: subdirectory)

        guard let url = omoURL ?? momURL else {
            fatalError("unable to find model in bundle")
        }

        guard let model = NSManagedObjectModel(contentsOf: url) else {
            fatalError("unable to load model in bundle")
        }

        return model
    }
}

I won't go into detail about what these extension methods do, as I believe their names do a good enough job.

How do we trigger a migration?

CoreDataManager handles both setting up the Core Data stack and triggering a migration (if needed):

class CoreDataManager {

    let migrator: CoreDataMigratorProtocol
    private let storeType: String

    lazy var persistentContainer: NSPersistentContainer = {
        let persistentContainer = NSPersistentContainer(name: "CoreDataMigration_Example")
        let description = persistentContainer.persistentStoreDescriptions.first
        description?.shouldInferMappingModelAutomatically = false //inferred mapping will be handled else where
        description?.shouldMigrateStoreAutomatically = false
        description?.type = storeType

        return persistentContainer
    }()

    lazy var backgroundContext: NSManagedObjectContext = {
        let context = self.persistentContainer.newBackgroundContext()
        context.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy

        return context
    }()

    lazy var mainContext: NSManagedObjectContext = {
        let context = self.persistentContainer.viewContext
        context.automaticallyMergesChangesFromParent = true

        return context
    }()

    // MARK: - Singleton

    static let shared = CoreDataManager()

    // MARK: - Init

    init(storeType: String = NSSQLiteStoreType, migrator: CoreDataMigratorProtocol = CoreDataMigrator()) {
        self.storeType = storeType
        self.migrator = migrator
    }

    // MARK: - SetUp

    func setup(completion: @escaping () -> Void) {
        loadPersistentStore {
            completion()
        }
    }

    // MARK: - Loading

    private func loadPersistentStore(completion: @escaping () -> Void) {
        migrateStoreIfNeeded {
            self.persistentContainer.loadPersistentStores { description, error in
                guard error == nil else {
                    fatalError("was unable to load store \(error!)")
                }

                completion()
            }
        }
    }

    private func migrateStoreIfNeeded(completion: @escaping () -> Void) {
        guard let storeURL = persistentContainer.persistentStoreDescriptions.first?.url else {
            fatalError("persistentContainer was not set up properly")
        }

        if migrator.requiresMigration(at: storeURL, toVersion: CoreDataMigrationVersion.current) {
            DispatchQueue.global(qos: .userInitiated).async {
                self.migrator.migrateStore(at: storeURL, toVersion: CoreDataMigrationVersion.current)

                DispatchQueue.main.async {
                    completion()
                }
            }
        } else {
            completion()
        }
    }
}

If you have ever seen a Core Data stack setup before, you will instantly notice how little code the CoreDataManager contains. Over the years, Core Data has evolved and become more developer-friendly. Above, we are taking advantage of a relatively new piece of the Core Data family - NSPersistentContainer which was introduced in iOS 10:

lazy var persistentContainer: NSPersistentContainer = {
    let persistentContainer = NSPersistentContainer(name: "CoreDataMigration_Example")
    let description = persistentContainer.persistentStoreDescriptions.first
    description?.shouldInferMappingModelAutomatically = false //inferred mapping will be handled else where
    description?.shouldMigrateStoreAutomatically = false
    description?.type = storeType

    return persistentContainer
}()

NSPersistentContainer simplifies the creation of the managed object model, persistent store coordinator and the managed object contexts by making smart assumptions on how we want our persistent store configured. It's still possible to access the NSManagedModel, NSPersistentStoreCoordinator and NSManagedObjectContext instances via this container, but we no longer have to handle their set-up code.

Our example project is called CoreDataMigration-Example; however, as you can see, when creating the NSPersistentContainer, we give CoreDataMigration_Example as our model's name - see Apple's documentation on why the - became a _.

As we only have one Core Data stack, CoreDataManager is a singleton:

static let shared = CoreDataManager()

init(storeType: String = NSSQLiteStoreType, migrator: CoreDataMigratorProtocol = CoreDataMigrator()) {
    self.storeType = storeType
    self.migrator = migrator
}

CoreDataManager is a little odd when it comes to being a singleton in that it has an explicit init implementation. This explicit init method allows for changing the type of persistent store used - by default it's NSSQLiteStoreType; however, when unit testing, we will actually create multiple instances of CoreDataManager using NSInMemoryStoreType to avoid persisting data between tests (and having tests potentially pollute each other). A persistent store type of NSInMemoryStoreType will cause our Core Data stack to only be created in-memory, and so be more cheaply torn down and set up than if we used NSSQLiteStoreType. In the accompanying example project, you can see how this is used in the CoreDataManagerTests class.

Loading the persistent store involves interacting with the disk, which, compared to memory interactions, is more expensive ⏲️; as such, the loadPersistentStores(completionHandler:) method on NSPersistentContainer is asynchronous. This is mirrored by the setup(), loadPersistentStore(completion:) and migrateStoreIfNeeded(completion:) methods:

func setup(completion: @escaping () -> Void) {
   loadPersistentStore {
       completion()
   }
}

private func loadPersistentStore(completion: @escaping () -> Void) {
   migrateStoreIfNeeded {
       self.persistentContainer.loadPersistentStores { description, error in
           guard error == nil else {
               fatalError("was unable to load store \(error!)")
           }

           completion()
       }
   }
}

private func migrateStoreIfNeeded(completion: @escaping () -> Void) {
    guard let storeURL = persistentContainer.persistentStoreDescriptions.first?.url else {
        fatalError("persistentContainer was not set up properly")
    }

    if migrator.requiresMigration(at: storeURL, toVersion: CoreDataMigrationVersion.current) {
        DispatchQueue.global(qos: .userInitiated).async {
            self.migrator.migrateStore(at: storeURL, toVersion: CoreDataMigrationVersion.current)

            DispatchQueue.main.async {
                completion()
            }
        }
    } else {
        completion()
    }
}

Before an attempt is made to load the persistent store, we check if the model needs to be migrated by calling migrateStoreIfNeeded(completion:).

If the answer is yes - the migrator attempts to migrate the user's data. As migrating can be a relatively slow process, the migration happens on a background queue to avoid hanging the UI. Once the migration is completed, the completion closure is called on the main queue.

If the answer is no - the completion closure is called straight away.

Once the persistent store is successfully loaded, the setup() method calls its completion closure, and the stack finishes setting up.

This setup method is called in the AppDelegate:

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
    CoreDataManager.shared.setup {
        self.presentMainUI()
    }

    return true
}

The above code snippet is from the example project where the user is shown a loading screen while the Core Data stack is being set up. Only once the setup is complete is the user allowed into the app proper. presentMainUI switches out the window's root view controller for a navigation stack that can freely use Core Data. While this is strictly not necessary, by splitting the UI into pre and post Core Data stack set up it is possible to avoid race conditions where the app is attempting to use Core Data before it has finished setting up.

💃🥂🎉🕺

Congratulations, that's all there is to the progressive migration approach.

The rest of this post is devoted to putting the above migration approach into practice by migrating an app through 3 Core Data model versions.

Colourful Posts

Colourful Posts is a simple app that allows the user to create posts that are persisted in Core Data. Each post consists of:

• A unique ID.
• A random associated colour represented as a hex string.
• The body/content of the post.
• The date the post was created.

So that the model looks like:

Each post that the user creates is then displayed in a tableview as a brightly coloured cell.

To keep this post to a reasonable length, I won't show any code from Colourful Posts that isn't connected to performing a migration.

It's a simple, fun app that we submit to Apple for approval 🤞.

Migrating to version 2

Despite not being able to edit posts, Apple not only approves Colourful Posts, they love it so much so that they feature it on the Today tab. Colourful Posts is instantly propelled to the top of the charts. After hundreds of thousands of downloads, we decide to hire a new developer to help steer the success-train we find ourselves on 🚂. However, in their first week, the new developer mistakes the information stored in the color property on Post as using RGB rather hex to store the color as a string. Unfortunately, we don't catch this mismatch until it's in production and leads to the app crashing on launch 😞. To avoid this issue from happening when we hire more developers, we decide to rename color to hexColor. As this is a change to the model, we need to create a new model version and handle the migration between the old and new version.

To create a new model version, select the *.xcdatamodel (it may be called *.xcdatamodeld) file in the Project Navigator, open the Editor menu from the top bar and click on the Add Model Version... option. In the wizard that opens, this new model will already be given a name, which typically follows [ModelName] [Number], so CoreDataMigration_Example 2, but this can be changed to whatever you want.

Lightweight migrations are typically a less intensive form of migration than Standard migrations (both from a developer and performance POV) because of this, I prefer to perform Lightweight migrations whenever possible. Lightweight migrations can handle the following transformations to the model:

• Adding an attribute.
• Removing an attribute.
• Changing a non-optional attribute to be optional.
• Changing an optional attribute to non-optional (by defining a default value).
• Renaming an entity, attribute or relationship (by providing a Renaming ID).
• Adding a relationship.
• Removing a relationship.
• Changing the entity hierarchy.

An impressive list of transformations that we get free (or almost free) with Lightweight migrations. The color to hexColor change is covered by the Renaming an entity, attribute or relationship, which has a small caveat: by providing a Renaming ID. The Renaming ID creates a link between the old attribute and the new attribute. All it requires is to add the old attribute name to the new attribute's metadata:

With this information, Core Data now knows that color and hexColor are the same attribute, just with different names and that rather than discarding color during a Lightweight migration, the value should be transferred to hexColor.

With that change, the only thing that's left to do is update CoreDataMigrationVersion to allow migrations from CoreDataMigration_Example to CoreDataMigration_Example 2:

enum CoreDataMigrationVersion: String, CaseIterable {
    case version1 = "CoreDataMigration_Example"
    case version2 = "CoreDataMigration_Example 2"

    //Omitting methods

    func nextVersion() -> CoreDataMigrationVersion? {
    switch self {
    case .version1:
        return .version2
    case .version2:
        return .nil
    }
}

A new case was added to CoreDataMigrationVersion - version2. As with version1, this new version has a raw value which maps to the name of its respective model version - CoreDataMigration_Example 2. nextVersion() has also been updated so that there is a migration path from version1 to version2.

Now that we have a migration path, let's look at unit testing it. Unit testing a migration path requires:

1. Populating a SQLite database using the CoreDataMigration_Example model.
2. Copying that SQLite database into the test target.
3. Asserting that the contents of that SQLite database migrated as expected.

Before copying your SQLite database, it's important to ensure it is, in fact, populated with test data. As we discussed above, Core Data uses Write-Ahead Logging to improve performance, so your data could be residing in the sqlite-wal file rather than the sqlite file. The easiest way to force any uncommitted changes is to fake a migration - add a breakpoint just after the forceWALCheckpointingForStore(at:) method, open the Application Support folder, copy the sqlite file and then abort the migration.

class CoreDataMigratorTests: XCTestCase {

    var sut: CoreDataMigrator!

    // MARK: - Lifecycle

    override class func setUp() {
        super.setUp()

        FileManager.clearTempDirectoryContents()
    }

    override func setUp() {
        super.setUp()

        sut = CoreDataMigrator()
    }

    override func tearDown() {
        sut = nil

        super.tearDown()
    }

    func tearDownCoreDataStack(context: NSManagedObjectContext) {
        context.destroyStore()
    }

    // MARK: - Tests

    // MARK: SingleStepMigrations

    func test_individualStepMigration_1to2() {
        let sourceURL = FileManager.moveFileFromBundleToTempDirectory(filename: "CoreDataMigration_Example_1.sqlite")
        let toVersion = CoreDataMigrationVersion.version2

        sut.migrateStore(at: sourceURL, toVersion: toVersion)

        XCTAssertTrue(FileManager.default.fileExists(atPath: sourceURL.path))

        let model = NSManagedObjectModel.managedObjectModel(forResource: toVersion.rawValue)
        let context = NSManagedObjectContext(model: model, storeURL: sourceURL)
        let request = NSFetchRequest.init(entityName: "Post")
        let sort = NSSortDescriptor(key: "postID", ascending: false)
        request.sortDescriptors = [sort]

        let migratedPosts = try? context.fetch(request)

        XCTAssertEqual(migratedPosts?.count, 10)

        let firstMigratedPost = migratedPosts?.first

        let migratedDate = firstMigratedPost?.value(forKey: "date") as? Date
        let migratedHexColor = firstMigratedPost?.value(forKey: "hexColor") as? String
        let migratedPostID = firstMigratedPost?.value(forKey: "postID") as? String
        let migratedContent = firstMigratedPost?.value(forKey: "content") as? String

        XCTAssertEqual(migratedDate?.timeIntervalSince1970, 1547494150.058821)
        XCTAssertEqual(migratedHexColor, "1BB732")
        XCTAssertEqual(migratedPostID, "FFFECB21-6645-4FDD-B8B0-B960D0E61F5A")
        XCTAssertEqual(migratedContent, "Test body")

        tearDownCoreDataStack(context: context)
    }
}

There is no need to test every object stored in the persistent store; rather, we just have to assert that each entity has the correct number of objects, and then select one object per entity and assert the values on that object.

In the above test, a migration is triggered between the CoreDataMigration_Example and CoreDataMigration_Example 2 models. An interesting point to note is that rather than making use of the Post subclass of NSManagedObject, the above test uses a plain NSManagedObject instance and KVC to determine if the migration was a success. This is to handle the very likely scenario that the Post structure defined in the CoreDataMigration_Example 2 model will not be the final Post structure. If we used Post instances, then as the Post entity changed in later versions of the model, those changes would be mirrored in Post NSManagedObject subclass, which would result in this test potentially breaking. By using plain NSManagedObject instances and KVC it is possible to ensure that this test is 100% accurate to the structure of the Post entity as defined in CoreDataMigration_Example 2 model.

As changes are being made to the file system, the last thing the test does is tear down the Core Data stack using the tearDownCoreDataStack(context:) method.

Just deleting the migrated SQLite files from the file system would result in a rather serious-sounding error BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use:.... being printed to the console. This is because the store would be being deleted from under an active Core Data stack. While the active Core Data stack in question will then be discarded, resulting in this error not actually creating any issues, having it clutter the console would make it that much harder to read and spot any genuine issues printed there, so it's best to tear things down properly.

In the above test class, a few extensions are being used to make things easier:

extension FileManager {

    // MARK: - Temp

    static func clearTempDirectoryContents() {
        let tmpDirectoryContents = try! FileManager.default.contentsOfDirectory(atPath: NSTemporaryDirectory())
        tmpDirectoryContents.forEach {
            let fileURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent($0)
            try? FileManager.default.removeItem(atPath: fileURL.path)
        }
    }

    static func moveFileFromBundleToTempDirectory(filename: String) -> URL {
        let destinationURL = URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true).appendingPathComponent(filename)
        try? FileManager.default.removeItem(at: destinationURL)
        let bundleURL = Bundle(for: CoreDataMigratorTests.self).resourceURL!.appendingPathComponent(filename)
        try? FileManager.default.copyItem(at: bundleURL, to: destinationURL)

        return destinationURL
    }
}

extension NSManagedObjectContext {

    // MARK: Model

    convenience init(model: NSManagedObjectModel, storeURL: URL) {
        let persistentStoreCoordinator = NSPersistentStoreCoordinator(managedObjectModel: model)
        try! persistentStoreCoordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: storeURL, options: nil)

        self.init(concurrencyType: .mainQueueConcurrencyType)

        self.persistentStoreCoordinator = persistentStoreCoordinator
    }

    // MARK: - Destroy

    func destroyStore() {
        persistentStoreCoordinator?.persistentStores.forEach {
            try? persistentStoreCoordinator?.remove($0)
            try? persistentStoreCoordinator?.destroyPersistentStore(at: $0.url!, ofType: $0.type, options: nil)
        }
    }
}

As stated above, I won't expand on the extension methods.

Migrating to version 3

After another successful release, we decide to expand our posting functionality by allowing the user to add multiple sections to a post. These sections will be stored alongside the post in Core Data. As with any model change, we need to create a new model version: CoreDataMigration_Example 3.

Each section consists of:

• A title.
• A body.
• An index.

Which in turn reduces a post to:

• A unique ID.
• A random associated colour represented as a hex string.
• The date the post was created.
• A collection of sections.

Such that:

Migrating from CoreDataMigration_Example 2 to CoreDataMigration_Example 3 is slightly trickier than the previous migration, as CoreDataMigration_Example 2 splits an existing entity into two entities and creates a relationship between them. This will require implementing both a mapping model and a migration policy.

To create a mapping model, open the File menu on the top bar, then click on New File->New. In the window that opens, scroll down to the Core Data section and double-tap on Mapping Model. This will open a wizard where you can select your source and destination model versions, so in this case: CoreDataMigration_Example 2 and CoreDataMigration_Example 3. After that, you need to give the mapping a name and save it. I tend to follow Migration[sourceVersion]to[destinationVersion]ModelMapping as a naming convention, so Migration2to3ModelMapping.

A mapping model defines the transformations required to migrate from the source model to the destination model. In Xcode, a mapping model is an xcmappingmodel file that, when opened, has a GUI that's very similar to the Core Data Model GUI. A mapping model handles mapping between entities, attributes and relationships. The mapping model GUI even allows for simple transformations. If the model had a percentage attribute that used to have a value between 0 - 100 but in the new model that value should be between 0 - 1, we could use the Expression field on that attribute to perform this transformation by setting the expression to: $source.percentage/100. Despite the range of transformations possible within the mapping model GUI, some changes are just too complex and require a more custom approach - this is handled by creating a migration policy. A migration policy is an NSEntityMigrationPolicy subclass that defines how to map between two entities from two different model versions using the full Core-Data/Swift toolkit.

Migrating from CoreDataMigration_Example 2 to CoreDataMigration_Example 3 will require a custom migration policy as we will need to move the current content attribute's value on Post to both the title and body attributes on a newly created Section instance:

final class Post2ToPost3MigrationPolicy: NSEntityMigrationPolicy {

    override func createDestinationInstances(forSource sourcePost: NSManagedObject, in mapping: NSEntityMapping, manager: NSMigrationManager) throws {
        try super.createDestinationInstances(forSource: sourcePost, in: mapping, manager: manager)

        guard let destinationPost = manager.destinationInstances(forEntityMappingName: mapping.name, sourceInstances: [sourcePost]).first else {
            fatalError("was expected a post")
        }

        let sourceBody = sourcePost.value(forKey: "content") as? String
        let sourceTitle = sourceBody?.prefix(4).appending("...")

        let section = NSEntityDescription.insertNewObject(forEntityName: "Section", into: destinationPost.managedObjectContext!)
        section.setValue(sourceTitle, forKey: "title")
        section.setValue(sourceBody, forKey: "body")
        section.setValue(destinationPost, forKey: "post")
        section.setValue(0, forKey: "index")

        var sections = Set()
        sections.insert(section)

        destinationPost.setValue(sections, forKey: "sections")
    }
}

Just like with mapping models, I have a naming convention for migration policies: [Entity][Version]To[Entity][Version]MigrationPolicy, this way, I can know at a glance exactly what the migration policy is doing.

The above migration policy overrides createDestinationInstances(forSource:in:manager) to allow for transforming existing CoreDataMigration_Example 2 model Post instances into CoreDataMigration_Example 3 model Post and Section instances. Again, in order to interact with attributes on each Post instance, we need to use KVC. First, a new CoreDataMigration_Example 3 model Post (destinationPost) is created using the mapping rules defined in the mapping model (these rules are set in the mapping model GUI). Then a Section instance from the new Section entity. As the old Post didn't have the concept of a title, we take the first 4 characters of that older post's body value and combine it with ... so that it can be used as the title of the new Section instance. After setting the other properties of the section, a relationship between this section and the new post is created.

For this migration policy to be used during the migration, we need to add it to the mapping model by setting the Custom Policy on the PostToPost entity mapping:

It's important to note that the migration policy class name is prefixed with the module name.

All that's left to do is to update: CoreDataMigrationVersion by introducing a version3 case and updating nextVersion:

enum CoreDataMigrationVersion: String, CaseIterable {
    case version1 = "CoreDataMigration_Example"
    case version2 = "CoreDataMigration_Example 2"
    case version3 = "CoreDataMigration_Example 3"

    //Omitting methods

    func nextVersion() -> CoreDataMigrationVersion? {
    switch self {
    case .version1:
        return .version2
    case .version2:
        return .version3
    case .version3:
        return nil
    }
}

And that's it - we now have a migration path from not only CoreDataMigration_Example 2 to CoreDataMigration_Example 3 but also from CoreDataMigration_Example to CoreDataMigration_Example 3.

Check out CoreDataMigratorTests for the unit test that supports this migration.

Migrating to version 4

The success of Colourful Posts knows no bounds, and we decide to release our next killer feature: deleting posts. This deletion functionality is actually a soft delete, which means that the post will still exist in Core Data but won't be shown to the user. We can achieve this by adding a new attribute to the Post entity - softDelete. Of course, this change will require a new model version and for us to handle the migration to that version. This migration can be handled as a Lightweight migration and, in fact, requires very little effort on our part. We only need to add a new case to CoreDataMigrationVersion and update nextVersion:

enum CoreDataMigrationVersion: String, CaseIterable {
    case version1 = "CoreDataMigration_Example"
    case version2 = "CoreDataMigration_Example 2"
    case version3 = "CoreDataMigration_Example 3"
    case version4 = "CoreDataMigration_Example 4"

    // Omitted methods

    // MARK: - Migration

    func nextVersion() -> CoreDataMigrationVersion? {
        switch self {
        case .version1:
            return .version2
        case .version2:
            return .version3
        case .version3:
            return .version4
        case .version4:
            return nil
        }
    }
}

Check out CoreDataMigratorTests for the unit test that supports this migration.

We got there 🏁

Core Data migration can often seem like a tedious and cumbersome process that punishes developers for mutating their models. However, hopefully, this post shows that by diverging from the default one-step migration approach, we can simplify the process and significantly cut down the amount of work required to perform a successful migration. This simplification makes it much easier to treat our users' data with the care that I hope others treat my data with.

I want to acknowledge that I based the above approach on the migration example shown in the excellent Core Data book by Florian Kugler and Daniel Eggert, which you can get here. I would highly recommend that you give that book a read as it's a treasure trove of Core Data knowledge.

In our day-to-day life, dates are pretty straightforward - we read them all the time, make plans around them, and share them with other people. Apart from the occasional missed birthday party, all of these date-based tasks go surprisingly smoothly, which is remarkable when you stop to think how complex our date systems are. It works this smoothly because everyone is making some fairly large, unspoken assumptions about the dates they see - what calendar is used, what time zone is used, the ordering of date elements, etc. While for most of us, these assumptions rarely create issues in our day-to-day lives, if we want to build a system that uses dates, we need to identify the assumptions we are making and remove them. For example take the following date:

02/12/06

If you are from the UK, then you would read this as:

2nd of December 2006

Whereas if you are from the US, then you would read this as:

February 12th, 2006

Both of these interpretations are valid; however, only one is correct. If the developer is from the UK, then the former interpretation is correct, and any US users will be either confused when they start seeing dates like 14/12/06 😕 or angry when they show up to an appointment on the wrong day 🤬. It's easy to fall into this date ordering trap, especially if everyone on the development and testing teams is working off the same set of date assumptions and conventions. This is the real danger with assumptions; often, we don't realise that we are making them.

Getting to know what's local

When it comes to formatting dates, we can choose to take the user's conventions into account or not, so for example, with the above date example 02/12/06, if we want to confuse our US users, we could hardcode the date element order to match UK conventions by:

let dateFormatter = DateFormatter()
dateFormatter.dateFormat = "dd/MM/yy"
let formattedDate = dateFormatter.string(from: date)

It's not uncommon to see this approach to configuring a DateFormatter, and while there are valid scenarios for hardcoding the dateFormat value (we will see an example of that later), I can't think of any valid scenarios for doing so when displaying a date to a user. By hardcoding the dateFormat value, we instruct the DateFormatter to disregard the user's date element ordering conventions and instead use a consistent element ordering for all users. By not meeting our user's date element ordering expectations, we degrade their experience by making something that should be as simple as reading a date into a surprise maths puzzle 📖. I've seen several novel but naive approaches to solving the expectation mismatch that a hardcoded dateFormat produces. These solutions tend to centre on two approaches:

1. Add conditional logic and set a dateFormat value that's unique for each user's conventions.
2. Move the dateFormat value into the localisation .strings file.

While it's possible to meet our user's expectations with either of these solutions, both have two major drawbacks:

1. The developer needs to actively decide which conventions each DateFormatter will support.
2. The developer needs to take on the responsibility of ensuring that any date and time formatting rules are correct for each supported set of conventions.

In order to define the formatting rules for each dateFormat value, the developer would need to answer questions such as:

• Which calendar to use?
• Which clock to use: 24-hour or 12-hour?
• What is the ordering of date elements?
• What is the date separator character(s)?

And the list goes on.

Answering all these questions correctly is a massive task and, as it turns out, an unnecessary one as iOS already answers them (and many more besides) for us. In iOS, the linguistic, cultural and technological conventions are collected within the Locale class (each locale's conventions are provided by the Unicode Common Locale Data Repository (CLDR) project). The conventions defined within each locale should match our user's expectations for how their world is measured and represented; as such, Locale plays a vital role in making our users comfortable when using our apps to complete their tasks. By working with the user's locale conventions, it's possible to produce formatted dates that match our user's expectations. Even better than just improving our user's in-app experience, this can all be achieved without actually having to know or care what those locale conventions actually are.

While each locale comes with default conventions, some of them can be customised by the user e.g. switching from 12-hour to 24-hour clock representation. So even if two users have the same locale, it would be a mistake to assume these locales were identical.

For the rest of this post, I'm going to assume your locale is the default US locale. To keep the examples comparable, each will use a date based on the Unix epoch timestamp: 1165071389, which equates to 2nd of December, 2006 at 14:56:29. You can see the completed playground with all of the below examples by following this link.

Staying local

When it comes to presenting date information to the user, it's best to leave all locale concerns to DateFormatter - there are two ways to do this:

1. Using DateFormatter.Style
2. Using a template

Let's explore both these approaches in more detail.

Using DateFormatter.Style

One way to ensure that a Date is always displayed to the user in a formatting style they expect is to use the DateFormatter.Style enum. The DateFormatter.Style enum is a set of predefined values that correspond to a date format, at the time of writing (iOS 12) this enum has 5 possible cases to choose from:

1. none
2. short
3. medium
4. long
5. full

Each case will take into account the user's locale settings when formatting dates. DateFormatter has two DateFormatter.Style properties: dateStyle and timeStyle. Each property works semi-independently of the other, and as such, each can take a different value - this flexibility allows for a wide range of formatting options (25 possible permutations):

let dateFormatter = DateFormatter()
dateFormatter.dateStyle = .full
dateFormatter.timeStyle = .medium
let formattedDate = dateFormatter.string(from: date)

formattedDate would be set to Saturday, December 2, 2006 at 2:56:29 PM.

In the above example, we can see the user's locale influencing the formattedDate value in several ways:

1. The date format ordering - `EEEE, MMMM d, y 'at' h:m:s a'.
2. The names of the months and days.
3. The separator between the various date elements - / for calendar date elements and : for time date elements.

It's also interesting to note that DateFormatter handles combining the calendar date and time elements into a sentence structure that is appropriate for both DateFormatter.Style values. So in the above example, at is being used as a connector between those two elements however, if dateStyle was changed to .short, then at becomes , and we get 12/2/06, 2:56:29 PM as the formatted date should be shown in a more compact form - pretty powerful stuff 🤯.

To demonstrate the power of this further, if we set the locale to Locale(identifier: "de_DE"), the above .full example becomes Samstag, 2. Dezember 2006 um 14:56:29 and the .short example becomes 02.12.06, 14:56:29 - all this localisation without me having to know anything about German date conventions or even anything about the German language - 🤯 in 🇩🇪.

Another way to use the DateFormatter.Style approach is by using the available class method:

let formattedDate = DateFormatter.localizedString(from: date, dateStyle: .short, timeStyle: .medium)

formattedDate has the same value as the property-based approach: 12/2/06, 2:56:29 PM.

When deciding which option to use I tend to favour the property(s) approach when I either need to cache the DateFormatter instance being used (see "Sneaky date formatters exposing more than you think" for more details on how to cache date formatters) or further customise it (outside of just the dateStyle and/or timeStyle properties). For all other scenarios, I favour the class method approach as it's easier to reason about and reads better (from the method name I know that I'm getting a localised string value back).

This is a helpful cheatsheet showing the output for each DateFormatter.Style case. To see the conventions used in different locales, set the locale property for the DateFormatter instance to the locale you are interested in e.g. dateFormatter.locale = Locale(identifier: "en_CA").

Using DateFormatter.Style works well if you want to display a date in one of the available formats, but if you want to display a custom format, you need to take a different path.

Using a template

A template is a customised instruction to DateFormatter of the date elements that the formatted date should contain - these date elements are specified using the same symbols as used with a fixed string date format approach: d', MM, y' etc. The beauty of the template approach is that it will take this customised instruction and produce a formatted date that takes into account the user's locale when displaying those elements. For example, to produce a formatted date only containing the day and month, the template could look like:

let dateFormatter = DateFormatter()
dateFormatter.setLocalizedDateFormatFromTemplate("d MM")
let formattedDate = dateFormatter.string(from: date)

formattedDate would be set to 12/2. The more eagle-eyed among you will have spotted that two transformations have occurred here. The first is that a locale-specific separator was added between the date elements, and the second is that the date elements have switched positions from the ordering in the template i.e. from 'd MMtoMM d' 🚀.

It's important to note that when defining the above template, I added a space between the different date elements. Strictly speaking, this space wasn't needed - dMM would have resulted in the same formattedDate value. I included the space here only to make the template easier to read.

Just like with the DateFormatter.Style, the template approach has both an instance and a class interface:

let localisedDateFormat = DateFormatter.dateFormat(fromTemplate: "d MM", options: 0, locale: Locale.current)
let dateFormatter = DateFormatter()
dateFormatter.dateFormat = localisedDateFormat
let formattedDate = dateFormatter.string(from: date)

Again, formattedDate would be set to 12/2. The class approach is a little more verbose than the instance approach, but could be more useful if you wanted to pass the localised formatting string around rather than a DateFormatter instance.

Going remote

You may be thinking at this point:

"Is using dateFormat with a fixed string ever the correct approach?"

The simple answer:

Yes.

The date formatting examples shown so far have focused on presenting a formatted date to the user. However, as iOS developers, we (often) have another consumer of our data: the backend. Typically, the backend will be expecting all date values to be sent using a fixed, locale-neutral format. To ensure that the user's locale does not affect these dates when converting from a Date instance to the formatted date value, we need to take complete control of our DateFormatter instances and hardcode the locale, timeZone and dateFormat properties to match the backend's expectations:

let dateFormatter = DateFormatter()
dateFormatter.locale = Locale(identifier: "en_US_POSIX")
dateFormatter.timeZone = TimeZone(identifier: "UTC")
dateFormatter.dateFormat = "yyyy-MM-dd'T'HH:mm:ss'Z'"
let formattedDate = dateFormatter.string(from: date)

formattedDate would be set to 2006-12-02T14:56:29Z regardless of the user's locale settings. An interesting side effect of setting the locale property is that the formatter's calendar property is always set to the default calendar for that locale (which for this locale is the Gregorian calendar). It's important to note that en_US_POSIX is not the same locale as en_US. While en_US_POSIX is based on en_US, it is a special locale that isn't tied to any country/region, so it shouldn't change even if en_US does change.

You may have noticed that the dateFormatter is using the ISO 8601 string format. In this case (and if your project has a base iOS version of at least iOS 10), I would recommend using ISO8601DateFormatter instead of the more generic DateFormatter class.

Letting go of (some) control

When displaying a date in our apps, it makes sense to do so in a format that the user expects and can easily understand - this isn't as simple as it first seems. Thankfully, DateFormatter has two very straightforward ways to achieve this:

1. Using DateFormatter.Style
2. Using a template

Both approaches work well and require minimal effort to solve what is a real iceberg of a problem. The only thing it costs us is that we need to give up a little bit of control on exactly how the date is formatted - that's a price I'm happy to pay to ensure that my users are shown dates in the most convenient format for them and means I don't need to think about which calendar a specific locale uses or if month comes before day, etc.

Now I need to convince those pixel-perfect designers on my team that giving up some control is actually a good thing... 😅

If you are interested, there is an accompanying playground that contains all of the above code snippets.

Ask any iOS developer about DateFormatter and one of the first things you will hear is that creating a DateFormatter instance is an expensive operation. The second thing you will hear is that after creating one, you need to cache it.

In this post, I want to examine the cost of creating DateFormatter instances and explore how we can cache them effectively.

To determine just how expensive they are, we will need to conduct a small experiment 🔬.

If you'd prefer not to have to build up the below examples manually, you can download the completed playground here.

The expense experiment

Our experiment will have two scenarios:

1. Create a new instance of DateFormatter for each date conversion.
2. Reuse the same instance of DateFormatter for all date conversions.

class DateConverter {

    let dateFormat = "y/MM/dd @ HH:mm"

    func convertDatesWithUniqueFormatter(_ dates: [Date]) {
        for date in dates {
            let dateFormatter = DateFormatter()
            dateFormatter.locale = Locale(identifier: "en_US_POSIX")
            dateFormatter.dateFormat = dateFormat

            _ = dateFormatter.string(from: date)
        }
    }

    func convertDatesWithReusedFormatter(_ dates: [Date]) {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = dateFormat

        for date in dates {
            _ = dateFormatter.string(from: date)
        }
    }
}

convertDatesWithUniqueFormatter represents the first scenario and convertDatesWithReusedFormatter the second scenario. Both methods follow a similar structure: loop over an array of dates and format each date into a string representation. The only difference is in how DateFormatter is used.

The XCTest framework makes it straightforward to measure the performance of our code using the appropriately named measure(_:) method. measure(_:) tracks the time in seconds that it takes for the code under test to finish executing. As factors outside of our control can affect performance, measure(_:) executes the same test 10 times and reports the average time as its result ⏲️.

class DateConverterTests: XCTestCase {
    var sut: DateConverter!
    let dates = Array(repeating: Date(), count: 100)

    // MARK: - Lifecycle

    override func setUp() {
        super.setUp()
        sut = DateConverter()
    }

    override func tearDown() {
        sut = nil
        super.tearDown()
    }

    // MARK: - Tests

    func test_convertDatesWithUniqueFormatter_performance() {

        measure {
            sut.convertDatesWithUniqueFormatter(dates)
        }
    }

    func test_convertDatesWithReusedFormatter_performance() {

        measure {
            sut.convertDatesWithReusedFormatter(dates)
        }
    }
}

Running the two above tests results in generating the following console output:

Of course, running the tests on your computer will give you different results.

The above results show that when converting 100 dates, reusing a DateFormatter instance takes on average 20% (0.02 vs 0.004 seconds) of the time that creating a unique DateFormatter instance for each conversion does. While, in real terms, the difference in time isn't much, as a percentage, the difference is quite conclusive - reusing a DateFormatter instance is five times cheaper. If we are thinking in terms of UX, the additional overhead from always creating a new instance of DateFomatter could be the difference between a smooth scrolling table view and a jumpy one.

Premature optimisation is the mother of many bugs, so before deciding to improve the performance of any one area, it's important to make sure that that area is actually a performance bottleneck - Instruments and Xcode itself are great tools for profiling. By optimising only actual performance bottlenecks, we can ensure that we are not wasting time and not unnecessarily making our codebase more complex than it needs to be.

Working with performant DateFormatters

Now that it's been determined that reusing a DateFormatter instance offers a performance improvement and we've identified that this performance improvement will lead to a better user experience, the question is:

"How do we reuse it?"

It's simple enough to extract the DateFormatter instance into a local variable or private property so it can be reused, and I won't explore these here - things really start to get interesting when the same formatter is needed across the project.

While normally derided as an overused pattern, I think sharing formatters would be greatly served by using the Singleton pattern:

class DateFormattingHelper {

    // MARK: - Shared

    static let shared = DateFormattingHelper()

    // MARK: - Formatters

    let dobDateFormatter: DateFormatter = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "y/MM/dd @ HH:mm"

        return dateFormatter
    }()
}

If we then wanted to format Date instances into date-of-birth strings, we could add a method similar to:

func formatDOB(_ date: Date, with dateFormatter: DateFormatter) -> String {
    let formattedDate = dateFormatter.string(from: date)
    return ("Date of birth: \(formattedDate)")
}

And when it's called, pass in the shared DateFormatter instance that is a property on DateFormattingHelper singleton like:

let dateFormatter = DateFormattingHelper.shared.dobDateFormatter
let dobFormattedString = formatDOB(date, with: dateFormatter)

It's a simple, easy-to-read and easy-to-test approach that ensures that DateFormatter instances are being reused. It's not too hard to imagine adding more DateFormatter properties to DateFormattingHelper and more helper methods like formatDOB to our view controllers to meet all of our date formatting needs.

But before we get too carried away with our success, let's examine our use of the Singleton pattern here in more detail. A key design consideration when using any pattern to expose shared state is: mutability. Ideally shared state should be immutable to prevent a situation where different parts of the codebase can change that state and so indirectly affect other parts that depend on that shared state. However, as we have seen, DateFormattingHelper only has one property, and that property is a constant (let), so mutability shouldn't be an issue here 👏.

Or is it? 🤔

Swift has two categories of Type:

1. Value - structs, enums, or tuples
2. Reference - classes, functions or closures

The main difference between the types is shown when they are used in an assignment.

When a value-type instance is assigned, an exact copy of that instance is made, and it is this copy that is then assigned. This results in two instances of that value-type existing, with both instances (at least initially) having the same property values:

struct PersonValueTypeExample: CustomStringConvertible {
    var name: String
    var age: Int

    var description: String {
        return ("name: \(name), age: \(age)")
    }
}

var a = PersonValueTypeExample(name: "Susie", age: 29)
var b = a
b.name = "Samantha"
a.age = 56

print("a: \(a)") // prints "a: name: Susie, age: 56"
print("b: \(b)") // prints "b: name: Samantha, age: 29" 

CustomStringConvertible is used here only to override the description property and ensure a consistent printout format between the value-type and reference-type examples.

As the above example shows, when a' is assigned to b, a copy of a' is made and assigned to b. Any change made to either a' or b` will be limited to that instance only - this can be seen in the description string printed for both.

Copying a value-type instance can be an expensive process, so several techniques are used to try to avoid making the copy. These techniques range from compiler optimisations to implementation details within the struct itself, such as Copy-On-Write. These optimisation techniques can result in our value-type instances behaving more like a reference-type under the hood. While this is useful to know, the important thing to note is that these optimisations are transparent to us as users of a given value-type. So we can build a system which is dependent upon the difference between value and reference types without needing to concern ourselves with how the instances of these types are actually stored in memory and accessed.

When a reference-type instance is assigned, a reference (or pointer) to that instance is created, and it is this reference that is then actually assigned (rather than the instance itself). This can result in an instance having multiple references so that the instance becomes an informal shared instance to those references:

class PersonReferenceTypeExample: CustomStringConvertible {
    var name: String
    var age: Int

    var description: String {
    return ("name: \(name), age: \(age)")
    }

    init(name: String, age: Int) {
        self.name = name
        self.age = age
    }
}

let c = PersonReferenceTypeExample(name: "Susie", age: 29)
var d = c
d.name = "Samantha"
c.age = 56

print("c: \(c)") // prints "c: name: Samantha, age: 56"
print("d: \(d)") // prints "d: name: Samantha, age: 56" 

As the above example shows, when c is assigned to d, a new reference to the same instance that c is pointing at is created and assigned to d. Any change made to either c or d will affect the other - this can be seen in the description string printed for both.

The sharper-eyed among you may have spotted that the above examples differ in one additional aspect other than just class/struct and the naming of the variables - in the struct example a is defined as a var whereas in the class example c is a let. When a value-type instance is assigned to a constant, all of its properties are treated as constant properties (and any mutating methods are no longer available) - this isn't true for reference-type instances, where var and let are always respected.

You may have heard of pass-by-reference and pass-by-value and currently be thinking that these terms are connected to the above types. This is a common misunderstanding; by default, Swift uses pass-by-value when passing both value-type and reference-type instances. The only difference, as we have seen, is what is copied - with a value-type it's the entire instance, with a reference-type instance it's a reference/pointer. Why this is the case took me a while to get my head around fully, I found the following Stack Overflow question very informative, as well as the wiki page on Evaluation Strategy especially the "Call by sharing" section which I think best describes what happens when we pass a reference-type into a method and why that isn't pass-by-reference. Using the linked wiki definition of pass-by-reference, the nearest that Swift gets to it is using the inout keyword.

While an interesting detour, you may be thinking:

"What does any of it have to do with DateFormattingHelper?"

Well, dobDateFormatter is a reference-type, so when passing it around, new references are being made that point to the same shared DateFormatter instance. On first glance, that's fine because dobDateFormatter is a constant property and so is immutable; however, the trouble lies in the fact that some of the properties on DateFormatter are mutable. As we saw above (unlike a value-type instance), a reference-type instance's property declarations are not affected by how that instance was defined. So while it looks like in DateFormattingHelper we have immutable shared state, we actually have mutable shared state - just hidden one layer deeper.

A bug that is introduced by changing shared state is often discovered not where the shared state is changed, but elsewhere in the codebase where the previous state of the shared state was expected - think about what the consequences of changing the badgeFormat value on dobDateFormatter would be. Even then, the other part of the app will only surface the bug if the code that introduces the bug is called first, i.e. the bug is dependent on how the user moves through the app 😱 - a nightmare of a bug that can take a long time to track down and resolve. We can avoid the many sleepless nights this bug would cause by turning hidden mutable properties into immutable properties.

Going from mutable to immutable is actually surprisingly straightforward in this example; we just need to hide our DateFormatter instances behind a protocol. When it comes to using the dobDateFormatter property above from outside the DateFormattingHelper class, we are actually only interested in using a DateFormatter to convert a Date into a String so we can produce an extremely limited protocol:

protocol DateFormatterType {
    func string(from date: Date) -> String
}

We just need to conform DateFomatter to our custom DateFormatterType using an extension:

extension DateFormatter: DateFormatterType { }

Now that we have this protocol, we can return DateFomatter instances wrapped up as DateFormatterType instances:

let dobDateFormatter: DateFormatterType = {
    let dateFormatter = DateFormatter()
    dateFormatter.locale = Locale(identifier: "en_US_POSIX")
    dateFormatter.dateFormat = "y/MM/dd @ HH:mm"

    return dateFormatter
}()

So whenever we use DateFomatter, we would use DateFormatterType instead:

func formatDOB(_ date: Date, with dateFormatter: DateFormatterType) -> String {
    let formattedDate = dateFormatter.string(from: date)
    return ("Date of birth: \(formattedDate)")
}

In fact, we can take this even further by absorbing the formatDOB method into DateFormattingHelper and hide DateFormatterType altogether:

private protocol DateFormatterType {
    func string(from date: Date) -> String
}

class DateFormattingHelper {

    // MARK: - Shared

    static let shared = DateFormattingHelper()

    // MARK: - Formatters

    private let dobDateFormatter: DateFormatterType = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "y/MM/dd @ HH:mm"

        return dateFormatter
    }()

    // MARK: - DOB

    func formatDOBDate(_ date: Date) -> String {
        let formattedDate = dobDateFormatter.string(from: date)
        return ("Date of birth: \(formattedDate)")
    }
}

DateFormatterType and dobDateFormatter are now private and so only accessible from inside DateFormattingHelper. At this point, you may be thinking:

"What's the point of still having the DateFormatterType protocol?"

I think it's important to understand that the issue that DateFormatterType is solving isn't one of malicious intent from someone deliberately trying to crash the app, but from a trusted colleague who, through either clumsiness or a lack of domain knowledge, accidentally misuses a DateFormatter instance. So if we made the dobDateFormatter property private, it would still be possible to introduce this type of bug into DateFormattingHelper (by modifying the properties of e.g. dobDateFormatter from inside formatDOBDate); however, by continuing to use the protocol approach, we can actually reduce this risk further. By being both private and hidden behind a protocol, we protect against unintended mutations of a shared instance, both external and internal to DateFormattingHelper.

Going a step beyond

Now that we have the basic mechanisms in place, let's look at a more complete version of DateFormattingHelper:

class DateFormattingHelper {

    // MARK: - Shared

    static let shared = DateFormattingHelper()

    // MARK: - Formatters

    private let dobDateFormatter: DateFormatterType = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "y/MM/dd @ HH:mm"

        return dateFormatter
    }()

    private let dayMonthTimeDateFormatter: DateFormatterType = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "dd MMM @ HH:mm"

        return dateFormatter
    }()

    private let hourMinuteDateFormatter: DateFormatterType = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "HH:mm"

        return dateFormatter
    }()

    private let dayMonthYearDateFormatter: DateFormatterType = {
        let dateFormatter = DateFormatter()
        dateFormatter.locale = Locale(identifier: "en_US_POSIX")
        dateFormatter.dateFormat = "d MMM 'of' y"

        return dateFormatter
    }()

    // MARK: - DOB

    func formatDOBDate(_ date: Date) -> String {
        let formattedDate = dobDateFormatter.string(from: date)
        return ("Date of birth: \(formattedDate)")
    }

    // MARK: - Account

    func formatLastActiveDate(_ date: Date, now: Date = Date()) -> String {
        let yesterday = Calendar.current.date(byAdding: .day, value: -1, to: now)!

        var dateFormatter = dayMonthTimeDateFormatter
        if date > yesterday {
            dateFormatter = hourMinuteDateFormatter
        }

        let formattedDate = dateFormatter.string(from: date)
        return ("Last active: \(formattedDate)")
    }

    // MARK: - Post

    func formatPostCreatedDate(_ date: Date) -> String {
        let formattedDate = dayMonthYearDateFormatter.string(from: date)
        return formattedDate
    }

    // MARK: - Commenting

    func formatCommentedDate(_ date: Date) -> String {
        let formattedDate = dayMonthTimeDateFormatter.string(from: date)
        return ("Comment posted: \(formattedDate)")
    }
}

This gives a better idea of what DateFormattingHelper can become. However, you can quickly see how just adding three more properties has bloated this class. There is room for further optimisation here.

In the above example, the only configuration difference between the DateFomatter instances is the dateFormat value. This shared configuration can be used to our advantage to remove the distinct DateFormatter properties:

class CachedDateFormattingHelper {

    // MARK: - Shared

    static let shared = CachedDateFormattingHelper()
    
    // MARK: - Queue
    
    let cachedDateFormattersQueue = DispatchQueue(label: "com.boles.date.formatter.queue")

    // MARK: - Cached Formatters

    private var cachedDateFormatters = [String : DateFormatterType]()

    private func cachedDateFormatter(withFormat format: String) -> DateFormatterType {
        return cachedDateFormattersQueue.sync {
            let key = format
            if let cachedFormatter = cachedDateFormatters[key] {
                return cachedFormatter
            }
            
            let dateFormatter = DateFormatter()
            dateFormatter.locale = Locale(identifier: "en_US_POSIX")
            dateFormatter.dateFormat = format
            
            cachedDateFormatters[key] = dateFormatter
            
            return dateFormatter
        }
    }

    // MARK: - DOB

    func formatDOBDate(_ date: Date) -> String {
        let dateFormatter = cachedDateFormatter(withFormat: "y/MM/dd @ HH:mm")
        let formattedDate = dateFormatter.string(from: date)
        return ("Date of birth: \(formattedDate)")
    }

    // MARK: - Account

    func formatLastActiveDate(_ date: Date, now: Date = Date()) -> String {
        let yesterday = Calendar.current.date(byAdding: .day, value: -1, to: now)!

        var dateFormatter = cachedDateFormatter(withFormat: "dd MMM @ HH:mm")
        if date > yesterday {
            dateFormatter = cachedDateFormatter(withFormat: "HH:mm")
        }

        let formattedDate = dateFormatter.string(from: date)
        return ("Last active: \(formattedDate)")
    }

    // MARK: - Post

    func formatPostCreatedDate(_ date: Date) -> String {
        let dateFormatter = cachedDateFormatter(withFormat: "d MMM 'of' y")
        let formattedDate = dateFormatter.string(from: date)
        return formattedDate
    }

    // MARK: - Commenting

    func formatCommentedDate(_ date: Date) -> String {
        let dateFormatter = cachedDateFormatter(withFormat: "dd MMM @ HH:mm")
        let formattedDate = dateFormatter.string(from: date)
        return ("Comment posted: \(formattedDate)")
    }
}

Above, we have added a private dictionary (cachedDateFormatters) where all the DateFormatter instances are stored and an accessor method (cachedDateFormatter) to determine if a new instance of DateFormatter needs to be created or not based on if it exists in that dictionary, with the dateFormat value itself acting as the key.

Setting or getting a cached DateFormatter instance has been wrapped in a sync operation on its own queue to make using cachedDateFormatter thread safe - effectively, the cachedDateFormattersQueue queue here is acting as a lock.

Credit for the above dictionary approach must be given to Jordon Smith, who describes it in this post.

Of course, if further customisation is needed in each of the formatters, this approach isn't suitable; however, for this example, it offers an elegant solution.

In the completed playground accompanying this post, I've added the same tests to CachedDateFormattingHelper that I use for DateFormattingHelper to show that externally both implementations behave the same way.

Looking back at those dates

In this post we questioned the common belief that creating DateFormatter instances is expensive (yes they are), looked at ways that we could avoid paying too much for them (local variables, properties and singletons), delved deep on why exposing shared DateFormatter instances was a bad idea (unexpected shared mutable state), how we could avoid that shared mutable state (helper methods and an immutable protocol) and even how we can take caching DateFormatter instances a step beyond (hidden dictionary). Overall, we covered a lot of ground, and if you've made it this far (as you have), I believe you deserve whatever you've been denying yourself all day 😉.

For me, it's a lovely hot chocolate with marshmallows ☕ - I can guarantee no unexpected shared state there!

Networking is central to how most apps add value to their users, which is why significant effort has been put into creating an easy-to-use networking layer by both Apple and the open-source community.

When iOS was first released, NSURLConnection was the native networking suite it shipped with. However, the NSURLConnection story actually began long before iOS, as it was first released in 2003 with Safari; this meant that NSURLConnection was never designed to support the magnitude of tasks it ultimately had to support. These add-ons resulted in NSURLConnection having a number of sharp edges - such as how much programming effort was required to construct the response of a request: gradually building up an NSMutableData instance by responding to the delegate calls. This made networking trickier than it needed to be. A significant portion of the code for making network requests was boilerplate code, which allowed us to abstract it away into base/parent classes that could then be reused for multiple requests. This not-quite-fit-for-purpose networking approach, coupled with boilerplate code, resulted in us creating generic, well-encapsulated networking layers to hide all that complexity away.

Then, great third-party libraries like AFNetworking began to appear, operating on top of NSURLConnection and aiming to soften those sharp edges. AFNetworking achieved this by implementing the boilerplate required by NSURLConnection and presenting a developer-friendly, closure-based interface. This closure-based networking approach proved to be hugely successful, and eventually Apple decided to embrace it with URLSession. However, not everything was rosy 🥀 - due to the ease with which we could now make networking requests and process the responses, you started to see a lot more networking code implemented directly in view controllers 💥. These bloated view controllers became responsible for multiple domains - interacting with views, making network requests, and parsing network responses into model objects. Classes that combine multiple domains are trickier to reason about, maintain and extend, which is why the single responsibility principle is a widespread design practice.

In this post, I want to explore how we can build an app that avoids combining multiple domains in its view controllers by utilising a distinct networking layer using URLSession and Operation.

Thinking about that layer

Whenever I think of layers in software engineering, I always picture that great engineering marvel: the Hoover Dam.

A very visible, physical divider between two different domains - the reservoir before and the river after. In order for the water to continue to flow through the canyons, it must pass through the dam in a manner dictated by the dam itself - just like how data flows around a system based on an agreed contract.

(Of course, like any good metaphor, the dam one doesn't quite work because water should only flow in one direction, unlike our data, but it will serve here as a visual aid)

Our networking layer will have three main responsibilities:

1. Scheduling network requests
2. Performing networking requests
3. Parsing the response from network requests

These responsibilities will come together to produce the following class structure:

Just like with the Hoover Dam, our networking layer will have a well-encapsulated outer edge that will transform data requests into distinct tasks, which will then be scheduled for execution. Each task will consist of the actual network request and its response parsing 🌹.

Recapping

At the heart of this approach will be the ability to transform a network request and the subsequent parsing of its response into one task to be completed together. With OperationQueue and Operation, iOS already has a simple, well-supported mechanism of isolating distinct tasks within our apps. Combined OperationQueue and Operation form one of the options for supporting the concurrent execution of tasks.

Before continuing, let's take a small recap of what OperationQueue and Operation are.

OperationQueue is a queue that controls the order of execution of its operations based on their readiness value (which must be true to initiate execution) and then scores each operation according to its priority level and the time spent in the queue. OperationQueue is built on top of GCD, which allows it to take advantage of an iOS device's multiple cores without the developer needing to know how it does this. By default, OperationQueue will attempt to execute as many operations in parallel as the device is capable of supporting. As OperationQueue maintains a link to its operations, it's possible to query those operations and perform additional tasks on them, such as pausing or cancelling operations (which can be useful if the user logs out).

Operation is an abstract class which needs to be subclassed to undertake a specific task. An Operation typically runs on a separate thread from the one that created it. Each operation is controlled via an internal state machine, the possible states are:

• Pending indicates that our Operation subclass has been added to the queue.
• Ready indicates that our Operation subclass is good to go, and if there is space on the queue, this operation's task can be started
• Executing indicates that our Operation subclass is actually doing work at the moment.
• Finished indicates that our Operation subclass has completed its task and should be removed from the queue.
• Cancelled indicates that our Operation subclass has been cancelled and should stop its execution.

A typical operations lifecycle will move through the following states:

It's important to note that cancelling an executing operation will not automatically stop that operation; instead, it is up to the individual operation to clean up after itself and transition into the Finished state.

Operations come in two flavours:

• Non-Concurrent
• Concurrent

Non-Concurrent operations perform all their work on the same thread, so that when the main method returns, the operation is moved into the Finished state. The queue is then be notified of this and removes the operation from its active operation pool, freeing resources for the next operation.

Concurrent operations can perform some of their work on a different thread, so returning from the main method can no longer be used to move the operation into a Finished state. Instead, when we create a concurrent operation, we assume the responsibility for moving the operation between the Ready, Executing, and Finished states.

Building the concurrent operation 🏗️

A networking operation is a specialised concurrent operation because when an URLSession makes a network request, it does so on a different thread from the thread that resumed that task. Rather than cramming everything into one operation, this solution will focus on building an abstract concurrent operation and more specific operations that actually do the networking.

This post will gradually build up to a working example; however, if time is tight, you can head over to the completed example and take a look at ConcurrentOperation, QuestionsRetrievalOperation, QuestionsDataManager and QueueManager to see how things end up.

As mentioned, a concurrent operation takes responsibility for ensuring that its internal state is correct. This state is controlled by manipulating the isReady, isExecuting and isFinished properties. However, these are read-only, so these properties will need to be overridden and the operations' current state tracked via a different property. When mapping state, I find it best to use an enum type:

class ConcurrentOperation: Operation {

    // MARK: - State

    private enum State {
        case ready
        case executing
        case finished
    }

    private var state = State.ready

    override var isReady: Bool {
        return super.isReady && state == .ready
    }

    override var isExecuting: Bool {
        return state == .executing
    }

    override var isFinished: Bool {
        return state == .finished
    }
}

In the code snippet above, the private mutable property state tracks this internal state, and the state value is then used in each of the overridden properties to return the correct boolean result. The cases of the State enum directly map to the state properties being overridden.

It's important to note that the isReady property is a little more complex than isExecuting or isFinished. One of the advantages of using operations is that it's possible to chain operations together, so that one operation depends on another operation finishing before it can begin execution. If isReady just checked state == .ready, ConcurrentOperation would lose the ability to create dependencies, as an operation that has an unfinished dependency has an isReady value of false.

I've seen examples of concurrent operation implementations that use three separate, settable state properties, e.g. _isReady, _isExecuting and _isFinished. While this is a valid approach, I find having six properties that are so similarly named hard to read and leads to class bloat.

In concurrent operations, it's not uncommon to see the isAsynchronous property being overridden to always return true; however, in this example, these operations are always going to be executed via an operation queue, so there is no need to actually override this property, as the queue will ignore that value.

OperationQueue uses KVO to know when its operations change state; however, so far ConcurrentOperation will not inform any observers. The key paths that the queue will observe map directly to the enum cases. So we can transform our basic State enum into a String backed enum and use the raw-value of each to hold the expected key path string:

enum State: String {
    case ready = "isReady"
    case executing = "isExecuting"
    case finished = "isFinished"
}

With this new, powerful enum, let's add property observers:

var state = State.ready {
    willSet {
        willChangeValue(forKey: newValue.rawValue)
        willChangeValue(forKey: state.rawValue)
    }
    didSet {
        didChangeValue(forKey: oldValue.rawValue)
        didChangeValue(forKey: state.rawValue)
    }
}

ConcurrentOperation will now trigger KVO notifications correctly; however, the state property isn't actually being mutated. The first state change is from Ready -> Executing, which is handled by overriding the start method:

override func start() {
    guard !isCancelled else {
        finish()
        return
    }

    if !isExecuting {
        state = .executing
    }

    main()
}

The start method is the entry point into an operation, it is called when the operation should begin its work. In the above code snippet a check is made to see if the operation has been moved into the Cancelled state (yep, an operation can begin in a cancelled state). If the operation has been cancelled, finish is immediately called (this will be shown soon), which will move the operation into the Finished state and then return. If the operation hasn't been cancelled, it is moved into the Executing state and main is called (for the actual work to begin). Each subclass of ConcurrentOperation would implement main to perform its specific task. main is actually the entry point for non-concurrent operations, and technically, for a concurrent operation, any method could have been chosen as the work method (provided each subclass implements it). However, by choosing main, the cognitive load on any future developer is reduced, as it allows them to transfer the expectation of how non-concurrent operations work to our concurrent operation implementation.

(In older versions of iOS, concurrent operations were responsible for calling main on a different thread - this is no longer the case)

It's important to note that super.start() isn't being called intentionally, as by overriding start, this operation assumes full control of maintaining its state.

func finish() {
    if isExecuting {
        state = .finished
    }
}

finish acts as a nice symmetrical opposite of the start method, ensuring that when an operation has finished, it is moved into the Finished state. It's essential that all operations eventually call this method. If you are experiencing odd behaviour where your queue seems to have jammed and no operations are being processed, one of your operations is probably missing a finish call somewhere.

Finally, the cancel method needs to be overridden:

override func cancel() {
    super.cancel()

    finish()
}

It's important that regardless of how a concurrent operation ends, it moves into the Finished state, and it does so when cancelled.

And that's it. By taking control of its state, the above operation has transformed itself from a boring non-concurrent operation into....a...well, slightly less boring concurrent operation.

Adding networking-specific add-ons 📡

Before we start using ConncurentOperation let's make one more change to make it more useful by adding a custom completion closure that will use the widespread Result enum type to return the outcome of the operation's task:

enum Result {
    case success(T)
    case failure(Error)
}

class ConcurrentOperation: Operation {

    typealias OperationCompletionHandler = (_ result: Result) -> Void

    var completionHandler: (OperationCompletionHandler)?

    // omitting properties and methods that we have already seen

    func complete(result: Result) {
        finish()

        if !isCancelled {
            completionHandler?(result)
        }
    }
}

ConcurrentOperation now supports returning results of an unknown type T, which each operation subclass will define. The custom completion closure is then triggered when the complete method is called.

And that's it, we have an abstract concurrent networking operation 🎓.

Now, let's build some concrete operations.

Making that call

StackOverflow has an excellent, open API that will be used below to build a networking operation. The networking operation will retrieve and parse the latest questions which are tagged with iOS via the /questions endpoint.

Instead of building the operation piece by piece like we did above, let's look at the complete operation and then examine 👀 more closely the interesting parts:

class QuestionsRetrievalOperation: ConcurrentOperation {

    private let session: URLSession
    private let urlRequestFactory: QuestionsURLRequestFactory
    private var task: URLSessionTask?
    private let pageIndex: Int

    // MARK: - Init

    init(pageIndex: Int, session: URLSession = URLSession.shared, urlRequestFactory: QuestionsURLRequestFactory = QuestionsURLRequestFactory()) {
        self.pageIndex = pageIndex
        self.session = session
        self.urlRequestFactory = urlRequestFactory
    }

    // MARK: - Main

    override func main() {
        let urlRequest = urlRequestFactory.requestToRetrieveQuestions(pageIndex: pageIndex)

        task = session.dataTask(with: urlRequest) { (data, response, error) in
            guard let data = data else {
                DispatchQueue.main.async {
                    if let error = error {
                        self.complete(result: .failure(error))
                    } else {
                        self.complete(result: .failure(APIError.missingData))
                    }
                }
                return
            }

            do {
                let page = try JSONDecoder().decode(QuestionPage.self, from: data)

                DispatchQueue.main.async {
                    self.complete(result: .success(page))
                }
            } catch let error {
                DispatchQueue.main.async {
                    self.complete(result: .failure(APIError.serialization))
                }
            }
        }

        task?.resume()
    }

    // MARK: - Cancel

    override func cancel() {
        task?.cancel()
        super.cancel()
    }
}

(I won't show the QuestionPage, QuestionsURLRequestFactory and APIError in this post, but if you are interested, you can see them by cloning the example project here. QuestionPage is a model struct that will be populated with the questions endpoint JSON response. QuestionsURLRequestFactory is a helper factory for returning a URLRequest object configured for accessing the /questions endpoint. APIError is a custom Error enum)

In the main method above, an instance of URLSession is used to create a URLSessionDataTask, which will make a network request and parse its response. Depending on the outcome of the network request and parsing of its response, complete is called, and either an error or a parsed QuestionPage model is passed back wrapped into a Result. When completing an operation, special attention needs to be paid to ensure that the completionHandler property is used directly, as this would cause the operation to end but not move it into the Finished state, and what would result eventually is a jammed queue (as spoken about above).

In the cancel method, the URLSessionTask instance used to mark the network request is cancelled, ensuring that no bandwidth is needlessly wasted.

And that's essentially what a networking operation looks like. Of course, each different network operation will be unique, but the general structure will be similar to QuestionsRetrievalOperation (if you want to see another concurrent operation example, take a look at this one).

Getting the managers involved

So far, 2 of the 3 responsibilities of the networking layer shown above have been built:

• Performing networking requests
• Parsing the response from network requests

Time to look at the final responsibility:

• Scheduling network requests.

If we refer back to the class structure above, this responsibility is handled by both the DataManager and Scheduler classes.

class QuestionsDataManager {

    private let queueManager: QueueManager

    // MARK: - Init

    init(withQueueManager queueManager: QueueManager = QueueManager.shared) {
        self.queueManager = queueManager
    }

    // MARK: - Retrieval

    func retrievalQuestions(pageIndex: Int, completionHandler: @escaping (_ result: Result) -> Void) {
        let operation = QuestionsRetrievalOperation(pageIndex: pageIndex)
        operation.completionHandler = completionHandler
        queueManager.enqueue(operation)
    }
}

The above manager is responsible for creating, configuring and scheduling operations on a queue. While this example only shows one method, the idea here is that this manager would handle all networking requests for the /questions endpoint.

The sharper reader will have spotted another custom class in the above code snippet: QueueManager. This class is responsible for the queue (or queues) to which operations will be added - it acts as the Scheduler class.

class QueueManager {

    lazy var queue: OperationQueue = {
        let queue = OperationQueue()

        return queue;
    }()

    // MARK: - Singleton

    static let shared = QueueManager()

    // MARK: - Addition

    func enqueue(_ operation: Operation) {
        queue.addOperation(operation)
    }
}

The above manager creates a shared OperationQueue instance and controls access to it via the enqueue. To ensure that operations are consistently added to the same queue, this manager is a singleton. While this class only has the one queue, it can be easily extended to hold multiple queues (perhaps one queue for user-driven events and another for system events - allowing greater control on suspending/cancelling queues) with each DataManager specifying which queue to add the operation to by passing an additional parameter when enqueuing that operation.

Looking at what we have

While using operations adds to the complexity of making a network request, when we step back and look at the benefit that having an isolated, well-encapsulated networking layer brings to our apps, I hope that we can all agree it is a cost worth paying 💰.

To see this networking layer in action, head over to the repo and clone the project.

The example in this post is built using GCD to control access to shared mutable state. If you would prefer to use Swift Concurrency, I would recommend that you instead read: "Keep Downloading with a Background Session".

In the past, iOS focused solely on what the user was doing, ensuring a responsive user experience on performance-constrained devices. While beneficial for users, this focus meant that as soon as an app went into the background, iOS suspended it. However, with more powerful and energy-efficient devices, iOS has changed. iOS now allows apps to perform limited background work while maintaining foreground responsiveness.

It is now possible for download and upload requests to continue when an app has been suspended or even terminated. These network requests are collectively known as background-transfers. Support for background-transfers was introduced in iOS 7. While not a recent change, it remains a powerful tool.

In this post, we'll explore background-transfers and build a working solution to use this functionality.

This post will gradually build up to a working example, but I get it, this is exciting stuff 😎, so if you are unable to contain that excitement, then head on over to the completed example and take a look at BackgroundDownloadService and BackgroundDownloadStore to see how things end up.

Different types of sessions

When Apple introduced the URLSession suite of classes, they addressed a number of the shortfalls that were present in a NSURLConnection based networking stack. One particular pain point was that with an NSURLConnection networking stack, there was no built-in way to group related types of requests together - each request in a group had to be individually configured. URLSession changed this by moving that group configuration to a session. A session represents a configurable object that handles the configuration and coordination of network requests. An app can have multiple sessions. Any request scheduled on that session would inherit that session's configuration. While useful for app developers to group requests, this session-level configuration also allowed iOS to offer more functionality as standard, enabling support for background-transfers on special background sessions.

Each session can be described as one of the following three flavours:

1. Default - supports all URLSessionTask subtypes for making a network request. These network requests can only be performed with the app in the foreground.
2. Ephemeral - similar to default but does not persist caches, cookies, or credentials to disk.
3. Background - supports all URLSessionTask subtypes for making a network request. URLSessionDataTask can only be performed with the app in the foreground, whereas URLSessionDownloadTask and URLSessionUploadTask can be performed with the app in the foreground, suspended or terminated.

Each session has its use case; let's explore how we can use a background session to enable our uploads and downloads to continue when the user leaves the app.

So, how do background-transfers work? 🕵️

When scheduling a transfer on a background session, that transfer is passed to the nsurlsessiond daemon to be processed. As nsurlsessiond lives outside of the lifecycle of any given app, any transfer scheduled on nsurlsessiond will continue even if the app that the transfer belongs to is suspended/terminated. Once a background-transfer is complete, if the app has been suspended/terminated, iOS will wake the app in the background and allow the app to perform any post-transfer processing (within a limited time frame); if the app is in the foreground, control will be passed back to the app as if the transfer has been scheduled on a default or ephemeral session (without the limited time frame for post-transfer processing).

You might be thinking:

"That sounds pretty amazing! Why isn't this the default behaviour for transfers?"

Well, there are a few reasons why this isn't the default behaviour:

• Resource Management - excessive background processing can drain battery life and consume unexpected bandwidth; Apple wants to ensure that, as app developers, we use it responsibly and only where it's actually adding value.
• Programming Complexity - implementing background-transfers requires foregoing the convenient closure-based methods of URLSession. Instead, it requires conforming to URLSessionDownloadDelegate to receive updates about the transfers, adding complexity when compared to foreground-only sessions.

Setting up the project

The app must be granted additional permissions to enable background-transfers:

1. Open the Signing & Capabilities tab in the target.
2. Add the Background Modes capability.
3. In the newly added Background Modes section, select the Background fetch and Background processing checkboxes.

After completing these steps, your settings should look like this:

With the project correctly configured, it is time to start coding.

Let's get downloading

Our background-transfer layer has five primary responsibilities:

1. Configuring a background session.
2. Scheduling the download request.
3. Responding to download updates.
4. Processing any completed downloads by moving files to a permanent location on the file system or reporting an error.
5. Recovering from a terminated/suspended state to process completed downloads.

And needs to be able to handle a download completing with the app in the following states:

1. Foregrounded - when the app is open, the app needs to behave as if the session used the default configuration.
2. Suspended - when the app has been suspended, the app needs to be woken up to finish the transfer.
3. Terminated - when iOS has terminated the app, the app needs to be relaunched to finish the transfer.

Together, these responsibilities and states produce the following class structure and interactions:

Each type has the following role:

• BackgroundDownloadService sets up the background session, schedules downloads, and responds to download updates.
• BackgroundDownloadStore manages the metadata for each download, ensuring data persistence beyond app launches.
• AppDelegate informs BackgroundDownloadService when the app has been woken/relaunched to complete background downloads.
• URLSession informs BackgroundDownloadService of any updates to in-progress downloads via the URLSessionDownloadDelegate.

From a high level, the steps involved in making a download are:

Some steps are marked as optional because they are either invoked or skipped depending on if the app is in the foreground or background.

1. A download is requested on BackgroundDownloadService.
2. The requested download's metadata is stored in BackgroundDownloadStore.
3. The download is kicked off.
4. The download completes.
   a. Optional: If the app is in the suspended/terminated state, it is woken up and AppDelegate passes a closure callback to BackgroundDownloadService to be called once all downloads are completed.
   b.BackgroundDownloadService is informed of the outcome of the download via URLSessionDownloadDelegate.
5. Stored metadata is retrieved from BackgroundDownloadStore for the download.
6. The downloaded content is moved to its permanent local file system location, and interested parties are informed of the outcome of the download.
   a. Optional: Trigger metadata closure callback, if it exists.
   b. Optional: Trigger AppDelegate closure callback once all downloads have been processed.

Don't worry if some parts of the above description about roles or the steps don't entirely make sense; each part will be explored in more depth as the solution is built up.

Three pieces of metadata are needed to perform a download:

1. Remote URL - the URL of the content to be downloaded.
2. File path URL - the URL of where the downloaded content should be stored on the local file system.
3. Completion handler - the closure to be called when the download is complete.

Let's take these three pieces of metadata and start implementing BackgroundDownloadStore, starting with adding the functionality to store that metadata:

typealias BackgroundDownloadCompletion = (_ result: Result<URL, Error>) -> ()

class BackgroundDownloadStore {
    //1
    private var inMemoryStore = [String: BackgroundDownloadCompletion]()

    // 2
    private let persistentStore = UserDefaults.standard

    // MARK: - Store

    // 3
    func storeMetadata(from fromURL: URL,
                       to toURL: URL,
                       completionHandler: @escaping BackgroundDownloadCompletion) {
        inMemoryStore[fromURL.absoluteString] = completionHandler
        persistentStore.set(toURL, forKey: fromURL.absoluteString)
    }
}

Here's what we did:

1. Background session downloads don't support the completion-handler methods on URLSession; instead, any download progress updates are provided via URLSessionDownloadDelegate. However, a closure-based interface is perfect for downloading content where the caller is really only interested in either having the downloaded content or an error. So, to recreate that convenient closure-based interface, BackgroundDownloadService takes a closure when a download is requested and passes it onto BackgroundDownloadStore to store it in the inMemoryStore dictionary. As the outcome of a download can either succeed or fail, the Result type can represent both outcomes in one type. As the closure here is a communication path between the caller and callee, when the app is terminated, the memory holding the caller and callee will be freed, so the storage of these closures is only in-memory.
2. As a download may complete while the app is terminated, details of where the downloaded content should end up on the local file system must be persisted outside of memory. UserDefaults is a simple and effective means of persisting that information.
3. The method to store a download's metadata into inMemoryStore and persistentStore using fromURL as the key.

Storing metadata is all good, but that metadata needs to be retrieved:

class BackgroundDownloadStore {
    // Omitted methods and properties

    // MARK: - Retrieve

    // 1
    func retrieveMetadata(for forURL: URL) -> (URL?, BackgroundDownloadCompletion?) {
        let key = forURL.absoluteString

        let toURL = persistentStore.url(forKey: key)
        let metaDataCompletionHandler = inMemoryStore[key]

        return (toURL, metaDataCompletionHandler)
    }
}

Here's what we did:

1. Any metadata associated with forURL is extracted from persistentStore and inMemoryStore. As there is no guarantee that the metadata associated with forURL was ever stored (or in the case of the closures stored in inMemoryStore, there is no guarantee the closure is still in memory), this method returns that metadata as a tuple with two optional values.

Finally, once a download has been completed, we need a way to remove that download's metadata from the store:

class BackgroundDownloadStore {
    // Omitted methods and properties

    // MARK: - Remove

    // 1
    func removeMetadata(for forURL: URL) {
        let key = forURL.absoluteString

        inMemoryStore[key] = nil
        persistentStore.removeObject(forKey: key)
    }
}

Here's what we did:

1. Any metadata associated with forURL is deleted from persistentStore and inMemoryStore.

That's all we need to do to store the metadata associated with a download; let's move on to configuring a background session and making a background-transfer.

Before looking at BackgroundDownloadService, let's take a short detour and look at a supporting type that will be used in BackgroundDownloadService:

enum BackgroundDownloadServiceError: Error {
    case missingInstructionsError
    case fileSystemError(_ underlyingError: Error)
    case networkError(_ underlyingError: Error?)
    case unexpectedResponseError
    case unexpectedStatusCode
}

BackgroundDownloadServiceError is an enum that conforms to the Error protocol and will be used to provide details about the unhappy download paths through BackgroundDownloadService.

Detour over 🗺️.

Let's build the skeleton of BackgroundDownloadService:

class BackgroundDownloadService: NSObject { // 1
    // 2
    private var session: URLSession!

    // MARK: - Singleton

    // 3
    static let shared = BackgroundDownloadService()

    // 4
    override init() {
        super.init()

        configureSession()
    }

    private func configureSession() {
        // 5
        let configuration = URLSessionConfiguration.background(withIdentifier: "com.williamboles.background.download.session")

        // 6
        configuration.sessionSendsLaunchEvents = true
        let session = URLSession(configuration: configuration,
                                 delegate: self,
                                 delegateQueue: nil)
        self.session = session
    }
}

Let's look at what we did above:

1. As mentioned earlier, background session downloads don't support the completion-handler methods on URLSession; Instead BackgroundDownloadService will need to become the delegate to the background URLSession session by implementing the URLSessionDownloadDelegate. The need to conform to URLSessionDownloadDelegate is why BackgroundDownloadService is a subclass of NSObject.
2. There exists a cyclic dependency between BackgroundDownloadService and URLSession, where BackgroundDownloadService is responsible for not only creating the URLSession instance but also being its delegate. However, the delegate of URLSession can only be set at initialisation of the URLSession. Both these conditions cannot be satisfied. To break the cyclic dependency between BackgroundDownloadService and URLSession, the session property is set after BackgroundDownloadService is initialised, and to avoid having an optional property, session is implicitly unwrapped.
3. BackgroundDownloadService is a singleton to ensure that any background-transfer updates are funnelled into the same type that holds the metadata of that transfer.
4. init() is private to ensure that another instance of BackgroundDownloadService cannot be created.
5. A session's configuration is defined through a URLSessionConfiguration instance, which offers numerous configuration options. Every URLSessionConfiguration for a background session requires a unique identifier within that app's lifecycle. If an existing URLSession already has that identifier, iOS returns the previously created URLSession instance. The session identifier plays a vital role in ensuring that a download can survive app termination - for any download update, iOS will attempt to find a URLSession instance with the same identifier that the download was kicked off on to inform of the update; that found URLSession instance does not need to be the same instance that was used to kick off the download. To ensure the app always has a URLSession that can pick up any download updates, session is consistently configured with the same identifier.
6. sessionSendsLaunchEvents tells iOS that any background-transfers scheduled on this session can launch the app if it is suspended/terminated.

So far, we have configured a session for performing background-transfers, but can't yet download anything; let's change that in:

class BackgroundDownloadService: NSObject {
    // Omitted other properties

    // 1
    private let store = BackgroundDownloadStore()

    // Omitted other methods

    // MARK: - Download

    func download(from remoteURL: URL,
                  saveDownloadTo localURL: URL,
                  completionHandler: @escaping ((_ result: Result<URL, BackgroundDownloadServiceError>) -> ())) {
        os_log(.info, "Scheduling to download: %{public}@", remoteURL.absoluteString)

        // 2
        store.storeMetadata(from: remoteURL,
                            to: localURL,
                            completionHandler: completionHandler)

        // 3
        let task = session.downloadTask(with: remoteURL)
        task.resume()
    }
}

Here's what we did:

1. A BackgroundDownloadStore instance is stored as a property to ensure access to a download's metadata.
2. The metadata associated with the current download request is added to the store.
3. A URLSessionDownloadTask instance is created using the background session, and the download request is kicked off.

To better see the downloads happening in the background, you may wish to add a slight delay before the download starts, making it easier to put the app into the background:

task.earliestBeginDate = Date().addingTimeInterval(5)

Now that BackgroundDownloadService can kick off a download, we need to add in the code to handle the happy-path when a download completes by implementing urlSession(_:downloadTask:didFinishDownloadingTo) from URLSessionDownloadDelegate:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    func urlSession(_ session: URLSession,
                    downloadTask: URLSessionDownloadTask,
                    didFinishDownloadingTo location: URL) {
        // 1
        guard let fromURL = downloadTask.originalRequest?.url else {
            os_log(.error, "Unexpected nil URL")
            // Unable to call the closure here as fromURL is the key to retrieving this download's closure
            return
        }

        // 2
        defer {
            store.removeMetadata(for: fromURL)
        }

        let fromURLAsString = fromURL.absoluteString

        os_log(.info, "Download request completed for: %{public}@", fromURLAsString)

        // 3
        let (toURL, completionHandler) = store.retrieveMetadata(for: fromURL)

        // 4
        guard let toURL else {
            os_log(.error, "Unable to find existing download item for: %{public}@", fromURLAsString)
            completionHandler?(.failure(BackgroundDownloadError.missingInstructionsError))
            return
        }

        os_log(.info, "Download successful for: %{public}@", fromURLAsString)

        // 5
        do {
            try FileManager.default.moveItem(at: location,
                                             to: toURL)
            // 6
            completionHandler?(.success(toURL))
        } catch {
            completionHandler?(.failure(BackgroundDownloadError.fileSystemError(error)))
        }
    }
}

This delegate method can be called when the app is foregrounded, suspended, or terminated, so all three states need to be handled in the same method:

1. In the download method, the remoteURL was used as the key for storing a download's metadata because that value is present when a download completes in the URLSessionDownloadTask instance. A URLSessionDownloadTask will automatically follow redirect instructions, so it is essential to ensure that the correct URL is used to connect the completed download with its metadata - specifically, the original URL stored in the originalRequest property. If originalRequest is nil, the method exits as no further action can be taken.
2. As there are multiple paths through urlSession(_:downloadTask:didFinishDownloadingTo), a defer block is used to clean up the download by deleting its metadata on any exit.
3. The downloads' associated metadata is retrieved from the store.
4. If toURL is nil, the download cannot be processed because there is no known location to move the downloaded content to. As a result, the method exits with no further action taken.
5. When URLSessionDownloadTask completes a download, it will store that downloaded content in a temporary location and allow us to move that downloaded file to a permanent location before iOS deletes it. Using FileManager, the downloaded content is moved from its temporary location to its permanent location. If the move is successful, the completion handler is called with the file's new location; if the move fails, the completion handler is called with an error.
6. If the app has been relaunched from a terminated state, any closure associated with this download is no longer available, so there is no enforcement that completionHandler must be non-nil.

That's the happy path completed; let's add in the unhappy-path.

The unhappy-path comes in two forms: server-side and client-side errors. Let's start with the server-side errors by updating urlSession(_:downloadTask:didFinishDownloadingTo:):

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    func urlSession(_ session: URLSession,
                    downloadTask: URLSessionDownloadTask,
                    didFinishDownloadingTo location: URL) {
        guard let fromURL = downloadTask.originalRequest?.url else {
            os_log(.error, "Unexpected nil URL")
            // Unable to call the closure here as fromURL is the key to retrieving this download's closure
            return
        }

        defer {
            store.removeMetadata(for: fromURL)
        }

        let fromURLAsString = fromURL.absoluteString

        os_log(.info, "Download request completed for: %{public}@", fromURLAsString)

        let (toURL, completionHandler) = store.retrieveMetadata(for: fromURL)

        guard let toURL else {
            os_log(.error, "Unable to find existing download item for: %{public}@", fromURLAsString)
            completionHandler?(.failure(BackgroundDownloadError.missingInstructionsError))
            return
        }

        // 1
        guard let response = downloadTask.response as? HTTPURLResponse,
                     response.statusCode == 200 else {
             os_log(.error, "Unexpected response for: %{public}@", fromURLAsString)
             completionHandler?(.failure(BackgroundDownloadError.serverError(downloadTask.response)))
             return
         }

        os_log(.info, "Download successful for: %{public}@", fromURLAsString)

        do {
            try FileManager.default.moveItem(at: location,
                                             to: toURL)

            completionHandler?(.success(toURL))
        } catch {
            completionHandler?(.failure(BackgroundDownloadError.fileSystemError(error)))
        }
    }
}

Here's what we changed:

1. To determine if a download was successful, a 200 HTTP status code is expected. Here, the response is checked. If the status code is anything other than a 200, then completionHandler is triggered with serverError.

If your server returns other status codes that should be treated as success, then it is simple enough to add those by refactoring the guard to accommodate those additional status codes.

Now that server-side errors are handled, let's take care of any client-side errors:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    // Omitted other methods

    func urlSession(_ session: URLSession,
                    task: URLSessionTask,
                    didCompleteWithError error: Error?) {
        // 1
        guard let error = error else {
            return
        }

        // 2
        guard let fromURL = task.originalRequest?.url else {
            os_log(.error, "Unexpected nil URL")
            return
        }

        let fromURLAaString = fromURL.absoluteString

        os_log(.info, "Download failed for: %{public}@", fromURLAaString)

        // 3
        let (_, completionHandler) = store.retrieveMetadata(for: fromURL)

        completionHandler?(.failure(BackgroundDownloadError.clientError(error)))

        // 4
        store.removeMetadata(for: fromURL)
    }
}

Here's what we did:

1. This delegate method is called when any network request completes, meaning it will be triggered for both successful and unsuccessful network requests. If a network request fails, error will contain a value. To exclude successful downloads from the unhappy-path, error is checked for nil. If error is nil, the method exits.
2. Like in urlSession(_:downloadTask:didFinishDownloadingTo:), the remote URL is extracted from the completed URLSessionTask instance as fromURL. If originalRequest is nil, the method exits as no further action can be taken.
3. The closure associated with this download is retrieved from store and triggered with networkError.
4. The now-complete downloads metadata is deleted.

If you've been putting together the code snippets, you should be able to download files in the foreground; however, as soon as the app is put into the background, a stream of errors will begin to appear in the Xcode console, similar to:

So far, BackgroundDownloadService only works when the app is in the foreground. Let's extend it to support downloads when the app enters a suspended/terminated state.

Keeping downloads going

When an app is in a suspended/terminated state and a download completes, iOS will wake the app up (in the background) by calling application(_:handleEventsForBackgroundURLSession:completionHandler:) on the AppDelegate. Regardless of how many downloads are in progress, application(_:handleEventsForBackgroundURLSession:completionHandler:) is only called once - when all downloads have been completed. If the user foregrounds the app before the above method is called, urlSession(_:downloadTask:didFinishDownloadingTo:) is called for each completed download.

class AppDelegate: UIResponder, UIApplicationDelegate {
    // Omitted properties and methods

    // 1
    func application(_ application: UIApplication,
                     handleEventsForBackgroundURLSession identifier: String,
                     completionHandler: @escaping () -> Void) {
        // 2
        BackgroundDownloadService.shared.backgroundCompletionHandler = completionHandler
    }
}

Here's what we did:

1. iOS provides the app with the identifier of the session that the download was scheduled on and a completion handler to be called once all post-download work is complete. As the background session always uses the same identifier, identifier can be ignored.
2. completionHandler is passed to BackgroundDownloadService so that it can be called once all post-download work is complete.

As the completionHandler closure is passed to the shared BackgroundDownloadService instance, a new property needs to be added to that class:

class BackgroundDownloadService: NSObject {
    var backgroundCompletionHandler: (() -> Void)?

    // Omitted properties and methods
}

When all the downloads are complete, urlSession(_:downloadTask:didFinishDownloadingTo:) will be called for each download. After all the calls to urlSession(_:downloadTask:didFinishDownloadingTo:) have been made, a special background-transfer only delegate call is made:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    // Omitted methods

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        // 1
        DispatchQueue.main.async {
            self.backgroundCompletionHandler?()
            self.backgroundCompletionHandler = nil
        }
    }
}

Here's what we did:

1. Triggering backgroundCompletionHandler instructs iOS to take a new snapshot of the app's UI for the app switcher preview. As such, backgroundCompletionHandler must be called from the main queue. Failure to call backgroundCompletionHandler will result in iOS treating the app as a bad citizen, reducing future background processing opportunities.

While it's possible to make additional network requests with the app running in the background, they are subject to a rate limiter to prevent abuse of background sessions - this rate limiter delays the start of those network requests. With each separate network request, the delay increases - this delay is reset when the user brings the app to the foreground or if your app does not make any additional network requests during the delay.

And that's all the changes required to continue network requests when the app is suspended/terminated.

Living in a multi-threaded world

Using the code we've written so far on a single thread works fine, but using it simultaneously from multiple threads will eventually cause the app to crash. When accessing BackgroundDownloadStore from multiple threads, one thread may be modifying the internal mutable state of BackgroundDownloadStore while another thread is also modifying it. As BackgroundDownloadStore doesn't lock its state during a modification operation, simultaneous modification operations will eventually result in data corruption and the app crashing.

Thoughtful use of a DispatchQueue can prevent these issues. Our DispatchQueue instance will allow concurrent read operations but serial write operations. If a write operation is being processed, any new operation (write or read) added to the queue will be blocked until the in-progress write operation completes. Using a DispatchQueue instance will ensure that only one write operation runs at a time, preventing data corruption and stopping the app crashes we are currently experiencing.

Let's add state access controls to BackgroundDownloadStore to protect its shared mutable state:

class BackgroundDownloadStore {
    // 1
    private let queue = DispatchQueue(label: "com.williamboles.background.download.service",
                                      qos: .userInitiated,
                                      attributes: .concurrent)

    // Omitted properties and methods
}

Here's what we did:

1. As the downloads are triggered via user action, the queue priority is set to the higher end of the scale by setting its qos to userInitiated; this will ensure that operations on this queue are given greater resources when compared to lower priority operations on other queues. By default, an instance of DispatchQueue is a serial queue; this default is overridden by creating a concurrent queue using the concurrent attribute.

With the above queue property, let's update the rest of BackgroundDownloadStore, starting with the storeMetadata(from:to:completionHandler:) method:

class BackgroundDownloadStore {
    // Omitted properties and methods

    func storeMetadata(from fromURL: URL,
                       to toURL: URL,
                       completionHandler: @escaping BackgroundDownloadCompletion) {
        // 1
        queue.async(flags: .barrier) { [weak self] in
            self?.inMemoryStore[fromURL.absoluteString] = completionHandler
            self?.persistentStore.set(toURL, forKey: fromURL.absoluteString)
        }
    }
}

Here's what we did:

1. Using our DispatchQueue instance, we asynchronously schedule the metadata storage to happen on that queue. Adding the operation asynchronously ensures that the calling thread won't be blocked. As this is a write operation, the barrier flag is set when adding the operation to queue; this flag will switch the dispatch queue from concurrent processing to serial processing for this operation.

The changes needed to protect removeMetadata(for:) are almost the same as those made for storeMetadata(from:to:completionHandler:):

class BackgroundDownloadStore {
    // Omitted properties and methods

    func removeMetadata(for forURL: URL) {
        queue.async(flags: .barrier) { [weak self] in
            let key = forURL.absoluteString

            self?.inMemoryStore[key] = nil
            self?.persistentStore.removeObject(forKey: key)
        }
    }
}

retrieveMetadata(for:) requires slightly more change as it needs to return a download's metadata without blocking the calling thread. The best way to achieve this is to return its result via a closure:

class BackgroundDownloadStore {
    // Omitted properties and methods

    func retrieveMetadata(for forURL: URL,
                          completionHandler: @escaping ((URL?, BackgroundDownloadCompletion?) -> ())) {
        // 1
        return queue.async { [weak self] in
            let key = forURL.absoluteString

            let toURL = self?.persistentStore.url(forKey: key)
            let metaDataCompletionHandler = self?.inMemoryStore[key]

            completionHandler(toURL, metaDataCompletionHandler)
        }
    }
}

Here's what we did:

1. The retrieval of the metadata associated with forURL is asynchronously scheduled on queue. Adding the retrieval operation asynchronously ensures that the calling thread won't be blocked. As this is a read operation, the operation can run concurrently with any other read operations, so unlike in storeMetadata(from:to:completionHandler:) and removeMetadata(for:), the operation is not flagged as a barrier operation.

We could have added the read operation as a synchronous operation to the dispatch queue using queue.sync, as we know that BackgroundDownloadService will call this method from a background thread, and so avoid having to add a closure to return the result of the retrieval. However, I felt that adding a blocking thread call here would lead to a nasty surprise if retrieveMetadata(for:completionHandler:) was ever used on the main thread.

With the changes to the interface of BackgroundDownloadStore to ensure thread safety, BackgroundDownloadService needs to be updated:

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    func urlSession(_ session: URLSession,
                    downloadTask: URLSessionDownloadTask,
                    didFinishDownloadingTo location: URL) {
        guard let fromURL = downloadTask.originalRequest?.url else {
            os_log(.error, "Unexpected nil URL")
            // Unable to call the closure here as fromURL is the key to retrieving this download's closure
            return
        }

        let fromURLAsString = fromURL.absoluteString

        os_log(.info, "Download request completed for: %{public}@", fromURLAsString)

        // 1
        let tempLocation = FileManager.default.temporaryDirectory.appendingPathComponent(location.lastPathComponent)
        try? FileManager.default.moveItem(at: location,
                                          to: tempLocation)

        // 2
        store.retrieveMetadata(for: fromURL) { [weak self] toURL, completionHandler in
            // 3
            defer {
                self?.store.removeMetadata(for: fromURL)
            }

            guard let toURL else {
                os_log(.error, "Unable to find existing download item for: %{public}@", fromURLAsString)
                completionHandler?(.failure(BackgroundDownloadError.missingInstructionsError))
                return
            }

            guard let response = downloadTask.response as? HTTPURLResponse,
                        response.statusCode == 200 else {
                os_log(.error, "Unexpected response for: %{public}@", fromURLAsString)
                completionHandler?(.failure(BackgroundDownloadError.serverError(downloadTask.response)))
                return
            }

            os_log(.info, "Download successful for: %{public}@", fromURLAsString)

            do {
                // 4
                try FileManager.default.moveItem(at: tempLocation,
                                                 to: toURL)

                completionHandler?(.success(toURL))
            } catch {
                completionHandler?(.failure(BackgroundDownloadError.fileSystemError(error)))
            }
        }
    }

   // Omitted other methods
}

Here's what we changed:

1. As mentioned before, iOS only guarantees until the end of this method that the downloaded content will be found at location. As retrieveMetadata(for:completionHandler:) is an asynchronous method, waiting on the completion handler being called with the download's metadata will mean that urlSession(_:downloadTask:didFinishDownloadingTo) will have exited and the downloaded content will have been deleted. So before attempting to fetch the download's metadata, the downloaded content is moved from its current temporary location to a different temporary location that BackgroundDownloadService controls. Moving to a temporary location ensures that the downloaded content will still be accessible when the completion handler calls back with the metadata.
2. Using the closure-based interface for retrieving the completed download's metadata.
3. Moved the defer block into the closure so that it is associated with the closure block rather than the method itself.
4. Using the tempLocation URL as the source URL for the move operation.

And finally, let's use the same pattern with urlSession(_:task:didCompleteWithError)

extension BackgroundDownloadService: URLSessionDownloadDelegate {
    // Omitted other methods

    func urlSession(_ session: URLSession,
                       task: URLSessionTask,
                       didCompleteWithError error: Error?) {
         guard let error = error else {
             return
         }

         guard let fromURL = task.originalRequest?.url else {
             os_log(.error, "Unexpected nil URL")
             return
         }

         let fromURLAaString = fromURL.absoluteString

         os_log(.info, "Download failed for: %{public}@", fromURLAaString)

         // 1
         store.retrieveMetadata(for: fromURL) { [weak self] _, completionHandler in
             completionHandler?(.failure(BackgroundDownloadError.clientError(error)))

             self?.store.removeMetadata(for: fromURL)
         }
     }
}

Here's what we did:

1. Using the closure-based interface for retrieving the completed download's metadata.

If you run the project now in a multi-threaded app, downloads should complete without app crashes due to data corruption.

And that's all the code needed to support background-transfers 💃.

However, there is one gotcha 🧌 when testing the above code. If an app is manually force-quit, iOS interprets that action as the user explicitly saying that they want to stop all activity associated with that app. As a result, all scheduled background-transfers are cancelled, preventing testing of the restoration functionality. To overcome this gotcha, app termination needs to happen programmatically using exit(0), which shuts down the app without user intervention:

class AppDelegate: UIResponder, UIApplicationDelegate {
    // Omitted properties and methods

    func applicationDidEnterBackground(_ application: UIApplication) {
        //Exit the app to test restoring the app from a terminated state. Comment out to test restoring the app from a suspended state.
        DispatchQueue.main.asyncAfter(deadline: .now()) {
            os_log(.info, "App is about to quit")

            exit(0)
        }
    }
}

Remember to remove this in production 😜.

This time we are really at the end 🎉 - congratulations on making it here.

Downloads keep going 🤯

Background-transfers provide a powerful tool to meet your users' ever-increasing expectations about what your app can do for them, even when your app isn't in the foreground. I believe that the enhanced user experience we gain from supporting background-transfers outweighs any additional complexity that supporting background-transfers adds. As we have seen with careful planning, that complexity can be isolated from the rest of the app, so we end up providing functionality without getting in the way of features.

To see the complete working example, visit the repository and clone the project.

Running the example project

I've used TheCatAPI in the example project to populate the app with images. TheCatAPI has a great JSON-based API and an extensive library of lovely, freely available cat photos. While free to use, TheCatAPI requires you to register to get an x-api-key token to access those cats. You must add this token as the APIKey property in NetworkService for any network requests to work.

It's 17:00 on a warm Friday evening. As you type the final characters into your PR description, your mind drifts to the coming weekend. During that weekend, you decide to work on that open-source you cloned a while back, so you head to GitHub to look through the open issues. Like many developers, you use the same GitHub account for both your personal and work projects, so when you open GitHub, you can't avoid seeing that a colleague has spent some of their weekend leaving comments on your PR. You're now faced with a dilemma:

A. Open your PR and read those comments.

Or

B. Try to ignore those comments.

Choosing A means that you spend a portion of your weekend completing work-related tasks when you should be relaxing and recharging. Do this too often, and those out-of-work tasks you are completing will lead to your in-work performance dropping and may even lead to you burning out.

Choosing B means that throughout the rest of the weekend, you repeatedly return to that PR in your thoughts, wondering what issues someone found in your solution. Not knowing what someone has commented on can be low-level stressful. This stress is because, as humans, we are programmed to remind ourselves about any uncompleted tasks. Your colleague leaving comments means that your previously completed task is now incomplete. In psychology, this is known as the Zeigarnik effect and will stop you from leaving work behind to focus on whatever you are doing during the weekend.

As you can see, choosing A or B leads to a negative outcome. The only way to truly enjoy your weekend is to leave work behind. In this case, what you don't know can't hurt you.

Having my mixed GitHub account kept stressing me out. I couldn't help but see work-related updates on the GitHub homepage while doing some recreational programming. I don't want to stop the recreational programming because I love it, but I couldn't let my mental health continue to suffer. The only way I could prevent this unneeded stress was to disconnect my personal and work GitHub usage by creating multiple GitHub accounts.

However, creating and using multiple GitHub accounts on the same computer proved to be slightly trickier than I thought it would be.

Adding multiple accounts

After creating a work-only GitHub account, I needed to generate an SSH key so that I could access that account from my machine. The only difference from the documented Git setup is that I couldn't use id_rsa as the file name to store my work account's SSH key (as it was storing my personal SSH key) instead I had to use a different name, i.e. id_rsa_work.

Now that I had both SSH keys on my Mac, I needed a way to inform Git which one to use with each repository.

There are numerous detailed articles already written on this subject, and I ended up with an SSH config (~/.ssh/config) of:

Host personal
  HostName github.com
  User git
  AddKeysToAgent yes
  UseKeychain yes
  IdentitiesOnly yes
  IdentityFile ~/.ssh/id_rsa

Host work
  HostName github.com
  User git
  AddKeysToAgent yes
  UseKeychain yes
  IdentitiesOnly yes
  IdentityFile ~/.ssh/id_rsa_work

When cloning a repo, I would specify which account to use, and any actions after on that repo would happen as that account, i.e.

git clone personal:wibosco/CoreDataServices.git

or

git clone git@personal:wibosco/CoreDataServices.git

However, sadly, I wasn't able to make this work on my Mac (Mojave or Catalina). No matter how I manipulated the structure of my SSH config, I ended up back at the same issue: Git always attempted to use the last key added to the SSH Agent, regardless of which account I specified when cloning the repository.

Just as I started to despair that there was no way to get this working, the thought occurred to me:

"Well, if SSH Agent is always serving the last key added I can just only tell it about the key for the account I want to use"

If the SSH Agent only knew about one SSH key, that key was the one it would serve.

So the first thing to do is make the SSH Agent forget about my personal and work SSH keys by removing them:

ssh-add -D

Once all SSH keys were removed (from the Agent, not the system) I could then add the correct key for the repo I wanted to access, so if it were a personal repository, I would add my personal key to the SSH Agent:

ssh-add -K ~/.ssh/id_rsa

Then, when I used any Git commands, they would be executed as my personal account. If I wanted to use my work account, I would remove the personal key and add my work key. While it was annoying having to keep switching between accounts, it worked!

Executing ssh-add -l shows you which keys are loaded.

Even with the inconvenience of having to manually add and remove keys from the SSH Agent, I was able to improve my work-life balance by eliminating the general sense of unease I would experience when avoiding work-related updates during my personal time. I don't have any data on how this change impacted my productivity, but I imagine it has improved it.

Recently, I've been experiencing the iOS equivalent of the movie Inception - putting a collection view inside a collection view. While exploring possible solutions, I stumbled upon this very informative post by Soroush Khanlou and his suggestion that the best way to implement a collection view inside a collection view was by using child view controllers - with each child view controller implementing its own collection view and having its view added as a subview on one of the parent view controller's cells.

If you haven't read that post, I recommend doing so, as it presents the argument for why you would want to put view controllers inside cells very well. And even if you don't need to be convinced, I would still recommend it, as the rest of this post won't make sense without it 😉.

The post itself is a few years old, written in Objective-C, so I converted it to Swift and integrated the solution into my project. I ended with the following collection view cell subclass:

class HostedViewCollectionViewCell: UICollectionViewCell {

    // MARK: - HostedView

    // 1
    weak var hostedView: UIView? {
        didSet {
          // 2
          if let oldValue = oldValue {
              oldValue.removeFromSuperview()
          }

          // 3
          if let hostedView = hostedView {
              hostedView.frame = contentView.bounds
              contentView.addSubview(hostedView)
          }
        }
    }

    // MARK: - Reuse

    // 4
    override func prepareForReuse() {
        super.prepareForReuse()

        hostedView = nil
    }
}

1. Each cell holds a reference to a (child) view controller's view via the hostedView. Whenever that hostedView property is set, the above didSet observer code is triggered.
2. If hostedView was previously set, that old hostedView is removed from the cell's view hierarchy.
3. If the current hostedView value is non-nil, it is added as a subview of the cell's contentView.
4. To improve performance, a collection view will only create enough cells to fill the visible UI and then a few more to allow for smooth scrolling. When a cell scrolls off-screen, it is marked for reuse on a different index path. prepareForReuse() is called just before the cell is reused and allows us to reset that cell. In the above prepareForReuse(), hostedView is set to nil, so triggering its removal from the cell's view hierarchy.

To begin with, this solution worked well. However, I started noticing that occasionally a cell would forget its content. It was infrequent and could be resolved by scrolling the collection view. However, I was quite dissatisfied with this experience and wanted to understand what was causing this UI breakdown.

Looking over that prepareForReuse() method, you can see that removeFromSuperview() is called to do the removing. What's interesting about removeFromSuperview() is that it takes no arguments and instead uses the soon-to-be-removed view's superview value to determine what view to remove the caller from. A view can only have one superview. If a view that already has a superview is added as a subview to a different view, the original connection to the first superview is broken and replaced with this new connection. For the most part, this 1-to-M mapping between a superview and its subviews works just fine, as most views once added as subviews do not tend to move around. However, cells are designed to be reused. The reusable nature of cells lies at the root of my cells forgetfulness. By using the solution above, we end up with the following unintended associations:

The diagram above shows how multiple cells can be associated (shown in purple) with the same view controller's view, but only one of those cells has the view of ViewController as a subview (shown in green). Because of the multiple references kept to the same view of ViewController, it's possible for any of Cell A, Cell B or Cell C to remove the hostedView from Cell C by calling removeFromSuperview() in their own prepareForReuse() method. Of course, it was not intentional for multiple cells to have an active reference to a view controller's view if that view was no longer part of the cell's view hierarchy.

Once those unintended left-over hostedView references were spotted, the solution for the bug became straightforward - only remove the hostedView if it is still in the cell's view hierarchy:

class HostedViewCollectionViewCell: UICollectionViewCell {
    var hostedView: UIView? {
        didSet {
            if let oldValue = oldValue {
                if oldValue.isDescendant(of: self) { //Make sure that hostedView hasn't been added as a subview to a different cell
                    oldValue.removeFromSuperview()
                }
            }

            //Omitted rest of observer
        }
    }

    //Omitted methods
}

Now, the cell will only remove the hostedView from its superview if that superview is part of the cell itself. This additional if statement addresses the forgetfulness that I was seeing. However, if you queried hostedView on Cell A or Cell B from the above diagram, you would still get back a reference to the view that Cell C. With the above if statement, we have only resolved part of the bug. Let's make the cell only return a hostedView value if that hostedView is actually part of its view hierarchy:

class HostedViewCollectionViewCell: UICollectionViewCell {

    // MARK: - HostedView

    // 1
    private weak var _hostedView: UIView? {
        didSet {
            if let oldValue = oldValue {
                if oldValue.isDescendant(of: self) { //Make sure that hostedView hasn't been added as a subview to a different cell
                    oldValue.removeFromSuperview()
                }
            }

            if let _hostedView = _hostedView {
                _hostedView.frame = contentView.bounds
                contentView.addSubview(_hostedView)
            }
        }
    }

    // 2
    weak var hostedView: UIView? {
        // 3
        get {
            guard _hostedView?.isDescendant(of: self) ?? false else {
                _hostedView = nil
                return nil
            }

            return _hostedView
        }
        //4
        set {
            _hostedView = newValue
        }
    }

    //Omitted methods
}

1. The private _hostedView property assumes the responsibilities of the previous hostedView property implementation and acts as the backing-store property to the new hostedView property implementation. The _hostedView now actually holds a reference to the view controller's view, even though the outside world still thinks it is hostedView that holds the reference. Just like before, the didSet observer checks if the oldValue of the _hostedView isn't nil and if it isn't, removes that view from the cell's view hierarchy if _hostedView is still a subview. If the current value of _hostedView is not nil, _hostedView is added as a subview of the current cell.
2. The externally accessible hostedView property now has a custom get and set.
3. The get only returns the _hostedView value if that view is still a subview of the cell. If hostedView isn't a subview, the get sets _hostedView to nil (which will cause it to be removed from this cell's view hierarchy) and returns nil. Dropping the reference once something tries to access hostedView feels a little strange. However, as the superview property of UIView isn't KVO compliant (and we don't want to get involved in the dark-arts of method swizzling to make it so) there is no way for us to know that the hostedView has a new superview without querying the hostedView and there is no point in querying the hostedView until something tries to access it - so a little self-contained strangeness is the only viable option here.
4. The set takes the new value and assigns it to the backing-store property.

With a backing-store property, it is essential that you practice good access hygiene - _hostedView should only be directly accessed in either hostedView or _hostedView; everywhere else, you should use hostedView.

You can see that by truthfully returning hostedView, we have added a lot of complexity to our cell. Still, none of that complexity leaks out. We can have confidence that hosting a view controller's view won't lead to unintended consequences.

Seeing the 🐛 for yourself

If you want to see this bug in the wild, you can download the example project from my repo. In that example project, I've added logging to track when the view controller's view isn't a subview on that cell anymore, so that you can easily see when that bug would have happened by watching the console. If you want to see the bug's impact on the UI, comment out the isDescendant(of:_) check in HostedViewCollectionViewCell.

Core Data and Unit Testing haven't always been the best of friends. Like two members of the same friend group who don't really know each other but really like their UIKit friend, Core Data and Unit Testing have, in fact, discovered that they have a lot in common and have gradually grown more friendly towards each other.

But before we delve into how they can take it one step further and become firm friends, we need to understand what makes each of them tick.

Getting to know each other

Core Data

Core Data is an object graph manager that helps you create the model layer of your app. An object graph is a collection of objects connected through relationships. Core Data abstracts away the lifecycle of the objects it controls, providing tools to allow CRUD operations to be performed on those objects in its graph. In one configuration, Core Data's object graph can be persisted between executions of the app, with those objects being persisted in either XML, binary, or SQLite stores. However, in another configuration, the object graph exists purely in-memory and is destroyed when the app is killed. In fact, the range of configuration options available to Core Data makes it very difficult to define what Core Data actually is. In one configuration, Core Data can resemble a SQL wrapper, while in another configuration, it appears more like an ORM. The truth is that Core Data sits somewhere between both, with the ability to intelligently load a subset of its store's data into memory as custom domain objects that can be manipulated like any other Swift object before being "saved" back to the store where those changes are then persistent (be that across app executions or just in that app execution).

A consequence of this flexibility is that it's not uncommon to see two apps using Core Data in significantly different ways. This has led to Core Data gaining a reputation as a hard-to-use framework (not helped by the significant changes the framework has undergone since its inception). To avoid any confusion around what configuration of Core Data we are going to use, in this post, our setup will look like:

• Persistent Container is a fairly new member of the Core Data family. It was introduced in iOS 10 to help simplify the creation of the managed object model, persistent store coordinator and any managed object contexts.
• Managed Object Context is a temporary, in-memory record of all NSManagedObject instances accessed, created or updated during its lifecycle. An app typically has multiple contexts in existence at any given time. These contexts will form a parent-child relationship. When a child context is saved, it pushes its changes to its parent's context, which then merges these changes into its own state. At the top of this parent-child hierarchy is the main context; this context, upon being saved, will push its changes into the persistent store.
• Persistent Store Coordinator acts as an aggregator between the various contexts to ensure the integrity of the persistent store(s). It does this by serialising read/write operations from the contexts to its store(s). There is only one coordinator per Core Data stack.
• Managed Object Model is a set of entities that define each NSManagedObject subclass. An entity can be thought of as a table in a database.
• Persistent Object Store is an abstraction over the actual storage of our data. Handles communication with that storage e.g. with SQLite storage, converts fetch requests into SQL statements.
• Storage It's common for this to be a SQLite store, but the store could also be XML, binary or in-memory.

Unit Testing

Unit Testing is ensuring that the smallest part (unit) of testable code in your app behaves as expected in isolation (from the other parts of your app). In object-oriented programming, a unit is often a particular method, with the unit test testing one scenario (path) through that method. A unit test does this by providing a strict, written contract that the unit under test must satisfy to pass. If there are multiple paths through a unit, i.e. an if and else branch, then more than one unit test would be required to cover each path.

Unit tests are then combined into a test suite within a test target/project. This suite can then be run to give an increased level of confidence that the code is behaving as expected.

Each unit test should be executed in isolation from other unit tests to ensure that the state from a previous test has no impact on the next test. It is up to the unit test to ensure that any conditions (test data, user permissions, etc.) that it depends on are present before the test is run, and it must tidy up after itself when it is finished. This helps ensure that the unit test is repeatable and not dependent on any state of the host environment. The unit test is also responsible for ensuring that the unit under test is isolated from other methods within the app and that any calls (relationships) it makes for information are mocked out. A mocked method will then return a known, preset value without performing any computation, so that if a unit test fails, we can have confidence that it has failed because of the code under test rather than having to hunt down the failure in its dependencies.

Each unit test should be as quick as possible to run to ensure that during development, the feedback loop between running the unit test and making code changes is as small as possible.

Building that friendship

From the above descriptions, we can see why Core Data and Unit Testing didn't instantly hit it off. Their differences centre on two issues:

Treatment of data

• Unit Testing treats data as ephemeral
• The main use case of Core Data is persisting data between app executions

Tolerance for delays

• Unit tests should be lightning quick to execute
• Core Data typically has to communicate with a SQLite file on disk, which is slow (when compared with pure memory operations)

And like building any good relationship, all the changes will come from one side 😜 - Core Data.

(Ok, ok, I'm sketching a little bit now, so please don't take that as genuine relationship advice)

CoreDataManager

Let's build a typical Core Data stack together and unit test it as we go.

If you want to follow along, head over to my repo and download the completed project.

Let's start with the manager that will handle setting up our Core Data stack:

class CoreDataManager {

    // MARK: - Singleton

    static let shared = CoreDataManager()
}

We could add a unit test here and assert that the same instance of CoreDataManager was always returned when shared was called however, when unit testing, we should only test code that we control and the logic behind creating a singleton is handled by Swift itself, so no need to create that test class yet.

Unit tests while asserting the correctness of an implementation also act as a living form of documentation, so if you did want to add a unit test to assert the same instance was returned, I won't put up too much of a fight.

As our project is being developed with an iOS deployment target of iOS 11, we can use the persistent container to simplify the setup of the Core Data stack.

lazy var persistentContainer: NSPersistentContainer! = {
    let persistentContainer = NSPersistentContainer(name: "TestingWithCoreData_Example")

    return persistentContainer
}()

In the above code snippet, we added a lazy property to create an instance of NSPersistentContainer. Loading a Core Data stack can be a time-consuming task, especially if migration is required. To handle this, we need to add a dedicated asynchronous setup method that can handle any time-consuming tasks.

While the below method doesn't actually show the migration itself, it's important to create and test a realistic Core Data stack and data migrations are very much a common task in iOS apps.

// MARK: - SetUp

 func setup(completion: (() -> Void)?) {
     loadPersistentStore {
         completion?()
     }
 }

 // MARK: - Loading

 private func loadPersistentStore(completion: @escaping () -> Void) {
     //handle data migration on a different thread/queue here
     persistentContainer.loadPersistentStores { description, error in
         guard error == nil else {
             fatalError("was unable to load store \(error!)")
         }

         completion()
     }
 }

Ok, so now we have something to test - does calling setup actually set up our stack? Let's create that unit test class to find out:

class CoreDataManagerTests: XCTestCase {

    // MARK: Properties

    var sut: CoreDataManager!

    // MARK: - Lifecycle

    override func setUp() {
        super.setUp()

        sut = CoreDataManager()
    }

    // MARK: - Tests

    // MARK: Setup

    func test_setup_completionCalled() {
        let setupExpectation = expectation(description: "set up completion called")

        sut.setup() {
            setupExpectation.fulfill()
        }

        waitForExpectations(timeout: 1.0, handler: nil)
    }
}

In the above class, we declare a property sut (subject under test) which will hold a CoreDataManager instance that we will be testing. I prefer to use sut as it makes it immediately obvious which object we are testing and which objects are collaborators/dependencies. It's important to note that the sut property is an implicitly unwrapped optional. It's primarily used to make our unit tests more readable by avoiding the need to handle its optional nature elsewhere. It is a technique that I would not recommend using too often in production code. The test suite's setUp method is where the CoreDataManager instance is being created and assigned to sut.

Let's take a closer look at the unit test itself:

func test_setup_completionCalled() {
    let setupExpectation = expectation(description: "set up completion called")

    sut.setup() {
        setupExpectation.fulfill()
    }

    waitForExpectations(timeout: 1.0, handler: nil)
}

When it comes to naming I follow the naming convention:

test_[unit under test]_[condition]_[expected outcome]

[condition] is optional.

The test method signature tells us that we are testing the setup method and that the completion closure should be triggered.

Now with this test, we are really jumping straight into the deeper end of unit testing by testing an asynchronous method, but as we can see, the code isn't actually that difficult to understand. The first thing we do is create an XCTestExpectation instance. It's important to note here that we are not directly creating an XCTestExpectation instance using XCTestExpectation's init method; instead, we are using the convenience method provided by XCTestCase. By creating it via XCTestCase, we will tie the XCTestExpectation and XCTestCase together, allowing us to use waitForExpectations and reduce some of the boilerplate required for expectations. If you have never used expectations before, you can think of them as promising that an action will happen within a certain time frame. Sadly, like actual promises, they can be broken, and when they are, the test fails.

As I'm sure you have noted, test_setup_completionCalled doesn't actually contain any asserts; this is because we are using the expectation as an implicit assert.

We've tested that the completion closure is called, but we haven't actually checked that anything was set up. A successful set-up should result in our persistent store being loaded, so let's add a test to check that:

func test_setup_persistentStoreCreated() {
   let setupExpectation = expectation(description: "set up completion called")

    sut.setup() {
        setupExpectation.fulfill()
    }

    waitForExpectations(timeout: 1.0) { (_) in
        XCTAssertTrue(self.sut.persistentContainer.persistentStoreCoordinator.persistentStores.count > 0)
    }
}

As we can see, test_setup_persistentStoreCreated contains an assert to check that the persistentStoreCoordinator has at least one persistentStore. It's important to note that as persistentContainer is lazy-loaded merely checking that it's not nil wouldn't be a valid test as calling the property would result in creating the persistentContainer.

The two unit tests that we have added are very similar, and as you can see, it's possible for test_setup_persistentStoreCreated to fail for two reasons:

1. Completion closure not triggered
2. Persistent store not created

The first reason is actually being tested in test_setup_completionCalled, so why have I created another test that's dependent on it here? The reason is that it's actually impossible not to check this condition, as it's an implicit dependency on any test that uses this method. Now the argument could be made that these two tests should be one - effectively a test_setup_stackCreated test. I opted for two tests, as it improved readability if one of the tests failed, providing a higher level of granularity for what had caused the failure. You sometimes hear people saying that a unit test should only ever have one assert and that any unit test with more than one assert is wrong. IMHO, this is foolhardy. There are very few hard and fast rules in life, just about everything is context based - in this context, having two asserts (one implicit, one explicit) in test_setup_persistentStoreCreated makes sense as both asserts are checking that the same unit of functionality is correct.

Now, the more eagled eyed 👀 among you will have spotted that we are using the default storage type for our Core Data stack (NSSQLiteStoreType) in the above tests - this creates a SQLite file on the disk and as we know, any I/O operation is going to be much slower than a pure in-memory operation. It would be great if we could tell the Core Data stack which storage type to use - NSSQLiteStoreType for production and NSInMemoryStoreType for testing:

class CoreDataManager {

    private var storeType: String!

    lazy var persistentContainer: NSPersistentContainer! = {
        let persistentContainer = NSPersistentContainer(name: "TestingWithCoreData_Example")
        let description = persistentContainer.persistentStoreDescriptions.first
        description?.type = storeType

        return persistentContainer
    }()

    // MARK: - Singleton

    static let shared = CoreDataManager()

    // MARK: - SetUp

    func setup(storeType: String = NSSQLiteStoreType, completion: (() -> Void)?) {
        self.storeType = storeType

        loadPersistentStore {
            completion?()
        }
    }

    // MARK: - Loading

    private func loadPersistentStore(completion: @escaping () -> Void) {
        persistentContainer.loadPersistentStores { description, error in
            guard error == nil else {
                fatalError("was unable to load store \(error!)")
            }

            completion()
        }
    }
}

As a long-term Objective-C iOS developer, I still get a thrill about being able to provide a default value to a parameter (like we do with the storeType parameter above) without having to create a whole new method. This change will enable us to utilise the significantly faster NSInMemoryStoreType storage type in our tests while maintaining a simple interface in production. However, it's not free and because we have introduced a new way of setting up our Core Data stack, we need to update our existing tests to test this new path:

class CoreDataManagerTests: XCTestCase {

    // MARK: Properties

    var sut: CoreDataManager!

    // MARK: - Lifecycle

    override func setUp() {
        super.setUp()

        sut = CoreDataManager()
    }

    // MARK: - Tests

    // MARK: Setup

    func test_setup_completionCalled() {
        let setupExpectation = expectation(description: "set up completion called")

        sut.setup(storeType: NSInMemoryStoreType) {
            setupExpectation.fulfill()
        }

        waitForExpectations(timeout: 1.0, handler: nil)
    }

    func test_setup_persistentStoreCreated() {
       let setupExpectation = expectation(description: "set up completion called")

        sut.setup(storeType: NSInMemoryStoreType) {
            setupExpectation.fulfill()
        }

        waitForExpectations(timeout: 1.0) { (_) in
            XCTAssertTrue(self.sut.persistentContainer.persistentStoreCoordinator.persistentStores.count > 0)
        }
    }

    func test_setup_persistentContainerLoadedOnDisk() {
        let setupExpectation = expectation(description: "set up completion called")
        
        sut.setup {
            XCTAssertEqual(self.sut.persistentContainer.persistentStoreDescriptions.first?.type, NSSQLiteStoreType)
            setupExpectation.fulfill()
        }
        
        waitForExpectations(timeout: 1.0) { (_) in
            self.sut.persistentContainer.destroyPersistentStore()
        }
    }

    func test_setup_persistentContainerLoadedInMemory() {
        let setupExpectation = expectation(description: "set up completion called")

        sut.setup(storeType: NSInMemoryStoreType) {
            XCTAssertEqual(self.sut.persistentContainer.persistentStoreDescriptions.first?.type, NSInMemoryStoreType)
            setupExpectation.fulfill()
        }

        waitForExpectations(timeout: 1.0, handler: nil)
    }
}

If we run the above tests, we can see the difference in running times between test_setup_persistentContainerLoadedInMemory and test_setup_persistentContainerLoadedOnDisk:

As you can see on the above execution of both tests, loading the store on disk took 17 times longer than loading the store into memory - 0.001 vs 0.017 seconds.

In real terms, this speed increase isn't much on its own, but once we start adding in tests that create, update and delete NSManagedObject instances, dealing with an in-memory store will allow these tests to be executed faster than if we used an on-disk store.

So far, we have made great progress on producing a Core Data stack that is unit testable, but a Core Data stack without a context (or two) isn't going to be very useful - let's add some:

lazy var backgroundContext: NSManagedObjectContext = {
    let context = self.persistentContainer.newBackgroundContext()
    context.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy

    return context
}()

lazy var mainContext: NSManagedObjectContext = {
    let context = self.persistentContainer.viewContext
    context.automaticallyMergesChangesFromParent = true

    return context
}()

And we need to add some unit tests for them:

func test_backgroundContext_concurrencyType() {
    let setupExpectation = expectation(description: "background context")

    sut.setup(storeType: NSInMemoryStoreType) {
        XCTAssertEqual(self.sut.backgroundContext.concurrencyType, .privateQueueConcurrencyType)
        setupExpectation.fulfill()
    }

    waitForExpectations(timeout: 1.0, handler: nil)
}

func test_mainContext_concurrencyType() {
    let setupExpectation = expectation(description: "main context")

    sut.setup(storeType: NSInMemoryStoreType) {
        XCTAssertEqual(self.sut.mainContext.concurrencyType, .mainQueueConcurrencyType)
        setupExpectation.fulfill()
    }

    waitForExpectations(timeout: 1.0, handler: nil)
}

The good news is that we have finished creating and testing our Core Data stack, and I think it wasn't actually too difficult 🎉.

Introducing ColorsDataManager

There is no point in creating a Core Data stack if we don't actually use it. In the example project, we populate a collectionview with instances of a subclass of NSManagedObject - Color. To help us deal with these Color objects, we will be using a ColorsDataManager:

class ColorsDataManager {

    let backgroundContext: NSManagedObjectContext

    // MARK: - Init

    init(backgroundContext: NSManagedObjectContext = CoreDataManager.shared.backgroundContext) {
        self.backgroundContext = backgroundContext
    }

    // MARK: - Create

    func createColor() {
        backgroundContext.performAndWait {
            let color = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: backgroundContext) as! Color
            color.hex = UIColor.random.hexString
            color.dateCreated = Date()

            try? backgroundContext.save()
        }
    }
    
    // MARK: - Deletion
    
    func deleteColor(color: Color) {
        let objectID = color.objectID
        backgroundContext.performAndWait {
            if let colorInContext = try? backgroundContext.existingObject(with: objectID) {
                backgroundContext.delete(colorInContext)
                
                try? backgroundContext.save()
            }
        }
    }
}

In the above class, we have a simple manager that handles creating and deleting Color instances. As a responsible member of the Core Data community, our example app treats the backgroundContext as a read-write context and the mainContext as a read-only context. This ensures that any time-consuming tasks don't block the main (UI) thread. You may have noticed that the above class doesn't actually contain any mention of CoreDataManager; instead, this class only knows about the background context. By injecting this context into the class, we can decouple ColorsDataManager from CoreDataManager, which should allow us to test ColorsDataManager more easily 😉.

Let's look at implementing our first test for ColorsDataManager:

class ColorsDataManagerTests: XCTestCase {

    // MARK: Properties

    var sut: ColorsDataManager!

    var coreDataStack: CoreDataTestStack!

    // MARK: - Lifecycle

    override func setUp() {
        super.setUp()

        coreDataStack = CoreDataTestStack()

        sut = ColorsDataManager(backgroundContext: coreDataStack.backgroundContext)
    }

    // MARK: - Tests

    // MARK: Init

    func test_init_contexts() {
        XCTAssertEqual(sut.backgroundContext, coreDataStack.backgroundContext)
    }
}

As we can see, the majority of the above class is devoted to setting up the test suite. The test itself merely checks that the context passed into ColorsDataManager is the same context assigned to the backgroundContext property.

You may also have noticed that coreDataStack isn't a CoreDataManager instance but instead a CoreDataTestStack instance.

Let's go on a slight detour and have a look at CoreDataTestStack:

class CoreDataTestStack {

    let persistentContainer: NSPersistentContainer
    let backgroundContext: NSManagedObjectContextSpy
    let mainContext: NSManagedObjectContextSpy

    init() {
        persistentContainer = NSPersistentContainer(name: "TestingWithCoreData_Example")
        let description = persistentContainer.persistentStoreDescriptions.first
        description?.type = NSInMemoryStoreType

        persistentContainer.loadPersistentStores { description, error in
            guard error == nil else {
                fatalError("was unable to load store \(error!)")
            }
        }

        mainContext = NSManagedObjectContextSpy(concurrencyType: .mainQueueConcurrencyType)
        mainContext.automaticallyMergesChangesFromParent = true
        mainContext.persistentStoreCoordinator = persistentContainer.persistentStoreCoordinator

        backgroundContext = NSManagedObjectContextSpy(concurrencyType: .privateQueueConcurrencyType)
        backgroundContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        backgroundContext.parent = self.mainContext
    }
}

CoreDataTestStack is very similar to CoreDataManager but with its asynchronous setup behaviour stripped out and the storeType always set to NSInMemoryStoreType. This class enables us to set up and tear down the stack more easily between tests without having to wait for any asynchronous tasks to complete. Another difference between CoreDataTestStack and CoreDataManager is that the two contexts that are being created are not in fact standard NSManagedObjectContext instances but are actually NSManagedObjectContextSpy instances.

If you are curious as to what a spy is, Martin Fowler has produced a very insightful article on naming test objects - it's in the section titled: The Difference Between Mocks and Stubs.

NSManagedObjectContextSpy is a subclass of NSManagedObjectContext that adds special state tracking properties. System classes occupy a grey area when it comes to whether you should use them in your tests or if you need to replace them with mock/stub instances. In this case, mocking out a context's functionality would be too much work and would actually be counter-productive to what the test is attempting to achieve, so I'm perfectly happy to use it directly.

class NSManagedObjectContextSpy: NSManagedObjectContext {
    var expectation: XCTestExpectation?

    var saveWasCalled = false

    // MARK: - Perform

    override func performAndWait(_ block: () -> Void) {
        super.performAndWait(block)

        expectation?.fulfill()
    }

    // MARK: - Save

    override func save() throws {
        save()

        saveWasCalled = true
    }
}

Ok, detour over. Let's get back to testing the ColorsDataManager class:

func test_createColor_colorCreated() {
    let performAndWaitExpectation = expectation(description: "background perform and wait")
    coreDataStack.backgroundContext.expectation = performAndWaitExpectation

    sut.createColor()

    waitForExpectations(timeout: 1) { (_) in
        let request = NSFetchRequest.init(entityName: Color.className)
        let colors = try! self.coreDataStack.backgroundContext.fetch(request)

        guard let color = colors.first else {
            XCTFail("color missing")
            return
        }

        XCTAssertEqual(colors.count, 1)
        XCTAssertNotNil(color.hex)
        XCTAssertEqual(color.dateCreated?.timeIntervalSinceNow ?? 0, Date().timeIntervalSinceNow, accuracy: 0.1)
        XCTAssertTrue(self.coreDataStack.backgroundContext.saveWasCalled)
    }
}

There is a bit more happening here than in the previous test that we saw. We create an XCTestExpectation instance that we then assign to the expectation property on the context. As we have seen above this expectation should be fulfilled when performAndWait is called. Once that expectation has been triggered, we then check that the Color instance was created and saved into our persistent store.

Testing the deletion of a Color follows a similar pattern:

func test_deleteColor_colorDeleted() {
    let performAndWaitExpectation = expectation(description: "background perform and wait")
    coreDataStack.backgroundContext.expectation = performAndWaitExpectation
    
    let colorA = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    let colorB = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    let colorC = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    
    sut.deleteColor(color: colorB)
    
    waitForExpectations(timeout: 1) { (_) in
        let request = NSFetchRequest.init(entityName: Color.className)
        let backgroundContextColors = try! self.coreDataStack.backgroundContext.fetch(request)
        
        XCTAssertEqual(backgroundContextColors.count, 2)
        XCTAssertTrue(backgroundContextColors.contains(colorA))
        XCTAssertTrue(backgroundContextColors.contains(colorC))
        XCTAssertTrue(self.coreDataStack.backgroundContext.saveWasCalled)
    }
}

We first populate our persistent (in-memory) store, call the deleteColor method and then check that the correct Color has been deleted. There is one special case - because we read on the main context and delete on the background context, the color instance passed into this method may be from the main context, the above test is not covering this case, so let's add another test that does:

func test_deleteColor_switchingContexts_colorDeleted() {
    let performAndWaitExpectation = expectation(description: "background perform and wait")
    coreDataStack.backgroundContext.expectation = performAndWaitExpectation
    
    let colorA = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    let colorB = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    let colorC = NSEntityDescription.insertNewObject(forEntityName: Color.className, into: self.coreDataStack.backgroundContext) as! Color
    
    let mainContextColor = coreDataStack.mainContext.object(with: colorB.objectID) as! Color
    
    sut.deleteColor(color: mainContextColor)
    
    waitForExpectations(timeout: 1) { (_) in
        let request = NSFetchRequest.init(entityName: Color.className)
        let backgroundContextColors = try! self.coreDataStack.backgroundContext.fetch(request)
        
        XCTAssertEqual(backgroundContextColors.count, 2)
        XCTAssertTrue(backgroundContextColors.contains(colorA))
        XCTAssertTrue(backgroundContextColors.contains(colorC))
        XCTAssertTrue(self.coreDataStack.backgroundContext.saveWasCalled)
    }
}

Pretty much the same as before, with the only difference being that we retrieve the color to be deleted from the main context before passing that in.

An interesting point to note when adding those three tests is that we haven't had to add any code to clear our persistent store. This is because by using an NSInMemoryStoreType store and ensuring that we create a new stack before each test, we never actually persist data. Not only does this save us time by eliminating the need to write the tidy-up code, but it also eliminates a whole category of bugs where leftover state from one test affects the outcome of another due to faulty/missing clean-up code.

To come back to the point about in-memory stores being quicker to use than on-disk stores, we can see a typical difference in running times for the above tests below:

In-memory store

SQLite store

Best Friends Forever?

In the above code snippets, we have seen that unit testing with Core Data doesn't need to be that much more difficult than unit testing in general, with most of that added difficulty coming in the set up of the Core Data stack. And while Core Data and Unit Testing may not become BFFs (let's be honest, UIKit has that position sealed down), we've seen that they can become firm friends 👫. That friendship is built on small alterations to our Core Data stack, which allows its data to be more easily discarded, and the use of special subclasses to enable us to track state better.

You can download the example project for this post here.