Vitor Enes on Vitor Enes

Planet-Scale Leaderless Consensus

Fri, 18 Nov 2022 00:00:00 +0000

Efficient Replication via Timestamp Stability

Tue, 27 Apr 2021 00:00:00 +0100

State-Machine Replication for Planet-Scale Systems

Wed, 29 Apr 2020 00:00:00 +0100

Paxos Made Wrong (part I)

Wed, 18 Mar 2020 19:45:00 +0000

In each blog post of this series, I will present a modification of Paxos, together with a sequence of protocol messages, that will lead to an invariant violation. Hopefully this will lead to a better understanding of Paxos (mine and yours).

On a quest for understanding

Maybe it’s just me, but I often gain intuition on why a protocol works with situations where it doesn’t. A way to do that, is to change one step of the protocol (e.g. change a comparison from $>$ to $\leq$ or reduce the size of a quorum by 1) and see what happens.

These protocol modifications typically lead to an invariant violation (e.g. processes disagree on the consensus value). Then, when examining this misbehavior, we can gain more intuition on why that protocol step had to be the way it was. The more we do this, the more understanding we build, and at some point, we will have a pretty good grasp of what’s going on.

If the modification doesn’t lead to an invariant violation and the protocol remains correct¹, then maybe the protocol is unnecessarily strict and there might be room for improvement. Flexible Paxos reminds me exactly of that: a rigid requirement in Paxos that, when lifted, opened the door to new optimizations. (To be clear, I’m not saying this is how it happened.)

Mutation testing

In a way, this is similar to mutation testing but with a different goal. In mutation testing, the program suffers mutations to check the quality of tests: if after the mutation the test still passes, then it’s not a good test, and we should improve it.

In our case:

the mutated program is the protocol
the tests validate the protocol invariants
but the goal is not to improve the tests
instead, a failing test – in our case, an invariant that didn’t hold – will help us understand the protocol a bit better
with a passing test – in our case, an invariant that did hold – we may be on our way to an optimization – yay! (this is unlikely and won’t be the goal)

“The tests validate the protocol invariants”

To better understand this, let’s think about unit tests. Typically a unit test will have a setup, run some program logic, and in the end check that everything went well. These checks in the end are the protocol invariants. But we still need the setup and the program logic to be executed.

As an example on what I mean by setup and program logic in the context of distributed protocols, let’s think about Paxos:

the setup would be the number of processes $n$ and the number of allowed faults $f$
the program logic would be the set of messages that are exchanged between processes

With something like property testing, the testing framework can generate the setup and protocol messages for us. Then, for each test case generated, the framework checks some property. In our case, this property would be a protocol invariant.

I’m not planning to, but I may need to resort to property testing if at some point I’m unable to generate an example with a failing invariant. Nevertheless, it doesn’t matter where the example comes from, as long as it helps building intuition, ultimately leading to a full understanding.

What’s next

In the next blog post I will probably start with a specification of Paxos that we can modify in later posts.

Before any parting words, I would like to thank Cláudia Brito (@cbrito_18) and Rogério Pontes (@rogerio_acp) for their feedback on earlier versions of this post.

Until next time. Stay safe!

Knowing that some protocol change maintains correctness is a more complex topic. My go-to option would be adapting previous pen-and-paper proofs and existing TLA+ models. ^{^}

Fault-tolerant Causal Stability (part I)

Mon, 22 Jul 2019 00:00:00 +0100

You guessed it! This post shows that 1) causal stability is not fault-tolerant but 2) we can do something about it.

Well, that was the intention, at least. The blog post was becoming too long, so I’ll divide it in (exactly those) two:

part I: revisits the concepts of causal consistency and causal stability; we’ll also discuss how these two concepts can be implemented in a simple way; we’ll conclude that causal stability is not fault-tolerant
part II: shows that causal stability can be fault-tolerant and the level of fault-tolerance provided simply depends on the size of the write quorum

Causal Consistency

In simple terms, causal consistency ensures that:

if operation $o$ causes operation $p$
then every process in the system will not observe $p$ without observing $o$

In other words, no effect is observed without its cause.

In Figure 1 we have an example where causal consistency is not respected. We start two objects $x$ and $y$ initialized at $0$. Alice writes $x = 1$ and then $y = 2$. At this point, in a causally consistent system, Bob can see:

$x = 0 \land y = 0$
$x = 1 \land y = 0$
$x = 1 \land y = 2$

However, it ended up reading $x = 0 \land y = 2$ (i.e. it saw the second write by Alice without seeing the first), and with that, the run in Figure 1 is not causally consistent.

A run that is not causally consistency.

Implementing causal consistency

We can implement a causally consistent system with two simple steps:

attach to each operation $o$ a set containing all its preceding operations; let’s call this set the dependencies of $o$ and denote it by $\textsf{dep}[o]$
execute $o$ once all operations in $\textsf{dep}[o]$ have been executed

(Note how this fixes the problem presented in Figure 1.)

Let’s also denote the set of executed operations by $\textsf{Executed}$.

If the process submits operation $o$, then the dependencies of $o$ will be the set of operations that have been executed at that moment, i.e. $\textsf{dep}(o) = \textsf{Executed}$.

A note on concurrency

If operation $o$ causes operation $p$, then $o$ is part of the set of dependencies of $p$:

$$o \in \textsf{dep}(p)$$

However, it can happen that $o$ doesn’t cause $p$ and $p$ doesn’t cause $o$.
When this is the case, we say that $o$ and $p$ are concurrent:

$$o \not \in \textsf{dep}(p) \land p \not \in \textsf{dep}(o)$$

Causal Stability

An operation $o$ is causally stable (denoted by $\textsf{stable}(o)$) if every operation $p$ that will be executed in the system must be caused by $o$:

$$\textsf{stable}(o) \equiv \forall p \cdot p \not \in \textsf{Executed} \implies o \in \textsf{dep}(p)$$

In other words, if $\textsf{stable}(o)$ then every operation will be in the future of $o$.

The concept of causal stability is exploited, for example, by CRDTs (Section 4.1) to perform garbage-collection (Section 7.2).

Getting some intuition with an example

Let’s pause for a bit with an example where we compute the set of stable operations.

In this example we have:

3 processes: a, b and c
4 operations: $o$, $p$, $q$ and $r$

Let’s say that:

a submitted $o$ and then $p$
b saw $o$ and $p$, and then submitted $q$
c saw $o$ and submitted $r$

So, we have the following $\textsf{Executed}$:

$\textsf{Executed}_{\textbf{a}} = \{o, p\}$
$\textsf{Executed}_{\textbf{b}} = \{o, p, q\}$
$\textsf{Executed}_{\textbf{c}} = \{o, r\}$

Recall that: the set of dependencies of an operation is the set of executed operations at the time the operation is submitted.

So at this point we know that any operation submitted (by any process) will include $o$ as a dependency. This means that $o$ is stable.

We can’t say the same about any of the remaining operations.

Implementing causal stability

Let’s denote the set of all processes in the system by $\mathcal{I}$.

Stability can be detected if we learn¹ what’s been executed by each process, i.e. we know $\textsf{Executed}_i$ for every process $i \in \mathcal{I}$.

Then, the set of stable operations is given by the intersection of all $\textsf{Executed}_i$’s:

$$\bigcap_{i \in \mathcal{I}} \textsf{Executed}_i$$

A (second) note on concurrency

Given operation $o$, $\textsf{stable}(o)$ implies that all operations concurrent with $o$ have also been executed.

Let’s see why that is:

for some operation $p$ be concurrent with $o$, then $o$ can’t be a dependency of $p$: $o \not \in \textsf{dep}(p)$
however, this contradicts $\textsf{stable}(o)$ that requires that $o$ is a dependency of every operation that will be executed, and in particular, that it is a dependency of $p$: $o \in \textsf{dep}(p)$.

Causal stability is a local property

What’s stable on one process might not (yet) be stable on other processes.

This is because processes compute stability based on what they know others have executed, and different processes might have different views on what’s been executed.

However, if an operation is stable on some process, it will eventually become stable on every process.

(Not-so-fault-tolerant) Causal Stability

At this point, we already know why causal stability is not fault-tolerant.

In order to advance stability, processes need to learn what all processes are executing, and if some process crashes, the set of stable operations will not grow. And that’s not fault-tolerant.

So in part II we’ll try to fix that!

UPDATE: I’ll address the concerns raised in the replies to my tweet in the next blog post.

This can be implemented, for example, by having processes periodically broadcasting what they have executed. ^{^}

Efficient Synchronization of State-based CRDTs

Tue, 16 Apr 2019 00:00:00 +0100

This week I was at ICDE’19 where I presented Efficient Synchronization of State-based CRDTs (here’s a link to paper and slides). Following Pedro’s idea, and since the talk is still fresh, this post will be a transcript of it.

Context

In Figure 1, I briefly explain the CRDT acronym. My previous post was just about that.

Conflict-free Replicated Data Types.

Outline

CRDT variants: First, we’ll take a look CRDT synchronization models. The main variants are operation-based and state-based. There’s also delta-based, that is a variant of state-based synchronization.
From filter to decomposition; filter; join: In classic delta-based, when a replica receives a delta, it checks whether this delta has something new (the filter), and if it has, the delta is propagated to the peers in the system. We’ll show that this is insufficient, and a more sophisticated approach is necessary: replicas should instead decompose the delta (the decomposition), select the relevant parts (the filter), and then merge them back together (the join), effectively extracting what’s new in the received delta.

A super-realistic example of decomposition; filter; join: Consider a wedding cake. When we decompose it, we get all the layers. Then, we decide that we just want the first and the third layer. In the end, we glue the interesting layers back together.

Decomposition of state-based CRDTs: The filter and join operations are already part of the CRDT framework. We’ll see how they look like in the simplest CRDT (a grow-only set). Then, we’ll introduce the missing part (the decomposition) and the three properties that define “good” decompositions.
And finally, some experimental results before we go.

The outline of the talk.

CRDT synchronization models

Two of the main differences between CRDTs synchronization models are:

the guarantees that the synchronization middleware should provide
the payload exchanged between replicas upon a new update

Operation-based

In the operation-based model, the middleware should provide exactly-once causal delivery of operations (which can be expensive). The upside: only the update operation (typically small) needs to be sent.

State-based

In the state-based model, messages can be dropped, reordered and duplicated, meaning no expensive middleware is required in state-based. The downside: the full CRDT state needs to be sent when synchronizing replicas.

Delta-state-based

Delta-state-based promises to be the best of both worlds by inheriting the middleware guarantees from state-based (i.e. none), and by exchanging deltas that are typically as small as operations.

Different CRDT variants (i.e. synchronization models).

But in reality…

There are inefficiencies in the delta-based synchronization model:

In terms of bandwidth (LHS of Figure 4), delta-based can be as expensive as state-based (i.e., effectively sending the full state on every synchronization).
And since large states are being exchanged, trying to compute these deltas results in a substantial CPU overhead, when comparing to state-based (RHS of Figure 4).

Delta-based can be as bad as state-based in terms of bandwidth, while incurring a substantial CPU overhead.

So, in this paper…

Our main contribution.

State-based CRDTs

As I promised above, we’ll explore the ideas behind the paper resorting to the simplest CRDT that exists: a set. And from sets, we’ll just need two binary operations: subset and set union.

I mentioned before that all CRDTs have a filter and a join operation. For sets, the filter is the subset, while the join is the set union.

State-based vs Delta-based

State-based CRDTs define functions that allow us to update the CRDT state. These are called mutators.

On the other hand, delta-based CRDTs define delta-mutators that returns deltas. We can think of these deltas as the state difference between the state before the update and the state after.

As an example, if we’re trying to add the element c to a set that contains a and b (i.e., {a, b}), the mutator will return the set {a, b, c}, while the delta-mutator simply returns the delta {c}.

This delta is then:

joined with the local state resorting to join operation (in this case, the set union); after this step, the local state is the same as it would have been if the mutator was used (in our example, we would get {a, b, c})
added to a delta-buffer that contains deltas to be propagated to peers

State-based CRDTs define mutators while delta-based CRDTs define delta-mutators.

Introducing some notation before an example

On the right of our replica, we’ll depict the current CRDT state (an empty set in Figure 7). At the bottom, we’ll have the current list of deltas in the delta-buffer (an empty delta-buffer in Figure 7).

Delta-buffer notation: on the right, the CRDT state; at the bottom, the delta-buffer.

The problem with the classic delta propagation

Consider the example in Figure 8 with four replicas, A, B, C, and D. All replicas start with an empty set and an empty delta-buffer.

When A adds the element x to the set, the delta-mutator produces the delta {x}. This delta is joined with the local state (that becomes {x}, since union of { } with {x} is {x}), and it is added to the delta-buffer.
A synchronizes with C by sending the new deltas in the delta-buffer (i.e. just {x}). When C receives this delta, it joins it with the local state and also adds the delta to its delta-buffer, so that this delta can be further propagated.
C syncs with D (the process is similar to the one above).
Now A adds element y to the set. The resulting delta, {y}, is joined with the local state (that becomes {x, y}) and added to the delta-buffer.
A syncs with B by sending the join of all the deltas never sent to B (i.e. {x, y}). B does the standard thing when receiving the delta.
B syncs with C and now we get to the interesting part. Recall that the local state of C before receiving this delta from B is {x}. Although I didn’t mentioned yet in this running example, in the outline of the talk we’ve seen that when a delta is received, a filter occurs: when C receives {x, y}, it checks whether this delta has something new (compared to the local state). And indeed, there is something new (element y)! Since there’s something new, the delta is joined with the local state and added to the delta-buffer.
And finally, C syncs with D by sending the new delta in its delta-buffer (i.e. {x, y}).

A distributed execution of classic delta-based synchronization showing an inefficiency in delta propagation.

The last step is problematic because C sends {x, y} to D, even though in its previous sync step with D it sent {x}. We would expect that now it would send the new state changes, i.e. only {y}.

This shows that the simple filter done by C when it received {x, y} is not enough! Instead, C must decompose the received delta {x, y}, select/filter the interesting bits, join them back together, and only then, add what results (that should be {y}) to the delta-buffer.

We already know the filter and join operation (at least for sets). Let’s now see how we do the decomposition.

Decomposition of state-based CRDTs

In Figure 9 we have a decomposition example: given the set {a, b, c}, the decomposition of this set should be {{a}, {b}, {c}}¹.

In the paper, we define three properties, that when respected, produce “good” decompositions:

The join everything in the decomposition produces the original element. As an example, {{b}, {c}} is not a good decomposition because its join only produces {b, c}, and not {a, b, c}.
All elements in the decomposition are needed. {{a, b}, {b}, {c}} is not a good decomposition because {b} is not needed to produce {a, b, c}.
No element can be further decomposed. {{a, b}, {c}} is not a good decomposition because one of its elements, namely {a, b}, can be further decomposed into {a} and {b}.

Only the decomposition {{a}, {b}, {c}} respects all three properties.

Example decomposition and the three properties of good decompositions.

(In the paper we show that for all states of CRDTs used in practice, a decomposition always exists, and furthermore, this decomposition is unique.)

Introducing the CRDT-difference operation

Once we have a decomposition, now we can build a CRDT-difference binary operation.

For sets, this operation boils down to the set difference that can be simply defined as:

$$ a \setminus b = \{ x \in a \mid x \not \in b \}$$

As an example (from Figure 10), the difference between {x, y} and {x} is {y} (i.e. {x, y}$\setminus${x} = {y}).

This operation can be generalized for CRDTs resorting to the decomposition we’ve just introduced. We don’t need to worry much about the formula in Figure 10, but the general idea is the following: for each element x in the decomposition of the first argument (i.e. a), we check whether that x “is needed” in the second argument (i.e. b) resorting to the filter operation; in the end, we join all x that passed the filter, in order to obtain the final CRDT state difference.

A new binary operation for CRDTs.

Going back to our example

Now we’re ready to “fix” our example.

When C receives {x, y} from B, instead of checking if {x, y} has anything new, it computes the difference between the received delta {x, y} and its local state {x}. This returns only the new pieces of information in the received delta, i.e. simply {y}.

The returned {y} is added to the delta-buffer, and now the last sync step is exactly what we expected: only {y} is sent from C to D.

Solving the inefficiency with the difference operation.

Since the replica receiving the delta, removes redundant state in the received delta, we denote this optimization by RR.

Note that this problem only occurred because C received the same piece of information ({x}) from two different replicas: both from A and from B. This means that we have a cycle in the network topology (this will be relevant in the experimental evaluation).

A very simple optimization

In Figure 12 we have depicted another problem in delta propagation: A produced a delta (by adding an element to the set), sent this delta to B, and B sent this delta back to A.

This behavior is undesirable but fortunately it is super simple to fix. Each delta in the delta-buffer should be tagged with its origin (in the example, B would tag {x} with A), and never sent back to the origin in the next sync steps. This optimization is denoted by BP given that replicas should avoid back-propagation of received deltas.

Avoiding back-propagation of deltas.

Evaluation

This post is already becoming super long, so let’s now try to finish it quickly.

As we’ve seen before, cycles in the network topology might result in redundant state being propagated between replicas. With that, in the evaluation we wanted to observe the behavior of the different synchronization models with and without cycles in the topology.

For that, we ran experiments with a tree (acyclic) and a partial-mesh (cyclic). Both these topologies are depicted in Figure 13.

The two network topologies used in the experiments: tree and partial-mesh.

Set micro-benchmark

In Figure 14 we have the bandwidth required by state-based, delta-based, delta-based BP, delta-based RR, and delta-based BP+RR, when synchronizing a replicated set. The results with the tree are on the left, and the results with the partial-mesh are on the right.

If we look at the first two bars, we can see that vanilla delta-based represents no improvement when compared to state-based, independently of the topology employed.
The third and fourth bars reveal something interesting: RR is only required when the topology has cycles, since BP alone achieves the best result (BP+RR) in the acyclic topology.

Bandwidth results from the set micro-benchmark.

You may be wondering what’s difficult about that!?!?

For sets, this looks super simple. And I really hope that’s indeed the case, since this was the goal. However, all of this generalizes for any CRDT, as long as we know how to decompose them.

An excerpt from the last appendix in the paper:

In this section we show that for each composition technique there is a corresponding decomposition rule. As the lattice join ⊔ of a composite CRDT is defined in terms of the lattice join of its components [35], decomposition rules of a composite CRDT follow the same idea and resort to the decomposition of its smaller parts.

You can find such rules in Figure 15.

The decomposition rule of each composition technique.

There’s much more in the paper

On the experimental side we have other micro-benchmarks (counter and map), comparisons with operation-based CRDTs and two variants of the Scuttlebutt algorithm, and a Retwis benchmark
On the theory side:
- Resorting to the CRDT-difference operation, now we know what optimal delta-mutators should return (optimal in the sense that they return the smallest delta)
- State-based CRDTs are not only join-semilattices, they are lattices, and even more than that, they are distributive lattices!!!

The end

I hope you’ve enjoyed this transcript, and that it didn’t end up being too dense. This paper has also been covered by The Morning Paper.

If any question comes up, don’t hesitate!

This is basically a set partition with a further restriction (property 3) on the possible elements in the partition. ^{^}

Genuine vs Artificial Conflict-freeness

Sat, 13 Apr 2019 00:00:00 +0100

This blog post started with me trying to write a transcript of the talk I gave at ICDE’19 this week on Efficient Synchronization of State-based CRDTs. Soon after, it derailed in this.

UPDATE: The full transcript of the talk can now be found here.

The figure above was my first slide, where I attempted to explain the CRDT acronym. From right to left:

Data Types

In CRDTs, the abstraction is richer than simply reading and writing registers: we can increment/decrement counters, add/remove elements to/from sets, and so on…

Replicated

This is not specific to CRDTs. We replicate for quality of service. For example, with users across the globe, if we deploy replicas also across the globe, we can have low latency when accessing data. Also, if some replica is unavailable, users can fail-over to another close-by replica.

Conflict-free

Recall the PACELC theorem: if we have a network Partition, we must pick between Availability and Consistency; Else, even when everything is fine network-wise, we still must decide between low Latency and Consistency.

CRDTs are PA/EL. To achieve this, all updates are local to a replica. Then, in the background, replicas communicate in order to propagate new updates between them.

This approach can lead to conflicts when replicas concurrently write to the same object. And if that happens, which write should win? Which write is the one we want?

Why having a richer abstraction helps?

If we just have a register abstraction, any two concurrent writes to the same object conflict. But if these writes are increments to counters, there’s no conflict. Similarly, when elements are added to a set, no conflict is produced.

The general rule is that if two operations commute sequentially, then they are genuinely-conflict-free. When that doesn’t happen, CRDTs can pick one of the operations, becoming artificially-conflict-free.

Examples of pairs of operations that are genuinely-CF:

increment / increment
increment / decrement
add A / add A
add A / add B
add A / remove B

Examples of pairs of operations that can be artificially-CF:

write / write
add A / remove A

Notice the “can be”: once we have identified pairs of conflicting operations, we can pick one of these operations or not. For example, a last-writer-wins register will pick one of the writes (based on a wall-clock timestamp), but a multi-value register will keep both (well, in fact, this is the only example I can think of in which both conflicting operations are kept).

Being artificial leads to concurrency semantics

Let’s focus on the last pair add A / remove A. Here we have three options:

pick the add, and have a add-wins set
pick the remove, and have a remove-wins set
pick whichever has the highest timestamp, and have a last-writer-wins set

For applications, now this means that they need to figure in which artificial way should potential conflicts be resolved. For some, a removal can be discarded in the presence of an addition, while for others, the removal should always prevail. (I was trying to come up with application examples for both, but all ideas felt too farfetched. Any suggestions?)

Before we go

If you’re curious on concurrency semantics for other data types, check Section 2.1 of this great CRDT overview by Nuno Preguiça: Conflict-free Replicated Data Types: An Overview.

And as a good-bye note, we can say that “only by being artificial, one can be completely conflict-free”.

Efficient Synchronization of State-based CRDTs

Tue, 09 Apr 2019 00:00:00 +0100

Threshold union: generalizing unions and intersections

Sun, 11 Nov 2018 23:04:00 +0000

Part I

So, yesterday I was wondering about operations on a set of sets (I’ll explain the Tweet content below, so you can just ignore it for now).

when we have a set of sets S, an element belongs to the:
- union of S, if it belongs to at least 1 sets in S
- intersection of S, if it belongs to at least |S| sets in S
- _____ of S, if it belongs to at least N sets in S
what should I write in _____?
— Vitor Enes (@vitorenesduarte) November 10, 2018

For example, consider that we have 3 sets:

$A = \{a, b, c\}$
$B = \{a, b, d\}$
$C = \{a\}$

and we build a set $S = \{A, B, C\}$ with the previous sets.

The union of all sets has all elements:

$$\bigcup S = \{a, b, c, d\}$$

The intersection of all sets has the elements common to all sets:

$$\bigcap S = \{a\}$$

Now, we can generalize these definitions and have:

an element belongs to $\bigcup S$ if it belongs to at least 1 set in $S$
an element belongs to $\bigcap S$ if it belongs to at least $|S|$ sets in $S$ ¹

Once we have this, we can define $\bigcup_n$:

an element belongs to $\bigcup_n S$ if it belongs to at least $n$ sets in $S$

and $\bigcup$ and $\bigcap$ can be written in terms of $\bigcup_n$:

$\bigcup S = \bigcup_1 S$
$\bigcap S = \bigcup_{|S|} S$

So, my question was: What’s the name of $\bigcup_n$?

After much googling and no success, it came to mind that maybe this was not about operations on sets, but instead on multisets:

Maybe this is about multisets. If we build a multiset with S, then the elements of _____ S are those with multiplicity bigger or equal to N. https://t.co/qs58gi0Urs
— Vitor Enes (@vitorenesduarte) November 10, 2018

Unlike a set, a multiset, (and quoting Wikipedia) “allows for multiple instances for each of its elements”.

In a multiset we keep count of the number of instances of each element - this number is called multiplicity. I’ll use $e_m$ to represent an element $e$ with multiplicity $m$.

So, with sets $A$, $B$ and $C$, we can build a multiset $\{a_3, b_2, c_1, d_1\}$.

Using these concepts, we can redefine $\bigcup_n$:

an element belongs to $\bigcup_n S$ if its multiplicity in the multiset is at least $n$

Part II

Today, I was back on my quest. This concept should be something classical, with like 100 years. I was convinced that with this new insight (the multiset), I would finally find the name of $\bigcup_n$.

What was I not expecting, was to find it in a recent (2005) multi-party computation (MPC) paper. The paper is called Privacy-Preserving Set Operations and is authored by Lea Kissner (@LeaKissner) and Dawn Song (@dawnsongtweets).

The high-level idea of MPC is to have a set of players computing some function on some inputs. The inputs are to be kept private, and only the function output should be revealed.

So, if the inputs are the sets $A, B, C$ in $S$, and we’re computing their intersection, then only $\bigcap S = \{a\}$ should be known in the end.

In Section 6.2, I found the following²:

We define the Threshold Set-Union problem as follows: all players learn which elements appear in the combined private input of the players at least a threshold number $t$ times. For example, assume that $a$ appears in the combined private input of the players $15$ times. If $t = 10$, then all players learn $a$. However, if $t = 16$, then no player learns $a$.

With this, I declare my quest over, and I’ll be calling $\bigcup_n$ threshold union.

What do you think? Is there a better name? Has this concept been named elsewhere?

$|S|$ represents the size of the set. In the example, $|S| = 3$. ^{^}
Although, there’s no mention of multiset in this definition, nor in the title, the paper contains several and I guess that’s why it came up in my Google search. ^{^}

Detecting message stability with partial memberships

Sat, 20 Oct 2018 12:35:00 +0100

Basic matrix-based technique.

Time-travel technique.

Borrowing an Identity for a Distributed Counter

Tue, 22 May 2018 00:00:00 +0100

Practical evaluation of the Lasp programming model at large scale: an experience report

Mon, 09 Oct 2017 00:00:00 +0100

Efficient Synchronization of State-based CRDTs

Tue, 03 Oct 2017 00:00:00 +0100

The Single-Writer Principle in CRDT Composition

Tue, 20 Jun 2017 00:00:00 +0100

Borrowing an Identity for a Distributed Counter: Work in progress report

Sun, 23 Apr 2017 00:00:00 +0100