I don't understand what this synchronization is doing. We don't synchronize the onNext so why would synchronization here help ensure no race?

Shouldn't the caughtUp function be used to handle this use case on the first onNext to handle any data between onStart and onAdded?

It can't. A subscriber arrives before the onNext and thus not yet visible for the caughtUp, subscriber gets registered, and not they sit doing nothing in the original version. One has to do a post-subscription check to see if new value arrived since the pre-subscription. BehaviorSubject does this as well and it is a startup window.

A subscriber arrives before the onNext and thus not yet visible for the caughtUp

That would mean it doesn't receive the onNext and it should receive that value on the next event. That sounds pretty normal for a natural race condition like this. Why wouldn't the next onNext or terminal event take care of catching up?

And what is the synchronized doing? I don't see how it is ever synchronizing between threads since onAdded would only be invoked once and we don't synchronize inside onNext.

Now imagine that the Subject receives an onNext and onCompleted in quick succession; if the subscription is delayed between onStart and onAdd, the subscriber will never be notified as there can't be any further events to trigger caughtUp.

Synchronized resolves the race for the replayObserver: it protects the first indicator. If the subscribing thread gets in there first, it starts to replay existing and incoming events until it gets some "break" and the regular caughtUp can pick up. If the emitter thread gets in there first, it will behave as a regular caughtUp and the subscriber thread does nothing.

Ah, so it's replayObserver and onAdded that are being synchronized. That's the connection I wasn't making. Thanks.