What's the behavior with respect to the label? I assume the label is used as a fallback if the window title is not set?

If that's correct, the behavoir should be documented, with the comment that it results in backward compatibility for previous users of set_label, but set_window_title is preferred over set_label.

What's the behavior with respect to the label? I assume the label is used as a fallback if the window title is not set?

Unfortunately it is not that simple. If the window title is not set then the default such as "Figure 1" is used instead from here:

This can be overridden by passing a label to the Figure constructor, but any subsequent use of Figure.set_label does not update the title. This behaviour is surprising but it is, for good or bad, the same as for other backends such as qtagg.

To demonstrate this use

import matplotlib as mpl
mpl.use("qtagg")
#mpl.use('webagg')
import matplotlib.pyplot as plt

# 1. Set title using set_window_title
fig1, ax1 = plt.subplots(figsize=(3, 2))
fig1.canvas.manager.set_window_title('Using set_window_title')

# 2. Set title using label passed to Figure constructor
fig2, ax2 = plt.subplots(figsize=(3, 2), label="Label in constructor")

# 3. Using Figure.set_label has no effect on the window title
fig3, ax3 = plt.subplots(figsize=(3, 2))
fig3.set_label("This is not used in the title")

plt.show()

and the title outputs are the same for qtagg and webagg here:

So this PR brings webagg in line with other backends. Whether Figure.set_label should also update the title (for all backends) would I think be a separate issue for discussion.