Proposal
https://github.com/prometheus/prometheus/blob/main/util/gate/gate.go this is used in the remote read handler to limit number of concurrent requests. However, if we are close to that limit then currently it is impossible to know that. Ideally, we'd know if requests are being held up at the gate. I suggest adding a "waiting duration" histogram metric there so that users could understand whether they are hitting that limit.
Proposal
https://github.com/prometheus/prometheus/blob/main/util/gate/gate.go this is used in the remote read handler to limit number of concurrent requests. However, if we are close to that limit then currently it is impossible to know that. Ideally, we'd know if requests are being held up at the gate. I suggest adding a "waiting duration" histogram metric there so that users could understand whether they are hitting that limit.