This PR supports to apply delay penalty on k2 ctc loss, to reduce the symbol delay. The encoder is streaming conformer model.
We provide two methods of delay penalty here:
(1) Add penalty scores on encoder ctc outputs after log-softmax. Use param --nnet-delay-penalty in train.py.
(2) Add penalty scores on ctc decoding lattice. Use param --delay-penalty in train.py. It requires k2-fsa/k2#1086 in k2.

For method (1), we want to add lambda * torch.arange(T) to the ctc blank probs (after log-softmax), to encourage blanks to align to later and encourage non-blank symbols to align earlier. However, it would also encourage blank over non-blanks. This is not what we want. This effect will be stronger for longer utterances, where the penalty scores lambda * torch.arange(T) get too large. To minimize the above "repeat-penalizing effect", we first split utterances into shorter sub-utterances, for purpose of adding the "arange" expression. We can declare that there is a sub-utterance boundary every time the blank prob is >= 0.99, for instance. Specifically, we
(1) compute a mask that is 1 when blank_prob >= 0.99 (or t==0);
(2) compute cummax_out = torch.cummax(torch.arange(T) * mask)
(3) compute the sawtooth-like "blank-bonus" values: penalty = torch.arange(T) - cummax_out
(4) add penalty * lambda to ctc blank probs (after log-softmax).

For method (2), we add penalty scores (frame offset relative to the middle frame, scaled by lambda) on the non-repeat and non-blank arcs in ctc decoding lattice, which are at the frames when the non-blank symbols are firstly emitted. It is the same as we did in transducer loss (see https://arxiv.org/pdf/2211.00490.pdf).

• Note: The delay penalty is applied when warmup >= 1.0. Otherwise the loss would not converge.