Optimizing Cache Aware Conformer w/ Riva Configs #5284
Unanswered
piraka9011
asked this question in
Q&A
Replies: 1 comment 1 reply
-
Riva does not support chunk aware Conformer as of now. It is planned to add support for it in the coming months. Since this is a very different model and way to do inference it will take a few months or so to get it into Riva. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
First of all, appreciate all the work being put into the new Conformer models. Lots of exciting progress!
We're fine-tuning new conformer models on our dataset and were wondering whether the parameters we specify during training should reflect the expected
riva-build
configuration?If they should, consider the default streaming conformer config provide here
n_layers
: 18downsampling
: 4xwindow_stride
(window_shift
, ms): 10Would the corresponding Riva build config be:
chunk_size
(ms): 40padding_size
(ms, left and right context):n_layers
* 2 *downsampling
*window_stride
= 1440Unsure if we should multiply the above equation by 2 since the docs say there's "2 right context in each layer" but not sure if that's referring to the left/right context or that you feed 2 contexts/chunks to the model each time.
Does the NeMo version also affect which Riva version we should use btw for conformers, since it seems FP16 support is unstable/sometimes results in NaN loss?
Beta Was this translation helpful? Give feedback.
All reactions