Why using position_embedding_type="rope" doesn't have position_embedding.weights key in saved_model?
#9971
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Using rope as position_embedding_type and inspecting nemo converted model doesn't contain "embedding.position_embeddings.weight" key. But word embedding do have key "embedding.position_embeddings.weight" in nemo model.
Beta Was this translation helpful? Give feedback.
All reactions