Hi, @mathuvu
The Muon optimizer appears to demonstrate faster and more stable convergence compared to the AdamW optimizer.
Would you consider adding it to the list of supported optimizers?
You can find a detailed write-up here: https://kellerjordan.github.io/posts/muon/
And the implementation is available here: https://github.com/KellerJordan/Muon
Hi, @mathuvu
The Muon optimizer appears to demonstrate faster and more stable convergence compared to the AdamW optimizer.
Would you consider adding it to the list of supported optimizers?
You can find a detailed write-up here: https://kellerjordan.github.io/posts/muon/
And the implementation is available here: https://github.com/KellerJordan/Muon