Update README.md
Browse files
    	
        README.md
    CHANGED
    
    | @@ -1,90 +1,91 @@ | |
| 1 | 
            -
            ---
         | 
| 2 | 
            -
            license: apple-amlr
         | 
| 3 | 
            -
            language:
         | 
| 4 | 
            -
            - en
         | 
| 5 | 
            -
            metrics:
         | 
| 6 | 
            -
            - wer
         | 
| 7 | 
            -
            pipeline_tag: automatic-speech-recognition
         | 
| 8 | 
            -
            tags:
         | 
| 9 | 
            -
            - asr
         | 
| 10 | 
            -
            - mixture-of-experts
         | 
| 11 | 
            -
            - speech
         | 
| 12 | 
            -
            ---
         | 
| 13 | 
            -
            # Model Card for Model ID
         | 
| 14 | 
            -
             | 
| 15 | 
            -
            Omni-router Transformer is a new Mixture-of-Experts (MoE) architecture that explicitly couples routing across layers using a shared router to learn strong and specialized experts. Omni-router's routing decisions appear to form consistent temporal segments and strutured usage across model depth, suggesting meaningful coordination between layers.
         | 
| 16 | 
            -
             | 
| 17 | 
            -
             | 
| 18 | 
            -
             | 
| 19 | 
            -
             | 
| 20 | 
            -
             | 
| 21 | 
            -
             | 
| 22 | 
            -
             | 
| 23 | 
            -
             | 
| 24 | 
            -
             | 
| 25 | 
            -
            - ** | 
| 26 | 
            -
            - ** | 
| 27 | 
            -
            - ** | 
| 28 | 
            -
             | 
| 29 | 
            -
             | 
| 30 | 
            -
             | 
| 31 | 
            -
             | 
| 32 | 
            -
             | 
| 33 | 
            -
             | 
| 34 | 
            -
             | 
| 35 | 
            -
             | 
| 36 | 
            -
             | 
| 37 | 
            -
             | 
| 38 | 
            -
             | 
| 39 | 
            -
             | 
| 40 | 
            -
             | 
| 41 | 
            -
             | 
| 42 | 
            -
             | 
| 43 | 
            -
             | 
| 44 | 
            -
             | 
| 45 | 
            -
             | 
| 46 | 
            -
             | 
| 47 | 
            -
             | 
| 48 | 
            -
             | 
| 49 | 
            -
             | 
| 50 | 
            -
             | 
| 51 | 
            -
             | 
| 52 | 
            -
             | 
| 53 | 
            -
             | 
| 54 | 
            -
             | 
| 55 | 
            -
             | 
| 56 | 
            -
             | 
| 57 | 
            -
             | 
| 58 | 
            -
             | 
| 59 | 
            -
             | 
| 60 | 
            -
             | 
| 61 | 
            -
             | 
| 62 | 
            -
             | 
| 63 | 
            -
             | 
| 64 | 
            -
             | 
| 65 | 
            -
             | 
| 66 | 
            -
             | 
| 67 | 
            -
             | 
| 68 | 
            -
            |  | 
| 69 | 
            -
            | dev- | 
| 70 | 
            -
            |  | 
| 71 | 
            -
            | test- | 
| 72 | 
            -
             | 
| 73 | 
            -
             | 
| 74 | 
            -
             | 
| 75 | 
            -
             | 
| 76 | 
            -
             | 
| 77 | 
            -
             | 
| 78 | 
            -
             | 
| 79 | 
            -
             | 
| 80 | 
            -
             | 
| 81 | 
            -
             | 
| 82 | 
            -
               | 
| 83 | 
            -
               | 
| 84 | 
            -
               | 
| 85 | 
            -
            }
         | 
| 86 | 
            -
             | 
| 87 | 
            -
             | 
| 88 | 
            -
             | 
| 89 | 
            -
             | 
| 90 | 
            -
             | 
|  | 
|  | |
| 1 | 
            +
            ---
         | 
| 2 | 
            +
            license: apple-amlr
         | 
| 3 | 
            +
            language:
         | 
| 4 | 
            +
            - en
         | 
| 5 | 
            +
            metrics:
         | 
| 6 | 
            +
            - wer
         | 
| 7 | 
            +
            pipeline_tag: automatic-speech-recognition
         | 
| 8 | 
            +
            tags:
         | 
| 9 | 
            +
            - asr
         | 
| 10 | 
            +
            - mixture-of-experts
         | 
| 11 | 
            +
            - speech
         | 
| 12 | 
            +
            ---
         | 
| 13 | 
            +
            # Model Card for Model ID
         | 
| 14 | 
            +
             | 
| 15 | 
            +
            Omni-router Transformer is a new Mixture-of-Experts (MoE) architecture that explicitly couples routing across layers using a shared router to learn strong and specialized experts. Omni-router's routing decisions appear to form consistent temporal segments and strutured usage across model depth, suggesting meaningful coordination between layers.
         | 
| 16 | 
            +
            Please refer to the [paper](https://arxiv.org/abs/2507.05724) for details. 
         | 
| 17 | 
            +
             | 
| 18 | 
            +
             | 
| 19 | 
            +
            ## Model Details
         | 
| 20 | 
            +
             | 
| 21 | 
            +
            ### Model Description
         | 
| 22 | 
            +
             | 
| 23 | 
            +
            This model is a dense model (84M).
         | 
| 24 | 
            +
             | 
| 25 | 
            +
            - **Developed by:** Apple Machine Learning Research
         | 
| 26 | 
            +
            - **Model type:** ASR
         | 
| 27 | 
            +
            - **Language(s):** English
         | 
| 28 | 
            +
            - **License:** apple-amlr
         | 
| 29 | 
            +
             | 
| 30 | 
            +
            ## Uses
         | 
| 31 | 
            +
             | 
| 32 | 
            +
            This model is a speech recognition model.
         | 
| 33 | 
            +
             | 
| 34 | 
            +
            ## How to Get Started with the Model
         | 
| 35 | 
            +
             | 
| 36 | 
            +
            Please refer to the [github](https://github.com/apple/ml-omni-router-moe-asr) page for detailed usage.
         | 
| 37 | 
            +
             | 
| 38 | 
            +
            ## Training Details
         | 
| 39 | 
            +
             | 
| 40 | 
            +
            ### Training Data
         | 
| 41 | 
            +
             | 
| 42 | 
            +
            <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
         | 
| 43 | 
            +
             | 
| 44 | 
            +
            It is trained on the [Libriheavy](https://github.com/k2-fsa/libriheavy) dataset.
         | 
| 45 | 
            +
             | 
| 46 | 
            +
            ## Evaluation
         | 
| 47 | 
            +
             | 
| 48 | 
            +
            <!-- This section describes the evaluation protocols and provides the results. -->
         | 
| 49 | 
            +
             | 
| 50 | 
            +
            ### Testing Data, Factors & Metrics
         | 
| 51 | 
            +
             | 
| 52 | 
            +
            #### Testing Data
         | 
| 53 | 
            +
             | 
| 54 | 
            +
            <!-- This should link to a Dataset Card if possible. -->
         | 
| 55 | 
            +
             | 
| 56 | 
            +
            This model is evaluated on [Librispeech](https://www.openslr.org/12) dev/test sets.
         | 
| 57 | 
            +
             | 
| 58 | 
            +
            #### Metrics
         | 
| 59 | 
            +
             | 
| 60 | 
            +
            <!-- These are the evaluation metrics being used, ideally with a description of why. -->
         | 
| 61 | 
            +
             | 
| 62 | 
            +
            Word Error Rate (WER).
         | 
| 63 | 
            +
             | 
| 64 | 
            +
            ### Results
         | 
| 65 | 
            +
             | 
| 66 | 
            +
            | | Dense | Switch | Omni-router |
         | 
| 67 | 
            +
            |---|---|---|---|
         | 
| 68 | 
            +
            | | 84M | 8 x 84M | 8 x 84M | 
         | 
| 69 | 
            +
            | dev-clean | 2.1 | 1.9 | 1.8 |
         | 
| 70 | 
            +
            | dev-other | 6.7 | 6.1 | 5.4 |
         | 
| 71 | 
            +
            | test-clean | 2.3 | 2.2 | 2.0 |
         | 
| 72 | 
            +
            | test-other | 6.2 | 5.8 | 5.2 |
         | 
| 73 | 
            +
             | 
| 74 | 
            +
             | 
| 75 | 
            +
            ## Citation
         | 
| 76 | 
            +
             | 
| 77 | 
            +
            <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
         | 
| 78 | 
            +
             | 
| 79 | 
            +
            If you find this work useful, please cite our paper:
         | 
| 80 | 
            +
            ```
         | 
| 81 | 
            +
            @article{gu2025omnirouter,
         | 
| 82 | 
            +
              title={Omni-router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition},
         | 
| 83 | 
            +
              author={Gu, Zijin and Likhomanenko, Tatiana and Jaitly, Navdeep},
         | 
| 84 | 
            +
              journal={arXiv preprint arXiv:2507.05724},
         | 
| 85 | 
            +
              year={2025}
         | 
| 86 | 
            +
            }
         | 
| 87 | 
            +
            ```
         | 
| 88 | 
            +
             | 
| 89 | 
            +
            ## Model Card Contact
         | 
| 90 | 
            +
             | 
| 91 | 
            +
            Contact zijin@apple.com for any issues.
         |