OpenELM

What is OpenELM?

OpenELM is a collection of open-source large language models (LLMs) developed by Apple. These models are specifically designed to run on-device rather than relying on cloud servers. OpenELM utilizes a layer-wise scaling strategy to efficiently allocate parameters within each layer of the transformer model, resulting in enhanced accuracy. With a parameter budget of approximately one billion parameters, OpenELM demonstrates a 2.36% improvement in accuracy compared to other models while requiring 2 times fewer pre-training tokens.