Microsoft has released a new series of AI models called the Phi-3.5 series. These models are designed to handle a variety of tasks and have outperformed competitors like Google’s Gemini 1.5 Flash, Meta’s Llama 3.1, and OpenAI’s GPT-4o in several benchmarks. The Phi-3.5 series includes three models: Phi-3.5-MoE-instruct, Phi-3.5-mini-instruct, and Phi-3.5-vision-instruct.
The Phi-3.5-MoE-instruct is the largest model with 41.9 billion parameters, of which 6.6 billion are active during generation. It was trained on 4.9 trillion tokens over 23 days using 512 H100-80G GPUs. This model excels at complex reasoning tasks across multiple languages.
The Phi-3.5-mini-instruct is a lightweight model with 3.8 billion parameters.
Microsoft’s Phi-3.5 series details
It supports a 128K token context length, making it suitable for tasks like document summarization and information retrieval.
This model was trained on 3.4 trillion tokens over 10 days using 512 H100-80G GPUs. Lastly, the Phi-3.5-vision-instruct model is designed for visual tasks. It has 4.15 billion parameters and combines an image encoder, connector, projector, and the Phi-3 Mini language model.
This model was trained on 500 billion tokens of visual and textual data over 6 days using 256 A100-80G GPUs. All three models are available on the AI platform Hugging Face under an MIT license, reflecting Microsoft’s commitment to open-source AI development. The release of the Phi-3.5 series marks a significant milestone for Microsoft as it aims to set new standards in the AI industry with cutting-edge models that cater to diverse applications.
