Gemini 1.5 Pro Announced

  File: /n/kris/news/gemini-1.5-pro-announced
  Size: 255 words
Access: (0644/-rw-r--r--)  Uid: (1000/kris)  Gid: (100/users)
Modify: 2024-02-18
Status: growing  confidence: certain  importance: 8
  Tags: google gemini language-models

Google’s next-gen Gemini 1.5 Pro uses Mixture-of-Experts to match Ultra performance at lower compute, with a breakthrough 1M token context window.

Gemini 1.5 Pro released this week with the promise.

"The model delivers dramatically enhanced performance, with a breakthrough in long-context understanding across modalities."

It was designed to be a mid-size multi modal model that matches the performance of 1.0 Ultra (their largest model) while simultaneously managing to use less compute than the prized heifer. 1.5 uses a transformer, and mixture of experts architecture. MoE allows the model to be split into smaller "expert" narrow llms rather than the traditional monolith neural net. Meaning for any given input, only relevant expert pathways active, leading to more effective training and inference.

The defining feature of 1.5 Pro is still it's context window however.