Frontier Models
Thinking Machines Lab Releases Inkling 975B Open-Weights Multimodal MoE
Ex-OpenAI CTO Mira Murati's lab introduces a customizable base model that prioritizes multimodal breadth and fine-tuning flexibility over benchmark dominance in frontier AI.
Inkling is a decoder-only multimodal Mixture-of-Experts transformer with 975B total parameters and 41B active parameters released by Thinking Machines Lab.
Ex-OpenAI CTO Mira Murati founded Thinking Machines Lab to develop open-weights models that emphasize customization and efficiency.
What background led to the development of Inkling?
The lab focuses on alternatives to closed frontier models by releasing base models suited for fine-tuning across diverse applications.
This strategy addresses developer needs for adaptable systems rather than fixed closed offerings.
What is the technical architecture of Inkling?
Inkling uses a 66-layer decoder-only transformer with a sparse Mixture-of-Experts feed-forward backbone.
The design includes 256 routed experts and 2 shared experts, activating 6 routed experts per token.
Images receive encoding through a hierarchical patch encoder while audio uses discrete token encoding.
Relative attention replaces RoPE and hybrid sliding-window plus global attention layers manage long sequences.
Controllable thinking effort lets users trade performance against token usage during inference.
How was Inkling pretrained and what context does it support?
Pretraining drew from 45 trillion tokens spanning text, images, audio, and video.
The context window extends to 1 million tokens for handling extended input sequences.
Native multimodal reasoning processes text, images, and audio inputs directly to produce text outputs.
What market implications arise from the Inkling release?
Open weights under Apache 2.0 on Hugging Face enable broad access for customization by researchers and enterprises.
Integration with the Tinker platform supports targeted fine-tuning workflows.
The approach offers developers an open alternative to closed models that restrict adaptation.
What expert reactions address the model's positioning?
The company positions Inkling as a practical base rather than a benchmark leader.
Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning.Thinking Machines Lab
What comes next for the Inkling model family?
Inkling serves as the initial release in a planned family of models.
A preview of the smaller Inkling-Small variant at 276B total and 12B active parameters has been shared.
| Variant | Total Parameters | Active Parameters | Context Window | License |
|---|---|---|---|---|
| Inkling | 975B | 41B | 1M tokens | Apache 2.0 |
| Inkling-Small | 276B | 12B | 1M tokens | Apache 2.0 |
- Access weights via the Hugging Face repository under Apache 2.0.
- Review the model card for architecture and encoding details.
- Deploy on Tinker for fine-tuning experiments.
- Adjust controllable thinking effort for efficiency tuning.
- Test native multimodal inputs including images and audio.
Frequently asked
What pretraining scale supports Inkling capabilities?
Inkling was pretrained on 45 trillion tokens of text, images, audio and video.
Where is the Inkling model available for use?
The model is released with open weights under Apache 2.0 on Hugging Face and supports fine-tuning on the Tinker platform.
How does Inkling handle multiple input types?
Inkling reasons natively over text, images, and audio inputs through dedicated encoders while outputting text.