Thinking Machines Lab Drops Its First Model - WIRED

Thinking Machines Lab, as reported by WIRED, has released its first open-weight model, Inkling. The model features 975 billion parameters and was trained from scratch on a multimodal dataset composed of audio, video, and text. Additionally, the model was used to fine-tune itself, indicating a recursive self-improvement capability. The exact training details and computational cost have not been disclosed, but the model's scale places it among the largest open-weight models released to date.

Open-weight models are those whose trained parameters are made publicly available, enabling researchers and developers to download, inspect, and modify them. This approach fosters transparency and collaboration, unlike closed-source models that are only accessible via API. The self-fine-tuning process is a technique where the model uses its own outputs as additional training data to adjust its weights, potentially improving performance without external human input. This technique has been explored in smaller contexts but is now demonstrated at a massive scale. The open-weight release allows the community to study the model's inner workings and evaluate its safety properties, which is crucial for understanding the model.

The release of Inkling marks a significant milestone for Thinking Machines Lab, establishing it as a player in the large-scale AI model landscape. The combination of multimodality, scale, and self-fine-tuning renders Inkling a notable technical achievement. However, the open-weight nature also raises concerns about potential misuse, as the model's weights could be used for harmful purposes without oversight, which is a major concern. The article reports that the release has generated both excitement and caution among AI researchers. Thinking Machines Lab has indicated that they will continue to develop new models and publish findings. The article concludes by noting that the field will be watching Thinking Machines Lab's next moves closely.