From Silicon Valley to DC, the tech world is suddenly obsessed with one concept in AI: Distillation

Distillation allows for efficient model compression, making advanced AI more accessible and affordable. This has prompted a policy letter from researchers and industry leaders urging the US government to avoid restrictions on open-weight models, arguing that openness is essential for innovation.

According to the article, distillation involves training a 'student' model on the outputs of a 'teacher' model, often achieving comparable performance with significantly fewer resources. The technique is not new but has gained urgency as Chinese open-weight models, which frequently use distillation, approach US performance levels. The policy letter, signed by academics and executives, warns that restrictions could harm US competitiveness and global research collaboration.

The article notes that the debate over distillation reflects broader tensions between the desire for open science and the need for security. Proponents of openness argue that distillation democratizes AI, while critics express concerns about potential misuse.