New distillation method cuts memory and cost for large language models
Multiverse Computing researchers have presented a knowledge-distillation method designed to reduce the memory and computing requirements of compressing large language models. The approach separates teacher-model processing from student training by calculating the teacher’s 100 most likely tokens for each position once and storing them for reuse.