Distilling Convolutional Neural Network Performance from Multiple Data Sources
If you're curious about how knowledge distillation can boost model performance using real and generated data, you're in the right place — this project is made for you.
We are a research team at UMass Boston. We are Can Do Crew Team. We are exploring how knowledge distillation helps student models perform better when trained on a mix of real images and AI-generated images. Our study focuses on 10 object classes and evaluates how training size, data sources, and augmentation impact model performance.
🟦 What’s This Project About?
Our goal is to investigate how model distillation can improve learning efficiency for image classification. In simple words, we train a student model to mimic the behavior of a powerful teacher model, hoping that the student can learn faster and perform better—even with less data.
This main idea is important when we want to:
- Reduce model size for deployment
- Train on limited or mixed-quality datasets
- Save computation but still keep high accuracy
🟦 Before We Started
We built a teacher model trained to classify 10 visual classes using CIFAR datasets. This model serves as the ground truth for guiding all student models. Please note: While we use the CIFAR dataset to train the teacher model, we intentionally avoid using CIFAR for the student models to prevent bias and ensure a more realistic evaluation.
We Asked Ourselves:
👉 How well can student models learn from the teacher under different dataset conditions?
