
IIT BHU
ResearchSummer Research Intern
- Distilled a 5-model teacher ensemble into LightDep, a small two-stream (face + voice) transformer. The INT8 ONNX build is 3.96 MB, 28× smaller than its teacher, and runs in 96 ms on one CPU thread. The same file also runs in a browser tab and on phones.
- Main finding: distillation only helps when the teacher is a stronger, well-calibrated screener. On D-Vlog it lifted recall by +10.4 (p = 0.045) and F1 by +4.3. On LMVD, where the teachers were poorly calibrated, it gave nothing significant.
- Reran it over 10 seeds on a second GPU and the gain held on all four metrics (p ≤ 0.003 after Holm correction).
- While reproducing the MMFformer teacher I found its weight_decay default was 1e-3, not the paper's 0.1.
- Paper: “LightDep: Knowledge-Distilled Multimodal Depression Screening for Mobile and Edge Devices”, accepted at IEEE ANTS 2026.
May–July 2026
Varanasi, India

