Compression
8 concepts, 8 reviewed and merged.
Evaluation
Foundations
Inference
Methods
- Knowledge DistillationTrain a small model to copy a large one's outputs rather than the original labels.verified
- Low-Rank FactorizationReplace a big weight matrix with the product of two thin ones.verified
- PruningMost weights in a trained network can be set to zero without hurting it.verified
- Structured SparsityZeros only help if the hardware can skip them.verified