Index

Algorithm

• A Few Useful Things to Know About Machine Learning - Domingos CACM '12 [Summary]

• Hidden Technical Debt in Machine Learning Systems - Sculley et al, NIPS '15 [Summary]

  • Identified the technical debt of Machine Learning system and provided some common mitigation strategy.

• Don't Decay The Learning Rate, Increase The Batch Size* - Smith et al, ICLR '18

  • Showed that we can increase the batch size, instead of decreasing the learning rate, to get equivalent test accuracies after the same number of training epochs, but with fewer parameter updates, leading to greater parallelism and shorter training times.

• Accelerating Deep Learning by Focusing on the Biggest Losers - Jiang et al., arXiv '19 [Summary]

  • Proposes an algorithm that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration.

Faster Inference

• End-to-end Learning of Action Detection from Frame Glimpses in Videos - Yeung et al., CVPR '16

• Watching a Small Portion could be as Good as Watching All: Towards Efficient Video Classification - Fan et al., IJCAI '18

• AdaFrame: Adaptive Frame Selection for Fast Video Recognition - Wu et al., CVPR '19

• Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition - Wu et al., ICCV '19

• Listen to Look: Action Recognition by Previewing Audio - Gao et al., CVPR '20

• FastBERT: a Self-distilling BERT with Adaptive Inference Time - Liu et al., ACL '20

Misc.

• Beyond Accuracy: Behavioral Testing of NLP Models with CheckList - Ribeiro et al., ACL '20