The best articles of the week [28-03, Nov 2019]
The latest articles from the most prominent A.I scientists.
RESEARCH
- “Capacity, Bandwidth, and Compositionality in Emergent Language Learning”: Most of recent works have focused on communicative bandwidth to evaluate the compositionality. While important, it is not the only contributing factor. Our foremost contribution is to explore how capacity of a neural network impacts its ability to learn a compositional language. We empirically verfiy that learning the underlying compositional structure requires less capacity than memorizing a dataset. Link
- “The future of A.I.” by Neil D. Lawrence: The promise of AI is to launch the first systems to adapt to us. In reality, the systems we have will not achieve this, and it is the biological sciences that teach us this lesson most starkly. In this talk I will review some of the successes and challenges of AI and its deployment and propose practical visions for the future based on approaches that have worked in the past. Link
- “Recognizing long-form speech using streaming end-to-end models”: We examine the ability of E2E models to generalize to unseen domains, where we find that models trained on short utterances fail to generalize to long-form speech. We propose two complementary solutions to address this: training on diverse acoustic data, and LSTM state manipulation to simulate long-form audio when training using short utterances. Link
- “Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement”: beam search, a graph search algorithm allowing for a non-greedy local decisions that can potentially lead to a sequence with a higher overall probability, has been used for decoding neural sequence models. However, the algorithm suffers from large beam widths(number best states at each level), and it is deterministic, hence, return the same answer from every response. The work explore the posibility of stochastic beam search. Link
- “Consistency regularization for generative adversarial networks”: The basic idea is simple: an input image is perturbed in some semantics-preserving ways and the sensitivity of the classifier to that perturbation is penalized. The perturbation can take many forms: it can be image flipping, or cropping, or adversarial attacks. The method improves state-of-the-art FID scores for conditional generation from 14.73 to 11.67 on CIFAR-10 and from 8.73 to 6.66 on ImageNet-2012. Link
APPLIED SCIENCE
- “Unseen Food Creation by Mixing Existing Food Images with Conditional StyleGAN”: enabled us to control the food categories of generated images and to generate unseen foods by mixing multiple kinds of foods. Link
- “Transformer-based Acoustic Modeling for Hybrid Speech Recognition”: Facebook A.I. acheives new state-of-the-art results on speech recognition task using transformer-based model. Link
- “Learning to predict the cosmological structure formation”: build a deep neural network to predict structure formation of the Universe. It could extrapolates far beyond its training data, like teaching image recognition software with lots of pictures of cats and dogs, but then it’s able to recognize elephants. Link