STAT946F17/ Teaching Machines to Describe Images via Natural Language Feedback

From statwiki
Revision as of 18:10, 1 November 2017 by Hamidreza (talk | contribs) (Introduction)
Jump to: navigation, search


In the era of Artificial Intelligence, one should ideally be able to educate the robot about its mistakes, possibly without needing to dig into the underlying software. Reinforcement learning has become a standard way of training artificial agents that interact with an environment. Several works explored the idea of incorporating humans in the learning process, in order to help the reinforcement learning agent to learn faster. In most cases, the guidance comes in the form of a simple numerical (or “good”/“bad”) reward. In this work, natural language is used as a way to guide an RL agent. The author argues that a sentence provides a much stronger learning signal than a numeric reward in that we can easily point to where the mistakes occur and suggest how to correct them.

Here the goal is to allow a non-expert human teacher to give feedback to an RL agent in the form of natural language, just as one would to a learning child. The author has focused on the problem of image captioning in which the quality of the output can easily be judged by non-experts.

Related Works


Phrase-based Image Captioning

Crowd-sourcing Human Feedback

Feedback Network

Policy Gradient Optimization using Natural Language Feedback

Experimental Results