F18-STAT946-Proposal

Project # 0 Group members:

Last name, First name

Last name, First name

Last name, First name

Last name, First name

Title: Making a String Telephone

Description: We use paper cups to make a string phone and talk with friends while learning about sound waves with this science project. (Explain your project in one or two paragraphs).

---

Project # 1 Group members:

Zhang, Xinyue

Zhang, Junyi

Chen, Shala

Title: Airbus Ship Detection Challenge

Description: The idea and data for this project is taken from https://www.kaggle.com/c/airbus-ship-detection#description. The goal for this project is to build a model that detects all ships in satellite images and put an aligned bounding box segment around the ships we locate. We are going to extract the segmentation map for the ships first, augment the images and train a simple CNN model to detect them.

---

Project # 2 Group members:

Nekoei, Hadi

Afify, Ahmed

Carrillo, Juan

Ganapathi Subramanian, Sriram

Title: Algorithmic Analysis and Improvements in Multi-Agent Reinforcement Learning in Partially Observable Settings

Description: Reinforcement learning (RL) is a branch of Machine Learning in which an agent learns to act optimally in an environment using weak reward signals, which is different from strong labels in supervised learning. Multi-Agent Reinforcement Learning (MARL) is composed of multiple agents that can be competing against each other or cooperating together to achieve a common goal.

Our project aims to investigate the performance of several state of the art Multi-Agent Reinforcement Learning (MARL) algorithms in playing the game of Pommerman. This game will be used as a benchmark during a competition that will be held in NIPS 2018 (https://www.pommerman.com). We plan to participate and compare the performance of our agents against agents created by other researchers. Our project also aims to make algorithmic improvements to the state of the art MARL algorithms and come up with a new algorithm that renders best performance in this partially observable multi-agent setting of Pommerman.

In Pommerman, we have two competing teams, each has two agents who work together to defeat the opponent team. The agents move inside the board leaving bombs that can eliminate other agents when exploding in their horizontal or vertical vicinity. The agents can obtain bonuses such as extra bombs, increased bomb range, or ability to kick installed bombs. Our two agents can choose one of the following actions: stop, move up, move left, move down, move right, or lay a bomb. Each agent will receive an 11x11 grid of integer values representing the board state. Additional information will be provided to the agents such as its own position, positions of his teammate and enemies, available bombs, blast strength, kicking ability, and surrounding walls and bombs.

The algorithms that we are considering are:

- Monte Carlo Tree Search and Reinforcement Learning: Combining MCTS with deep neural networks.

- Multi-Agent Deep Deterministic Policy Gradient (DDPG): A technique developed by OpenAI, based on the Deep Deterministic Policy Gradient technique that outperforms traditional Reinforcement Learning algorithms (DQN/DDPG/TRPO) in several environments.

- Opponent Modelling in Deep Reinforcement Learning: based on DQN to model opponents through a Deep Reinforcement Opponent Network (DRON).

We will use Convolutional Neural Networks for data pre-processing, where we extract features from inputs. We will also be using Feed Forward Deep Networks along with Reinforcement learning frameworks in all the algorithms we implement (Deep Reinforcement learning).

---

Project # 3 Group members:

Fisher, Wesley

Pafla, Marvin

Rajendran, Vidyasagar

Title: Deep Reinforcement Learning for Angry Birds

Description: According to Artificial Intelligence (AI) researchers, AI’s performance in the game Angry Birds will exceed human performance in the next 3-4 years [1]. We propose a final project that will hopefully bring us closer to this goal by developing an AI model based on deep reinforcement learning to play the game Angry Birds. While AI has been applied to Angry Birds in the past, there are only a few approaches that utilize deep learning such as in [3]. We plan to implement Yuan et al.’s recommendations by creating an Angry Birds reinforcement learning model with more learning dimensions [3]. To further add novelty to our research, we want to explore the potential of extending our model with evolutionary algorithms [2]. To realize this project, we plan to use an existing implementation of Angry Birds (either https://github.com/estevaofon/angry-birds-python or the one provided for the Angry Birds AI competition which can be found at https://aibirds.org).

References:

[1] Grace, K., Salvatier, J., Dafoe, A., Zhang, B., & Evans, O. (2017). When will AI exceed human performance? Evidence from AI experts. arXiv preprint arXiv:1705.08807.

[2] Risi, S., & Togelius, J. (2017). Neuroevolution in games: State of the art and open challenges. IEEE Transactions on Computational Intelligence and AI in Games, 9(1), 25-41.

[3] Yuan, Y., Chen, Z., Wu, P., & Chang, L. Enhancing Deep Reinforcement Learning Agent for Angry Birds. https://aibirds.org/2017/aibirds_BNU.pdf

---

Project # 4 Group members:

Heydari, Nargess

Manuel, Jacob

Ravi, Aravind

Title: Deep Learning for Detection of Steady State Visually Evoked Potentials

Description: Brain Computer Interfaces (BCIs) enable users to control an external device by modulating their neuronal activity. Steady state visual evoked potential (SSVEP) based BCIs are of particular interest due to their high information transfer rate (ITR) and relatively low amount of training required for use. SSVEP responses are elicited when a user focuses on a flickering light source and are observed prominently in the occipitoparietal area of the cortex. These responses manifest as an increase in amplitude of the frequency components of the EEG signal at the stimulus frequency and harmonic frequencies. Therefore, by analyzing the frequency component dominant in the EEG signals recorded from occipitoparietal area, the stimulus with user’s visual engagement can be identified.The goal of this project is to identify and compare deep learning architectures for classifying SSVEP responses to use in BCIs. Different architectures will be compared with state of the art classification methods (e.g. Canonical Correlation Analysis) through a sensitivity analysis of their accuracy across multiple BCI variables (e.g. analysis window size, subject variability, size of training data, etc.). The goal of this comparison is to establish a new system design to support application of Deep Neural Networks in SSVEP-based BCI. The proposed study will be performed on the SSVEP dataset collected by the eBionics Lab at the University of Waterloo

References

N. S. Kwak, K. R. M ̈uller, and S. W. Lee, “A convolutional neural network for steady state visual evoked potential classification under ambulatory environment,” PLoS One, 2017.

---

---

Project # 5 Group members:

Khan, Salman

Naik, Abdul

Koundinya, Shubham

Title: Deep Learning for Image Captioning

Description: Image captioning is the automatic generation of textual descriptions from images. It involves identifying the contents of an image, understanding relationships between what has been detected and generating textual descriptions.

It is a challenging task as it includes both Computer Vision and Natural Language Processing components. Furthermore, an image can be described by multiple text statements. We will explore various state-of-the-art translation models focussing primarily on different ways of describing an image.

References

StyleNet: Generating Attractive Visual Captions with Styles.

---

Project # 5 Group members:

Amirpasha Ghabussi

Kumar, Dhruv

Sahu, Gaurav

Khan, Kashif

Title: Deep learning model for Question Answering & Machine Comprehension

Description: Question answering is a computer science discipline within the fields of information retrieval and natural language processing, which is concerned with building systems that automatically answer questions posed by humans in a natural language.

We will try to improve the accuracy of the models that have shown promising results in most of the highly active datasets SQuAD or MS-Marco.

References

Bidaf: https://arxiv.org/abs/1611.01603 QANet: https://arxiv.org/abs/1804.09541

---