F21-STAT 940-Proposal
Use this format (Don’t remove Project 0)
Project # 0 Group members:
Abdelkareem, Youssef
Nasr, Islam
Huang, Xuanzhi
Title: Automatic Covid-19 Self-Test Supervision using Deep Learning
Description:
The current health regulations in Canada mandate that all travelers arriving who have to quarantine take a Covid-19 Self-test at home. But the process involves a human agent having a video call with you to walk you through the steps. The idea is to create a deep learning pipeline that takes a real-time video stream of the user and guides him through the process from start to end with no human interference. We would be the first to try to automate such a process using deep learning.
The steps of the test are as follows: - Pickup the swab - Place the swab in our nose up to a particular depth - Rotate in place for a certain period of time - Return back the swab to the required area - https://www.youtube.com/watch?v=jDIUFDMmBDo
Our pipeline will do the following steps: - Use a real-time face detector to detect bounding box around User's face (Such as SSD-MobileNetV2) - Use a real-time landmark detector to accurately detect the location of the nose. - Use a novel model to classify whether the swab is (Inside Left Nose, Inside right Node, Outside Nose) - Once the swab is detected to be inside the nose, the swab will have a marker (such as aruco marker) that can be automatically tracked in real-time using opencv, we will detect the location of the marker relative to the nose location and make sure it's correctly placed and rotated for the determined period. - This whole pipeline will be developed as a state machine, whenever the user violates the rule of a certain step he goes back to the corresponding step and repeats again.
Challenges: - There is no available dataset for training our novel model (to classify whether the swab is inside the nose or not), we will need to build our own dataset using both synthetic and real data. We will build a small dataset (5000-6000 images) and will rely on transfer learning to finetune an existing face attribute (such as moustache and glasses) classification model on our small dataset. - The whole pipeline will need to operate at a minimum of 30 FPS on CPU in order to match the FPS of a real-time video stream.
Project # 1 Group Members:
Feng, Jared
Salm, Veronica
Jones-McCormick, Taj
Sarangian, Varnan
Title: An Arsenal of Augmentation Strategies for Natural Language Processing
Description: Data augmentation is an effective technique for improving the generalization power of modern neural architectures, especially when trained over smaller datasets. Strategies for modifying training examples in domains such as computer vision and speech processing are typically straightforward, however it becomes more challenging in text processing where such generalized approaches are non-trivial. Unlike with images, where most geometric operations typically preserve the images' significant features, incorporating augmentation in text at the syntactic level can reduce the data quality as it may produce noisy examples that are no longer human-readable.
The purpose of this project is to investigate data augmentation techniques in NLP to accommodate for training robust neural networks over smaller datasets. Ideally, this may act as an alternative to relying on finetuning computationally demanding pretrained models. We will start with a survey of existing text augmentation techniques and evaluate them on several downstream tasks (using existing benchmark datasets and possibly novel ones). We will further attempt to adapt popular augmentation approaches from external domains (i.e. computer vision) into an NLP setting. This will include an empirical investigation into determining whether its appropriate to augment at the syntactic (i.e. raw text) or semantic (i.e. encoded vectors) level.