F21-STAT 940-Proposal: Difference between revisions

From statwiki
Jump to navigation Jump to search
(Replaced content with "Use this format (Don’t remove Project 0) Project # 0 Group members: Last name, First name Last name, First name Last name, First name Last name, First name Title:...")
No edit summary
Line 3: Line 3:
Project # 0 Group members:
Project # 0 Group members:


Last name, First name
Abdelkareem, Youssef


Last name, First name
Nasr, Islam


Last name, First name
Huang, Xuanzhi


Last name, First name


Title: Making a String Telephone
Title: Automatic Covid-19 Self-Test Supervision using Deep Learning


Description: We use paper cups to make a string phone and talk with friends while learning about sound waves with this science project. (Explain your project in one or two paragraphs).
Description:  
 
 
The current health regulations in Canada mandate that all travelers arriving who have to quarantine take a Covid-19 Self-test at home. But the process involves a human agent having a video call with you to walk you through the steps. The idea is to create a deep learning pipeline that takes a real-time video stream of the user and guides him through the process from start to end with no human interference. We would be the first to try to automate such a process using deep learning.
 
'''The steps of the test are as follows:'''
  - Pickup the swab
  - Place the swab in our nose up to a particular depth
  - Rotate in place for a certain period of time
  - Return back the swab to the required area
  - https://www.youtube.com/watch?v=jDIUFDMmBDo
 
'''Our pipeline will do the following steps:'''
  - Use a real-time face detector to detect bounding box around User's face (Such as SSD-MobileNetV2)
  - Use a real-time landmark detector to accurately detect the location of the nose.
  - Use a novel model to classify whether the swab is (Inside Left Nose, Inside right Node, Outside Nose)
  - Once the swab is detected to be inside the nose, the swab will have a marker (such as aruco marker) that can be automatically tracked in real-time using opencv, we will detect the location of the marker relative to the nose location and make sure it's correctly placed and rotated for the determined period.
  - This whole pipeline will be developed as a state machine, whenever the user violates the rule of a certain step he goes back to the corresponding step and repeats again.
 
'''Challenges:'''
  - There is no available dataset for training our novel model (to classify whether the swab is inside the nose or not), we will need to build our own dataset using both synthetic and real data. We will build a small dataset (5000-6000 images) and will rely on transfer learning to finetune an existing face attribute (such as moustache and glasses) classification model on our small dataset.
  - The whole pipeline will need to operate at a minimum of 30 FPS on CPU in order to match the FPS of a real-time video stream.

Revision as of 13:19, 6 October 2021

Use this format (Don’t remove Project 0)

Project # 0 Group members:

Abdelkareem, Youssef

Nasr, Islam

Huang, Xuanzhi


Title: Automatic Covid-19 Self-Test Supervision using Deep Learning

Description:


The current health regulations in Canada mandate that all travelers arriving who have to quarantine take a Covid-19 Self-test at home. But the process involves a human agent having a video call with you to walk you through the steps. The idea is to create a deep learning pipeline that takes a real-time video stream of the user and guides him through the process from start to end with no human interference. We would be the first to try to automate such a process using deep learning. 

The steps of the test are as follows: 
 - Pickup the swab 
 - Place the swab in our nose up to a particular depth
 - Rotate in place for a certain period of time 
 - Return back the swab to the required area 
 - https://www.youtube.com/watch?v=jDIUFDMmBDo

Our pipeline will do the following steps: 
 - Use a real-time face detector to detect bounding box around User's face (Such as SSD-MobileNetV2) 
 - Use a real-time landmark detector to accurately detect the location of the nose. 
 - Use a novel model to classify whether the swab is (Inside Left Nose, Inside right Node, Outside Nose) 
 - Once the swab is detected to be inside the nose, the swab will have a marker (such as aruco marker) that can be automatically tracked in real-time using opencv, we will detect the location of the marker relative to the nose location and make sure it's correctly placed and rotated for the determined period. 
 - This whole pipeline will be developed as a state machine, whenever the user violates the rule of a certain step he goes back to the corresponding step and repeats again.

Challenges:
 - There is no available dataset for training our novel model (to classify whether the swab is inside the nose or not), we will need to build our own dataset using both synthetic and real data. We will build a small dataset (5000-6000 images) and will rely on transfer learning to finetune an existing face attribute (such as moustache and glasses) classification model on our small dataset.
 - The whole pipeline will need to operate at a minimum of 30 FPS on CPU in order to match the FPS of a real-time video stream.
Retrieved from "http://wiki.math.uwaterloo.ca/statwiki/index.php?title=F21-STAT_940-Proposal&oldid=49951"