Search results

Jump to navigation Jump to search

Page title matches

Page text matches

  • MULTIPLE OBJECT RECOGNITION WITH VISUAL ATTENTION
    #REDIRECT [[mULTIPLE OBJECT RECOGNITION WITH VISUAL ATTENTION]] ...
    63 bytes (7 words) - 09:46, 30 August 2017
  • Summary for Saliency-based Sequential Image Attention with Multiset Prediction
    #REDIRECT [[stat441w18/Saliency-based Sequential Image Attention with Multiset Prediction]] ...
    91 bytes (10 words) - 12:47, 15 March 2018
  • Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
    ...IRECT [[show, Attend and Tell: Neural Image Caption Generation with Visual Attention]] ...
    90 bytes (12 words) - 09:46, 30 August 2017
  • learning2reasoning
    ...ence and machine learning and machine reasoning have received considerable attention given the short history of computer science. The statistical nature of mach ...
    852 bytes (116 words) - 09:46, 30 August 2017
  • stat940F21
    ...Benyamin Jamialahmad || || Perceiver: General Perception with Iterative Attention || [https://arxiv.org/abs/2103.03206] ||[https://www.youtube.com/watch?v=N |Week of Nov 25 || Mina Kebriaee || || Synthesizer: Rethinking Self-Attention for Transformer Models ||[https://arxiv.org/pdf/2005.00743.pdf] || [https:/ ...
    5 KB (642 words) - 23:29, 1 December 2021
  • show, Attend and Tell: Neural Image Caption Generation with Visual Attention
    ...v2.pdf "Show, attend and tell: Neural image caption generation with visual attention."] arXiv preprint arXiv:1502.03044 (2015). </ref> introduces an attention based model that automatically learns to describe the content of images. It ...
    12 KB (1,882 words) - 09:46, 30 August 2017
  • mULTIPLE OBJECT RECOGNITION WITH VISUAL ATTENTION
    ...pecific manner for determining possible object locations. In this paper an attention-based model for recognizing multiple objects in images is presented. The pr = Deep Recurrent Visual Attention Model:= ...
    11 KB (1,714 words) - 09:46, 30 August 2017
  • Do Vision Transformers See Like CNN
    ...LSTMs, …) were experiencing at the time by introducing the concept of self-attention. ...the sentence as keys. Self-attention effectively tells the model how much attention the query should give to other words in the sentence. ...
    13 KB (2,006 words) - 00:11, 17 November 2021
  • STAT946F20/BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
    ...ion and the sequential mask in the decoder and usually performs Multi-head attention to derive more features from the different subspace of sentence for the ind ...ture and the only difference is that <math>BERT_{BASE}</math> makes use of attention masks and gets and improvement of 4.5%. It can also be seen that <math>BERT ...
    9 KB (1,342 words) - 06:36, 10 December 2020
  • Hierarchical Question-Image Co-Attention for Visual Question Answering
    ...ur model reasons about the question (and consequently the image via the co-attention mechanism) in a hierarchical fashion via a novel 1-dimensional convolution Recently, ''visual-attention'' based models have gained traction for VQA tasks, where the ...
    27 KB (4,375 words) - 19:50, 28 November 2017
  • One-Shot Imitation Learning
    ...tance of the same task. Strong generalization was achieved by using a soft attention mechanism on both the sequence of actions and states that the demonstration * Attention Modelling: ...
    20 KB (3,247 words) - 00:27, 21 April 2018
  • f15Stat946PaperSignUp
    ...per]||[[Show, Attend and Tell: Neural Image Caption Generation with Visual Attention|Summary]] ....org/pdf/1412.7755v2.pdf Paper]||[[MULTIPLE OBJECT RECOGNITION WITH VISUAL ATTENTION | Summary]] ...
    11 KB (1,453 words) - 13:01, 16 October 2018
  • Attend and Predict: Understanding Gene Regulation by Selective Attention on Chromatin
    ...1708.00339 "Attend and Predict: Understanding Gene Regulation by Selective Attention on Chromatin."] by Singh, Ritambhara, et al. It was published at the Advanc ...o attend to the important marks and their interactions. In this context, ''attention'' refers to weighing the importance of different items differently. ...
    33 KB (4,924 words) - 20:52, 10 December 2018
  • DCN plus: Mixed Objective And Deep Residual Coattention for Question Answering
    ...g: Models like coattention, bidirectional attention flow and self-matching attention build codependent representations of the question and the document. After b ...odels Intuition 2]. Coattention, bidirectional attention and self-matching attention are some of the methods that build codependent representation between the q ...
    24 KB (3,769 words) - 17:49, 14 December 2018
  • Convolutional Sequence to Sequence Learning
    ...ast models employ RNNs for this problem, with bidirectional RNNs with soft attention being the dominant approach. ...ases gradient propagation and equipping each decoder layer with a separate attention module adds a negligible amount of overhead. ...
    27 KB (4,178 words) - 20:37, 28 November 2017
  • ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
    .... The authors set the feed-forward/filter size to be 4*H and the number of attention heads to be H/64 (where H is the size of the hidden layer). Next, we explai ...example, one may only share feed-forward network parameters or only share attention parameters. However, the default choice for ALBERT is to simply share all p ...
    14 KB (2,170 words) - 21:39, 9 December 2020
  • stat441w18
    || 9|| Saliency-based Sequential Image Attention with Multiset Prediction ...
    5 KB (694 words) - 18:02, 31 August 2018
  • One-Shot Object Detection with Co-Attention and Co-Excitation
    ...t image based on the query image. This operation can be thought of as a co-attention mechanism. The second contribution is proposing a Squeeze and Co-Excitation ...e. The same can be observed for the query image. This weighted sum is a co-attention mechanism and with the help of extended feature maps, better proposals are ...
    22 KB (3,609 words) - 21:53, 6 December 2020
  • This Looks Like That: Deep Learning for Interpretable Image Recognition
    ...ing classification but are rather created after this phase. There are also attention-based models that determine parts of the input they are looking at but with ...s as well as the car models are compared to the baseline models as well as attention-based deep models that were trained on the same datasets that ProtoPNet was ...
    10 KB (1,573 words) - 23:36, 9 December 2020
  • f17Stat946PaperSignUp
    |Nov 28 || Shivam Kalra ||29 || Hierarchical Question-Image Co-Attention for Visual Question Answering || [https://arxiv.org/pdf/1606.00061.pdf Pape ...
    10 KB (1,213 words) - 19:28, 19 November 2020
  • Neural Speed Reading via Skim-RNN
    ...achieves higher accuracy compared to skipping tokens, implying that paying attention to unimportant tokens is better than completely ignoring them. As the popularity of neural networks has grown, significant attention has been given to make them faster and lighter. In particular, relevant wor ...
    27 KB (4,321 words) - 05:09, 16 December 2020
  • Roberta
    ...d fine-tuning approach. Very briefly, the transformer architecture defines attention over the embeddings in a layer such that the feedforward weights are a func ...it, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. ...
    14 KB (2,156 words) - 00:54, 13 December 2020
  • Semantic Relation Classification——via Convolution Neural Network
    ...STM, and CNN models with various variations applied, such as two models of attention, negative sampling, entity embedding or sentence-only embedding, etc. ..., without attention, has significantly better performance than all others. Attention-based pooling, up-sampling, and data augmentation are also tested, but they ...
    15 KB (2,408 words) - 21:25, 5 December 2020
  • Pre-Training Tasks For Embedding-Based Large-Scale Retrieval
    ...An alternate option is BERT [3] or transformer-based models [4] with cross attention between query and passage pairs which can be optimized for a specific task. ...and <math>d</math>, <math> \theta </math> are the parameters of the cross-attention model. The architectures of these two models can be seen below in figure 1. ...
    22 KB (3,409 words) - 22:17, 12 December 2020
  • stat946F18
    ...c/paper/7255-attend-and-predict-understanding-gene-regulation-by-selective-attention-on-chromatin.pdf Paper] || [https://wiki.math.uwaterloo.ca/statwiki/index ...
    14 KB (1,851 words) - 03:22, 2 December 2018
  • Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
    ...nterest for this problem. Structured prediction has attracted considerable attention because it applies to many learning problems and poses unique theoretical a ...alterations to the functions <math>\alpha</math> and <math>\phi</math>. In attention each node aggregates features of neighbors through a function of neighbor's ...
    29 KB (4,603 words) - 21:21, 6 December 2018
  • STAT946F17/ Teaching Machines to Describe Images via Natural Language Feedback
    ...l which is similar to most image captioning models except that it exploits attention and linguistic information. Several recent approaches trained the captionin ...the authors have reasoned out about the type of phrases and exploited the attention mechanism over the image. The model receives an image as input and outputs ...
    23 KB (3,760 words) - 10:33, 4 December 2017
  • F18-STAT946-Proposal
    '''Title:''' Bi-Directional Attention Flow for Question Answering [1] Bi-Directional Attention Flow For Machine Comprehension - https://arxiv.org/abs/1611.01603 ...
    17 KB (2,400 words) - 15:50, 14 December 2018
  • DETECTING STATISTICAL INTERACTIONS FROM NEURAL NETWORK WEIGHTS
    ...puting input gradients [13] and decomposing predictions [8], 2) developing attention-based models, which illustrate where neural networks focus during inference ...: Bahdanau et al. (2014) - These are a different class of models which use attention modules(different architectures) to help focus the neural network to decide ...
    21 KB (3,121 words) - 01:08, 14 December 2018
  • F21-STAT 940-Proposal
    ...ese advanced methods in field of vision. This paper claims that neither of attention, and convolutions are necessary which the claim is proved by its well-stabl ...
    13 KB (2,036 words) - 12:50, 16 December 2021
  • Unsupervised Machine Translation Using Monolingual Corpora Only
    The model uses a sequence to sequence model with attention, without input-feeding. Both the encoder and decoder are 3 layer LSTMs, and ...
    8 KB (1,359 words) - 22:48, 19 November 2018
  • Mask RCNN
    ...t would be interesting to use this backbone with Mask R-CNN and see if the attention helps capture longer range dependencies and thus produce better segmentatio ...
    20 KB (3,056 words) - 22:37, 7 December 2020
  • a Dynamic Bayesian Network Click Model for Web Search Ranking
    ...ecreases towards the lower ranks on result pages due to the reduced visual attention from the user. A more recent click model, referred to as the ''cascade mode ...
    11 KB (1,852 words) - 09:45, 30 August 2017
  • Point-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence
    ...er also described related work in Personalized location recommendation and attention mechanism in the recommendation. The recent studies on location recommendat ...nd <math>W_a</math> and <math>w_t</math> are the learned parameters in the attention layer and aggregation layer. ...
    17 KB (2,662 words) - 05:15, 16 December 2020
  • stat946w18/Synthetic and natural noise both break neural machine translation
    ...The sequence of hidden units is then processed by the decoder, a GRU with attention, to produce probabilities over sequences of output characters. ...symbol list of the desired size, we apply a standard encoder-decoder with attention. ...
    17 KB (2,634 words) - 00:15, 21 April 2018
  • Dialog-based Language Learning
    ...n the figure, the memory is read twice, which is termed multiple “hops” of attention. ...e controller, where <math> R_1</math> is a $d$ × $d$ rotation matrix . The attention over the memory can then be repeated using <math> u_1</math> instead of $q$ ...
    26 KB (4,081 words) - 13:59, 21 November 2021
  • U-Time:A Fully Convolutional Network for Time Series Segmentation Applied to Sleep Staging Summary
    Attention is all you need. ''CoRR'', abs/1706.03762. ...
    8 KB (1,170 words) - 01:41, 26 November 2021
  • strategies for Training Large Scale Neural Network Language Models
    ...rrent Neural Network and Maximum Entropy-based models have gained a lot of attention and are considered the most successful models. However, the main drawback o ...
    9 KB (1,542 words) - 09:46, 30 August 2017
  • STAT946F17/Decoding with Value Networks for Neural Machine Translation
    ...e fact that better translation is generated when using more context in the attention mechanism. ...ighted contextual information summarizing the source sentence x using some attention mechanism. ...
    22 KB (3,543 words) - 00:09, 3 December 2017
  • from Machine Learning to Machine Reasoning
    ...quently, machine learning and machine reasoning have received considerable attention given the short history of computer science. The statistical nature of mach Little attention has been paid to the rules that describe how to assemble trainable models t ...
    21 KB (3,225 words) - 09:46, 30 August 2017
  • STAT946F17/Cognitive Psychology For Deep Neural Networks: A Shape Bias Case Study
    MNs are trained to assign label $\hat{y}$ to probe image $\hat{x}$ using an attention mechanism $a$ acting on image embeddings stored in the support set S: ...as an input sequence. The embedding $f(\hat{x}, S)$ is an LSTM with a read-attention mechanism operating over the entire embedded support set. The input to the ...
    22 KB (3,531 words) - 20:30, 28 November 2017
  • stat946w18/Unsupervised Machine Translation Using Monolingual Corpora Only
    The decoder makes use of the attention mechanism of Bahdanau et al. (2014). To compute the probability of a given with an attention mechanism. Then without any constraint, the auto-encoder tempts to merely c ...
    28 KB (4,522 words) - 21:29, 20 April 2018
  • Poison Frogs Neural Networks
    ...e to be misclassified as a base. Finally the paper hopes that it can raise attention for the important issues of data reliability and data sourcing. ...
    11 KB (1,590 words) - 18:29, 26 November 2021
  • Augmix: New Data Augmentation method to increase the robustness of the algorithm
    ...d be really curious to test their methodology in Large Models, adding self-attention layers to models improves robustness. To test the abstraction properties, w ...
    11 KB (1,652 words) - 18:44, 6 December 2020
  • Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
    ...s in a learned metric space using e.g. Siamese networks or recurrence with attention mechanisms”, the proposed method can be generalized to any other problems i ...xperimented on by the authors) would be to explore methods of attaching an Attention Kernel which results in a simple and differentiable loss. It has been imple ...
    26 KB (4,205 words) - 10:18, 4 December 2017
  • stat441w18/Saliency-based Sequential Image Attention with Multiset Prediction
    {{DISPLAYTITLE:stat441w18/Saliency-based Sequential Image Attention with Multiset Prediction}} ...
    12 KB (1,840 words) - 14:09, 20 March 2018
  • Unsupervised Neural Machine Translation
    The translation model uses a standard encoder-decoder model with attention. The encoder is a 2-layer bidirectional RNN, and the decoder is a 2 layer R ...ervised model to perform translations with monolingual corpora by using an attention-based encoder-decoder system and training using denoise and back-translatio ...
    28 KB (4,293 words) - 00:28, 17 December 2018
  • stat946F18/differentiableplasticity
    ..., the RNN is attached with an external content-addressable memory bank. An attention mechanism within the controller network does the read-write to the memory b ...memories must be embedded in fixed-size vectors and retrieved through some attention mechanism. In contrast, trainable synaptic plasticity translates into very ...
    27 KB (4,100 words) - 18:28, 16 December 2018
  • neural Machine Translation: Jointly Learning to Align and Translate
    ...t annotations to generate each target word; this implements a mechanism of attention in the decoder. ...
    14 KB (2,221 words) - 09:46, 30 August 2017
  • on using very large target vocabulary for neural machine translation
    ...ased on specific implementation of neural machine translation that uses an attention mechanism, as recently proposed in <ref> ...
    14 KB (2,301 words) - 09:46, 30 August 2017
  • stat441w18/e-gan
    ...the previous sections, alternatively updating G and D requires significant attention. We modify the way we update the generator G to improve stability and gener ...
    15 KB (2,279 words) - 22:00, 14 March 2018
  • stat946F18/Autoregressive Convolutional Neural Networks for Asynchronous Time Series
    This idea is also successfully used in attention networks[13] such as image captioning and machine translation. In this pape ...o, K., Courville, A., and Bengio, Y. Describing multi- media content using attention-based Encoder–Decoder networks. IEEE Transactions on Multimedia, 17(11): 18 ...
    29 KB (4,577 words) - 10:13, 14 December 2018
  • deflation Methods for Sparse PCA
    ...oped in the literature. The latter sub-task has received relatively little attention and is typically borrowed without justification from the PCA context. In th ...
    20 KB (3,332 words) - 09:45, 30 August 2017
  • conditional neural process
    ...ls which can do few-shot estimations of data. This can be implemented with attention mechanisms (Reed et al., 2017) or additional memory units in a VAE model ( ...their case features of samples are compared with target features using an attention kernel. At a higher level one can interpret this model as a CNP where the a ...
    32 KB (4,970 words) - 00:26, 17 December 2018
  • STAT946F17/ Coupled GAN
    ...ng generative adversarial networks[8], variational autoencoders (VAE)[17], attention models[18], have shown that a deep network can learn an image distribution ...
    32 KB (4,965 words) - 15:02, 4 December 2017
  • stat946F18/Beyond Word Importance Contextual Decomposition to Extract Interactions from LSTMs
    Attention-based models: #Bahdanau et al. (2014): These are a different class of models which use attention modules(different architectures) to help focus the neural network to decide ...
    31 KB (5,069 words) - 18:21, 16 December 2018
  • Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness
    ...red solution in image recognition and computer vision problems, increasing attention has been dedicated to evolving the network architecture to further improve ...
    16 KB (2,542 words) - 17:26, 26 November 2018
  • a fair comparison of graph neural networks for graph classification
    field has attracted the attention of a wide research community, which resulted in ...
    16 KB (2,430 words) - 00:54, 7 December 2020
  • One pixel attack for fooling deep neural networks
    Neural network first caught people’s attention during the 2012 imageNet contest. A solution using neural network achieve 8 ...
    17 KB (2,650 words) - 23:54, 30 March 2018
  • proposal for STAT946 projects Fall 2010
    Independent component analysis has been given more attention recently. It is become a popular method for estimating the independent feat ...
    17 KB (2,679 words) - 09:45, 30 August 2017
  • BERTScore: Evaluating Text Generation with BERT
    ...calculated using BERT, Roberta, XLNET, and XLM models, which utilize self-attention and nonlinear transformations. ...
    17 KB (2,510 words) - 01:32, 13 December 2020
  • Dense Passage Retrieval for Open-Domain Question Answering
    ...ieval can potentially improve QA system performance, and has received more attention. ...
    17 KB (2,691 words) - 22:57, 7 December 2020
  • THE LOGICAL EXPRESSIVENESS OF GRAPH NEURAL NETWORKS
    ...uivalently, to the description logic ALCQ, which has received considerable attention in the knowledge representation community (Baader et al., 2003; Baader & Lu ...
    17 KB (2,786 words) - 17:02, 6 December 2020
  • SuperGLUE
    ...it, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, page ...
    16 KB (2,331 words) - 16:58, 6 December 2020
  • IPBoost
    ...n approaches for solving machine learning problems have gained significant attention. In this paper, the non-convex boosting in classification using integer pro ...
    18 KB (2,846 words) - 00:18, 5 December 2020
  • Fix your classifier: the marginal value of training the last weight layer
    ...ts are fixed to need only 1-bit precision, it is now possible to focus our attention on the features preceding it. ...szkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. 2017. ...
    34 KB (5,105 words) - 00:39, 17 December 2018
  • Predicting Floor Level For 911 Calls with Neural Network and Smartphone Sensor Data
    ...911 calls: victims trapped in a tall building who seek immediate medical attention, locating emergency personnel such as firefighters or paramedics, or a mino ...
    18 KB (2,896 words) - 18:43, 16 December 2018
  • A universal SNP and small-indel variant caller using deep neural networks
    ...ining parameters still needs to be clarified. In addition, it is worthy of attention and further explanation about the results of prediction accuracy, under wha ...
    18 KB (2,856 words) - 04:24, 16 December 2020
  • Learning What and Where to Draw
    * Gregor et al. (2015) used a recurrent variational autoencoder with attention mechanisms for reading and writing different portions of the image canvas. ...
    18 KB (2,781 words) - 12:35, 4 December 2017
  • measuring Statistical Dependence with Hilbert-Schmidt Norm
    Now we turn our attention back to the the problem of measuring independence between two (generally mu ...
    27 KB (4,561 words) - 09:45, 30 August 2017
  • F18-STAT841-Proposal
    ...a result, data-intensive promotional strategies are getting more and more attention nowadays from marketing teams to further improve company returns. ...
    20 KB (2,757 words) - 14:41, 13 December 2018
  • Deep Double Descent Where Bigger Models and More Data Hurt
    ...zkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. ...
    19 KB (2,731 words) - 21:29, 20 November 2021
  • Hierarchical Representations for Efficient Architecture Search
    ...use of algorithms for finding optimal DNN architectures has attracted the attention of researchers who have tackled the problem through four main groups of tec ...
    30 KB (4,568 words) - 12:53, 11 December 2018
  • stat946w18/Wavelet Pooling For Convolutional Neural Networks
    * At the beginning, the study mentions that the pooling method is not under attention as it should be. In the end, results show that choosing the pooling method ...
    26 KB (3,974 words) - 20:50, 11 December 2018
  • Learning the Number of Neurons in Deep Networks
    ...to be an unaccounted component of tuning the model but this receives scant attention in the current paper. Several numerical comparisons should be carried out t ...
    24 KB (3,886 words) - 01:20, 3 December 2017
  • Adacompress: Adaptive compression for online computer vision services
    ...erent image sets and backend cloud services. Also, by taking a look at the attention maps for some of the images, we will figure out why the agent has chosen th ...
    27 KB (4,274 words) - 00:07, 8 December 2020
  • Surround Vehicle Motion Prediction
    ...ory prediction algorithms using other machine learning algorithms, such as attention-aware neural networks. ...
    29 KB (4,569 words) - 23:12, 14 December 2020
  • proposal Fall 2010
    ...ization is a key issue that has recently attracted a significant amount of attention in a wide range of applications. Navigation, vehicle tracking, Emergency Ca ...
    28 KB (4,210 words) - 09:45, 30 August 2017
  • stat841
    From the above code, we should pay attention to the following aspects when comparing with SVD method: ...ki/Vapnik Vapnik], Chervonenkis et al., however the ideas did not gain any attention until strong results were shown in the early 1990s. ...
    263 KB (43,685 words) - 09:45, 30 August 2017
  • stat341f11
    ...so when we need to determine <math>\!x_{t+1}</math>, we only need to pay attention in <math>\!x_{t}</math>. ...
    139 KB (23,688 words) - 09:45, 30 August 2017
  • stat841f11
    ...e started in the late seventies (Vapnik, 1979), it is receiving increasing attention recently by researchers. It is such a powerful method that in the few years ...ssified points are given higher weight to ensure the classifier "pays more attention" to them, to fit better in the next iteration. The idea behind boosting is ...
    314 KB (52,298 words) - 12:30, 18 November 2020
  • stat841f10
    ...ferent function altogether, such as a <math>\,sin(x)</math> dimension. Pay attention, We don't do QDA with LDA. If we try QDA directly on this problem the resul SVM was introduced after neural networks and gathered attention by outperforming neural networks in many applications e.g. bioinformatics, ...
    451 KB (73,277 words) - 09:45, 30 August 2017
  • stat340s13
    Attention:There is a "dot" between sqrt(d) and "*". It is because d and tet are vecto ...
    370 KB (63,356 words) - 09:46, 30 August 2017
Retrieved from "http://wiki.math.uwaterloo.ca/statwiki/index.php?title=Special:Search"