A Position Aware Decay Weighted Network
for Aspect based Sentiment Analysis

Early works of ATSA, employ lexicon based feature selection techniques like Parts of Speech Tagging (POS), unigram features and bigram features [4]. However, these methods do not consider aspect terms and perform sentiment analysis on the given sentence.
Phrase Recursive Neural Network for Aspect based Sentiment Analysis (PhraseRNN) [6] was proposed based on Recursive Neural Tensor Network [10] primarily used for semantic compositionality. PhraseRNN uses dependency and constituency parse trees to obtain aspect representation. An end-to-end neural network model was introduced for jointly identifying aspect and polarity [9]. This model is trained to jointly optimize the loss of aspect and the polarity. In the final layer, the model outputs one of the sentiment polarities along with the aspect. [14] introduced Aspect Fusion LSTM (AF-LSTM) for performing ATSA.

Let V be the vocabulary size considered and $X$ $\in$ $\mathbb{R}^{V\times d_{w}}$ represent the embedding matrix¹¹1https://nlp.stanford.edu/data/glove.840B.300d.zip, where for each word $X_{i}$ is a $d_{w}$ dimensional word vector. Words contained in the embedding matrix are initialized to their corresponding vectors whereas words not contained are initialized to 0’s. $I$ $\in$ $\mathbb{R}^{n\times d_{w}}$ denotes the pretrained embedding representation of a sentence where $n$ is the maximum sentence length.

In the ATSA task, aspect $A$ is contained in the sentence $S$. A can be a word or a phrase. Let $k_{s}$ denote the starting index and $k_{e}$ denote the ending index of the aspect term(s) in the sentence. Let $i$ be the index of a word in the sentence. The position encoding of words with respect to aspect are represented using the formula

The position encodings for the sentence “granted the space is smaller than most it is the best service” where “space” is the aspect is shown in Figure 2. This number reflects the relative distance of a word from the closest aspect word. The position embeddings from the position encodings are randomly initialized and updated during training. Hence, $P$ $\in$ $\mathbb{R}^{n\times d_{p}}$ is the position embedding representations of the sentence. $d_{p}$ denotes the number of dimensions in the position embedding.

As shown in Figure 2, PDN comprises of two sub-networks: Position Aware Attention Network(PDN) and Decay Weighting Network (DWN).

We performed experiments on two datasets, Restaurant and Laptop from SemEval 2014 Task 4 [8]. Each data point is a triplet of sentence, aspect and sentiment label. The statistics of the datasets are shown in the Table 2. As most existing works reported results on three sentiment labels positive,negative,neutral we performed experiments by removing conflict label as well.

We compare proposed model to the following baselines:

Every word in the input sentence is converted to a 300 dimensional vector using pretrained word embeddings. The dimension of positional embedding is set to 25 which is initialized randomly and updated during training. The hidden units of LSTM are set to 100. The number of hidden units in the layer fully connected to LSTM is 50 and the layer fully connected to positional embedding layer is 50. The number of hidden units in the penultimate fully connected layer is set to 64. We apply a dropout [11] with a probability 0.5 on this layer. A batch size 20 is considered and the model is trained for 30 epochs. Adam [3] is used as the optimizer with an initial learning rate 0.001.

The proposed model outperforms other recent and popular architectures as well, these architectures use a separate architecture which takes the aspect input distinctly from the sentence input. In doing so they loose the positional information of the aspect within the sentence. We hypothesize that this information is valuable for ATSA and our results reflect the same. Additionally since proposed architecture does not take any additional aspect inputs apart from position, we get a fairer comparison on the benefits of providing aspect positional information over the aspect words themselves.

Furthermore, while avoiding learning separate architectures for weightages, decay functions act as good approximates. These functions rely on constants alone and lack any parameters thereby expressing their efficiency. The reason these functions work is because they consider an assumption intrinsic to the nature of most natural languages. It is that description words or aspect modifier words come close to the aspect or the entity they describe. For example in Figure 2, we see the sentence from the Restaurant dataset, “granted the space is smaller than most, it is the best service you can…”.The proposed model is able to handle this example which has distinct sentiments for the aspects “space” and “service” due to their proximity from “smaller” and “best” respectively.