DeepSentiment

Introduction

Sentiment classification plays a crucial role in various applications such as social media analysis and customer feedback processing. In this study, we implement three approaches for sentiment classification: graph convolutional networks (GCN), multi-channel convolutional neural networks (MC-CNN), and adaptive multi-channel GCN (AM-GCN). These models have been proposed in separate papers and have shown promising results in capturing complex patterns in text data for sentiment analysis tasks.

Our objective is to compare the performance of these three models and evaluate their effectiveness in handling sentiment classification tasks. By implementing these models and conducting a comparative analysis, we aim to gain insights into their strengths and weaknesses, and identify which model performs best for sentiment classification.

Related work

MC-CNN: https://aclanthology.org/N19-1137.pdf
GCN: https://arxiv.org/pdf/1809.05679.pdf
AM-GCN: http://shichuan.org/doc/86.pdf

Data

https://huggingface.co/datasets/stanfordnlp/imdb Stanford Large Movie Review Dataset. This is a commonly used dataset for binary sentiment classification. It contains 25,000 highly polar movie reviews for training and 25,000 for testing, along with additional unlabeled data. The dataset provides a substantial amount of data for sentiment analysis tasks, allowing for more robust model training. We do not need to perform much preprocessing other than tokenizing the text and possibly handling issues like punctuation, stopwords, and lowercasing.

Methodology

Architecture

Graph Convolutional Network (GCN): GCNs are specialized neural networks designed to work with graph-structured data, which is particularly useful for tasks where the input can be represented as a graph. In our case, we will represent the text data as a graph where nodes represent words or tokens, and edges represent relationships between them (e.g., co-occurrence). The GCN will learn to perform sentiment classification based on this graph representation of the text.
Multi-Channel Convolutional Neural Network (CNN): This model will consist of multiple parallel channels, each processing the input text in a different way (e.g., using different filter sizes or kernel sizes). The outputs of these channels will be concatenated and fed into a fully connected layer for classification. This architecture is known for its effectiveness in capturing different aspects of the input data.
Adaptive Multichannel GCN: This is an extension of the GCN model that simultaneously extracts specific and common embeddings from node features, topological structures, and their combinations. We use attention mechanisms to learn adaptive importance weights for these embeddings, allowing us to capture the most relevant information from both node features and topological structures

Training We will train all 3 models on the datasets proposed in their original respective papers. Then we will train them on the Stanford Large Movie Review Dataset and compare the accuracy of these 3 models.

Metrics For assessing our model's performance, we will primarily look at accuracy, precision, recall, and F1-score.

Challenges Converting code from PyTorch to Tensorflow

Goals

Base goal: Have 2 working models and train on the original datasets
Target goal: Have 3 models working and train on IMDB dataset
Reach goal: Finetune the models and train on multiple original datasets

Ethics

What is your dataset? Are there any concerns about how it was collected, or labeled? Is it representative? What kind of underlying historical or societal biases might it contain? The Stanford Large Movie Review Dataset consists of movie reviews labeled as positive or negative sentiment. While this dataset is widely used, there may be concerns about its representativeness, as it focuses on a specific domain (movies) and may not capture sentiment nuances present in other domains. Additionally, there could be biases in the dataset, such as certain movies or genres being overrepresented, which could impact the model's performance and generalizability.
Who are the major “stakeholders” in this problem, and what are the consequences of mistakes made by your algorithm? The main stakeholders include filmmakers, movie studios, marketing agencies, and audiences since the sentiment analysis is meant to help them identify whether their movie was well received or not. Mistakes of the algorithm may potentially mislead them on the type of feedback that they would receive. The people reviewing the movies might also want their reviews to be considered more carefully.