This repository contains the code and scripts required to train a BERT model for sentiment analysis on financial news using GSB Yen's GPU node at Stanford. The BERT model is trained on a Kaggle dataset of financial phrase bank. The project utilizes the transformers library along with PyTorch Lightning for streamlined model training and evaluation.

Please refer to this RCpedia Topic Guide for details.

• Getting Started
• Repository Structure
• Training Script Details
• Usage
• Monitoring Training
• Output
• Authors

Clone the repository to the Yens to run the example trainig script.

$ git clone https://github.com/gsbdarc/yens-gpu-demo.git 
$ cd yens-gpu-demo

• Sentences_AllAgree.txt: dataset of 2,264 financial news headlines labeled with a sentiment (Malo et al., 2014).
• finBERT-train.slurm: Slurm script for training the BERT model to be run on the Yens.
• finBERT-train.py: Python script to train the model on Yen's GPU node.

The training script finBERT-train.py performs the following:

• Defines a custom dataset to read in financial news texts and their labels.
• Utilizes the BERT model from the transformers library for sentiment analysis.
• Trains the model using PyTorch Lightning's Trainer on Yen's GPU.

Modify the Slurm script to include your email address. Slurm will report useful metrics via email such as queue time, runtime, CPU and RAM utilization and will alert you if the job has failed.

Submit the Slurm script to initiate the model training on gpu partition on the Yens:

$ sbatch finBERT-train.slurm

Monitor the training progress by checking the Slurm queue for your username:

$ squeue -u $USER

Instructions for monitoring GPU utilization and other training metrics can be found in the Topic Guide.

After the training is complete, check the output file finBERT-train.out for training and evaluation metrics:

$ cat finBERT-train.out

• Stanford GSB DARC Team ([email protected])