Peanut BERTer

Inspiration

Online review platforms are flooded with noise; up to 70% of location reviews can be spam, irrelevant content, or fake rants that mislead consumers and harm legitimate businesses. We were inspired by the challenge of helping people make better decisions by automatically filtering out low-quality reviews while preserving authentic feedback, including legitimate negative experiences. Additionally, our team comprises people who are all reliant on online Google reviews to make decisions on which businesses to patronize, hence we felt a greater need to resolve this issue.

What it does

Peanut BERTer is an AI-powered review quality assessment system that evaluates Google location reviews across two key dimensions:

Relevance – whether the review meaningfully discusses the business, measured via semantic similarity between the review text and the business description.
Spam Detection – whether the review is duplicated or inauthentic, identified by checking for unusually high similarity between reviews posted across different businesses. The solution also filters responses based on these 3 rules
No Promotions - Reviews should not contain promotional links
No irrelevant content - Reviews must be about the location, not unrelated topics
No rant without visits - Rants/complaints must come from actual visitors (inferred via content/metadata). Features and Functionality:
Relevance scoring with sentence-transformers/all-MiniLM-L6-v2, comparing each review against the business description.

Spam detection with the same model by measuring semantic similarity across reviews submitted to different businesses.
Multilingual support through automatic language detection and translation to English for consistent processing.
Data preprocessing pipeline to handle raw gzipped JSONL review files and convert them into efficient Parquet storage.

Our solution specifically addresses the relevancy portion of this problem, focusing on filtering out irrelevant and spammy reviews while surfacing those that provide genuine value.

How we built it

We built Peanut BERTer using a multi-stage ML pipeline in Google Colab:

Data Processing: Converted gzipped JSONL Google Local Reviews data into efficient Parquet format for processing 100,000+ reviews from Alaska.

Feature Engineering: Extracted textual features (review length, URI detection, picture presence) and user behavior patterns (review frequency, rating patterns).

Semantic Analysis: Used sentence-transformers/all-MiniLM-L6-v2 to generate embeddings for relevance scoring by comparing reviews to business descriptions and peer reviews.

Sentiment Analysis: Implemented Twitter-RoBERTa-base-sentiment to detect review polarity and identify potential inconsistencies with star ratings.

Spam Detection: Built an efficient similarity search using FAISS IVF indexing to identify near-duplicate reviews across different businesses at scale.

Penalty System: Developed weighted scoring to penalize very short generic reviews and long rambling off-topic content.

Challenges we ran into

Scale and Memory: Processing 100,000+ reviews required careful memory management, chunked processing, and efficient storage formats to avoid crashes in Google Colab.

Multilingual Complexity: Handling reviews in multiple languages required implementing translation pipelines while maintaining semantic meaning.

Balance Sensitivity: Creating a system that filters spam without removing legitimate negative reviews required careful threshold tuning and feature weighting.

Ground Truth Labeling: Without pre-labeled data, we had to develop synthetic datasets and pseudo-labeling techniques to validate our approach.

Accomplishments that we're proud of

Scalable Architecture: Successfully processed 100,000+ real Google reviews with efficient memory management and GPU acceleration. Multilingual Pipeline: Built comprehensive translation support that maintains review authenticity across languages. Balanced Detection: Developed a system that identifies spam while preserving legitimate criticism—crucial for maintaining trust. Real-world Performance: Achieved strong relevance scoring using semantic similarity that captures genuine business-review relationships.

What we learned

Real-world review data is messy, handling edge cases, missing fields, and encoding issues is 80% of the work.
BERT-based embeddings effectively capture relevance between reviews and business descriptions, even with limited training data.
Cross-business similarity detection is powerful for identifying template spam, but requires careful threshold tuning.

What's next for Peanut BERTer

Real-time Deployment: Integrate with review platforms for live filtering as reviews are submitted. Advanced ML Models: Experiment with larger language models and fine-tuning on review-specific datasets. User Feedback Loop: Implement mechanisms for users to provide feedback on filtering decisions to continuously improve accuracy. Business Intelligence: Expand beyond filtering to provide businesses with insights about genuine customer sentiment trends. Platform Integration: Build APIs for easy integration with existing review platforms and business management systems.