Bringing good news to thousands of readers around the world
.png)
70%+
Reduction in fake news, verified by the Nomad content team
94%+
Accuracy, verified by the Nomad content team
200,000+
News articles and posts used for training
TL;DR
.svg)
Nomad runs a mobile app that spreads positive news, but identifying positive stories is hard when coverage is dominated by negative news.
We built a news classification engine on XLNet, orchestrated in Airflow with Spark preprocessing, PostgreSQL storage and automatic retraining when recall drops.
The engine reached 94%+ accuracy and cut fake news by over 70%, with 89% recall on production data.
Nomad built a mobile news app that spreads positive news to thousands of readers around the world, as a way to support mental health.
We helped develop a news classification engine that scans RSS feeds, news sites and search crawlers daily to identify positive news and push it to readers in the Goodable app
The challenges
Identifying positive news is hard in environments dominated by negative coverage.
1. Sentiment hidden in context
2. Volume of negative news
What we built
1. An XLNet classification model
XLNet models use transformer architecture and excel at understanding complex relationships in text and detecting subtle sentiment cues, enabling reliable classification even amid negative news.
.png)
2. A daily inference pipeline
A unified Airflow workflow runs daily as a batch job, capturing all news published during the day:
Aggregates articles from multiple sources into PostgreSQL
Pulls the latest model artifacts from S3
Cleans and transforms data with Apache Spark
Runs predictions and saves results to the backend database
3. Retraining on performance decay
A separate training pipeline retrains the model when recall falls below 85%. Experts review predictions daily, focusing on high-risk articles labeled positive, and the model achieved 89% recall on production data.
.png)
Conclusion
The classification engine improved content quality and reliability on the platform, reducing fake news by over 70% with 94%+ accuracy, trained on more than 200,000 curated news articles and posts.
Building a machine learning pipeline that has to stay accurate in production? Get in touch.
70%+
Reduction in fake news, verified by the Nomad content team
94%+
Accuracy, verified by the Nomad content team
200,000+
News articles and posts used for training
Use case: AI, machine learning, design | Industry: news
Model: XLNet transformer classifier
Orchestration: Apache Airflow, daily batch | Processing: Apache Spark
Storage: PostgreSQL, model artifacts on S3 | Retraining: triggered when recall drops below 85%
FAQ
How do you build an AI positive news classifier?
Train a transformer model such as XLNet on curated articles, and run it in a daily Airflow pipeline that aggregates sources, preprocesses with Spark and stores predictions.
How do you keep a news classification model accurate after launch?
Have experts review high-risk predictions daily, track recall, and retrain automatically when recall falls below 85%.
Why use a daily batch instead of streaming?
A daily batch captures all news published during the day and produces consolidated predictions at day's end, which matched the business need.
What results did the engine deliver?
94%+ accuracy, over 70% less fake news and 89% recall on production data. See what we learned deploying AI in production.
Airflow, Spark
PostgreSQL, S3
Read more
You’re all set!
.svg)
