Fake News Detection

Misinformation has become increasingly difficult to distinguish from factual reporting, and its spread carries real social and political consequences. This project applies NLP and classical machine learning to automatically evaluate the credibility of a news article from its title and body text. The end product is a Streamlit application where a user pastes in an article and instantly receives a real/fake prediction with a confidence percentage (e.g., 90% real / 10% fake), plus a page for downloading and preprocessing the underlying training dataset.

Misinformation spreads quickly and is often difficult for the average reader to identify, especially as fake news is increasingly written to mimic the tone and structure of legitimate reporting. Manual fact-checking doesn't scale to the volume of content published daily. The goal was to build a reliable, automated classifier that evaluates the linguistic and sentiment patterns of an article's text to predict whether it is credible or fabricated, and to package that model behind an interface accessible to non-technical users.

Started with exploratory analysis using matplotlib, wordcloud, and nltk to visualize article length, top words in real vs. fake articles, and sentiment differences between classes. Built a text-cleaning pipeline that drops rows with missing values, lowercases text, strips punctuation and parenthetical content, removes redundant whitespace, and filters stop words, then generated a sentiment score for every article with NLTK's SentimentIntensityAnalyzer, parallelized with swifter to make processing the full 72K-article dataset feasible. Combined the cleaned title and body into one field per article, vectorized it with TF-IDF (max_features=5000, fit on the training split), and merged the TF-IDF matrix with the sentiment score via scipy.hstack to form the final feature matrix. Split 80/20 into train/test and trained two base classifiers — logistic regression for linear word-credibility relationships, random forest for non-linear ones — then, rather than simply averaging the two, trained a third logistic regression as a meta-classifier on the base models' predicted probabilities, learning how to weight each one's output for the final prediction and confidence score. Used GridSearchCV to tune both base models and reduce overfitting, landing on LogisticRegression(max_iter=1000, C=10) and RandomForestClassifier(max_depth=15, max_features='sqrt', min_samples_leaf=1, min_samples_split=5, n_estimators=200). Evaluated all three models on accuracy, precision, recall, and F1, plus confusion matrix heatmaps, ROC curves, and a precision-recall/calibration curve for the meta-classifier. Saved the trained models and TF-IDF vectorizer with joblib and built a Streamlit interface for real-time predictions and for downloading/preprocessing the source dataset via a user-supplied Kaggle API key.

All metrics computed on the held-out 20% test split: Logistic Regression reached 94.87% accuracy (precision/recall/F1 ≈ 0.95 across fake and real), Random Forest reached 92.42% (≈ 0.92–0.94), and the stacked meta-classifier reached 95.51% (≈ 0.95–0.96) — outperforming both individual models on every metric, confirming that combining a linear model's word-level signal with a random forest's ability to capture non-linear word interactions adds predictive value beyond simple averaging. Two challenges shaped the build: slow sentiment scoring on 72K+ articles was resolved by parallelizing with swifter, and early overfitting was resolved with GridSearchCV hyperparameter tuning. Stacking two models' outputs as input to a third was a new technique for this project, implemented as the meta-classifier. Possible next steps: expand training data by scraping additional, more diverse news sources to improve generalization, and package the model as a browser extension that flags potential misinformation in real time as users browse.

outcome: 95.51% accuracy from a stacked meta-classifier — beating both individual base models on every metric on a 20% held-out test split of 72,134 articles.

• 

  Precision-recall curve — meta-classifier (AP = 0.99), the model that ships.

• 

  ROC curve — logistic regression base model (AUC = 0.99).

• 

  ROC curve — random forest base model (AUC = 0.98).

• scikit-learn
• NLTK
• TF-IDF
• Streamlit
• GridSearchCV