Student Dropout
Risk: ML vs
Deep Learning
A complete walkthrough of the Student Academic Performance & Dropout Risk dataset from Kaggle. Build 5 scikit-learn models and 2 TensorFlow neural networks to predict academic scores (regression) and classify high vs. low performers (classification). Every line of code is explained for beginners — with copy buttons on every block.
# Student Dropout Risk — KaggleHub import kagglehub from kagglehub import KaggleDatasetAdapter df = kagglehub.load_dataset( KaggleDatasetAdapter.PANDAS, "mohankrishnathalla/student...", file_path, ) print("First 5 records:", df.head())
Six steps from raw CSV
to model verdict.
The complete pipeline for the Student Dropout Risk machine learning project. Click any step to jump straight to its code and beginner-friendly explanation.
Data Loading with KaggleHub
Stream the student academic performance & dropout risk dataset straight into a pandas DataFrame. Check shape, head, info, and describe. No manual download needed.
Exploratory Data Analysis
Histograms + KDE for academic scores and key features. Scatter plots of attendance vs. performance. Correlation heatmap of all numeric features.
Preprocessing & Encoding
Handle missing values, label encode categorical features, auto-detect the academic performance column, create high vs. low performer classification target, scale features without leakage.
Machine Learning Models
Train Linear/Logistic Regression, Random Forest, Gradient Boosting, SVM and KNN — to predict academic score (regression) and classify high vs. low performer (classification).
Deep Learning Models
Two 4-layer TensorFlow networks (128→64→32→1) with ReLU, BatchNorm, Dropout, EarlyStopping and ReduceLROnPlateau — one regression, one classification.
ML vs DL Comparison
Side-by-side leaderboard: R², RMSE, MAE for regression; accuracy, precision, recall, F1 for classification. The verdict on tree ensembles vs neural networks on education data.
Load the dataset with KaggleHub.
Stream the student academic performance & dropout risk dataset straight into a pandas DataFrame — no manual download needed.
What is this dataset? The Student Academic Performance & Dropout Risk dataset contains data about students’ academic records, attendance, demographics, and risk factors associated with dropping out of school. Each row is one student. It’s perfect for learning ML because it combines academic predictors (attendance, previous grades, study hours) with an outcome variable (performance score / dropout risk) — exactly the kind of real-world problem ML solves in education.
What is KaggleHub? Kaggle is a website where people share datasets for machine learning. KaggleHub is a Python library that lets you load those datasets directly into your code with one line — no manual downloading.
What is a DataFrame? A pandas DataFrame is like an Excel spreadsheet inside Python. It has rows (one per student) and columns (attendance, GPA, age, dropout risk, etc.). The variable df we create here holds all the student data.
Why check shape, head(), info(), describe()?
df.shape→ How many rows (students) and columns?df.head()→ Show the first 5 rows so you can see column names and sample valuesdf.info()→ What type of data is in each column? (text vs. numbers) How many missing values?df.describe()→ Basic statistics: mean GPA, mean attendance, mean study hours, min, max
Important: The file_path is the CSV filename inside the dataset. Visit the Kaggle dataset page → Files tab to find the exact filename. Also, check df.head() to identify the exact column name for academic performance — it might be called gpa, academic_score, grade, or something else. The code in Step 3 auto-detects it.
# Install dependencies as needed:
# pip install kagglehub[pandas-datasets] pandas numpy matplotlib seaborn scikit-learn tensorflow
import kagglehub
from kagglehub import KaggleDatasetAdapter
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
# Set the path to the file you'd like to load
# Check the Kaggle dataset page → Files tab for the exact filename
file_path = "student_performance_dropout.csv"
# Load the latest version
df = kagglehub.load_dataset(
KaggleDatasetAdapter.PANDAS,
"mohankrishnathalla/student-academic-performance-and-dropout-risk",
file_path,
)
print("Dataset shape:", df.shape)
print("First 5 records:")
print(df.head())
print(df.info())
print(df.describe())kagglehub loads the dataset, pandas handles data tables, numpy handles math, matplotlib and seaborn create charts."mohankrishnathalla/student-academic-performance-and-dropout-risk", (3) load the file we specified. Result stored in df.object = text (categorical), int64/float64 = numbers. Also shows missing values.Exploratory Data Analysis in seaborn.
Before any model touches the data, we visualize distributions, plot attendance vs. performance, and check correlations.
What is EDA? Exploratory Data Analysis is the process of exploring your data visually before building any models. It’s like reviewing student records before designing an intervention — you want to know what patterns exist.
What is a histogram? A histogram counts how many students fall into each range. For example, how many have a GPA of 2.0–2.5, how many 2.5–3.0, etc. The KDE curve (smooth line on top) estimates the overall shape.
What is a scatter plot? A scatter plot places each student as a dot, with one feature on the x-axis and academic performance on the y-axis. Plotting Attendance vs. GPA reveals whether students with higher attendance tend to perform better — the core question for dropout prevention.
What is correlation? Correlation measures how strongly two variables move together, from -1 to +1. If attendance and GPA correlate positively (above 0.5), it means students who attend more classes tend to get higher grades. The heatmap shows all pairwise correlations at once.
Why does EDA matter? The scatter plots reveal whether the relationship between attendance, study hours, and academic performance is linear (a clear trend) or noisy. This tells you whether to expect high or low R² from your models. Also, the dropout risk distribution shows how many students are at risk.
2.1 · Missing values, target detection & distributions
# Check for missing values
print("Missing values:\n", df.isnull().sum())
print("\nDuplicates:", df.duplicated().sum())
categorical_cols = df.select_dtypes(include=['object']).columns
numeric_cols = df.select_dtypes(include=[np.number]).columns
print(f"\nCategorical columns: {list(categorical_cols)}")
print(f"Numeric columns: {list(numeric_cols)}")
# Auto-detect the academic performance/target column
target_col = None
for candidate in ['gpa', 'academic_score', 'performance',
'grade', 'test_score', 'academic_performance',
'scores', 'final_grade', 'semester_grade',
'cgpa', 'grade_point_average']:
if candidate in df.columns.str.lower().tolist():
target_col = df.columns[df.columns.str.lower() == candidate][0]
break
if target_col is None:
target_col = numeric_cols[-1]
print(f"Warning: Using '{target_col}' as target.")
print(f"Academic performance target: '{target_col}'")
# Distributions of key numeric features
key_cols = list(numeric_cols[:4])
if len(key_cols) > 0:
fig, axes = plt.subplots(1, len(key_cols), figsize=(5*len(key_cols), 5))
colors = ['#4C72B0', '#55A868', '#C44E52', '#8172B2']
for ax, col, color in zip(axes, key_cols, colors):
sns.histplot(df[col], kde=True, color=color, ax=ax)
ax.axvline(df[col].mean(), color='black', linestyle='--', label=f"Mean: {df[col].mean():.1f}")
ax.set_title(f'Distribution of {col}')
ax.legend()
plt.tight_layout()
plt.show()gpa, academic_score, grade, cgpa, etc.). If none match, it falls back to the last numeric column. Always verify the detected column name in the output.2.2 · Scatter plots & correlation heatmap
# Scatter: first numeric feature vs target
if len(numeric_cols) >= 2:
feat = numeric_cols[0]
plt.figure(figsize=(10, 6))
sns.scatterplot(data=df, x=feat, y=target_col, s=60, color='#4C72B0')
plt.title(f'{feat} vs {target_col}')
plt.show()
# Correlation heatmap
plt.figure(figsize=(10, 8))
sns.heatmap(df[numeric_cols].corr(), annot=True, cmap='coolwarm', fmt='.2f',
square=True, linewidths=0.5)
plt.title('Correlation Between Numeric Features')
plt.show()The scatter plot of attendance vs. GPA is the key visualization. If you see an upward trend, it confirms that students with higher attendance tend to perform better academically. The correlation heatmap shows all pairwise relationships — look for strong correlations (above 0.5 or below -0.5) between features and the academic performance target.
Encode, engineer, scale — without leakage.
Handle missing values, encode categorical features, create a high vs. low performer target, and scale features.
Why do we need preprocessing? Machine learning models only understand numbers. If the dataset has text columns (like gender or dropout risk category), we need to convert them to numbers. Also, some rows might have missing values that need to be handled.
Handling missing values: We use dropna() to remove rows with missing data. If the dataset has many missing values, you might want to fill them with the mean or median instead.
Label encoding for categorical features: We use LabelEncoder to convert text columns (gender, dropout_risk category, etc.) to numbers. Example: Low=0, Medium=1, High=2 for dropout risk.
What is a target variable? For regression, the target is the academic performance column (GPA, score, etc.). For classification, we create a high_performer target: 1 if performance ≥ median, 0 if below. This splits students into “high performers” and “low performers” — a proxy for dropout risk.
What is train/test split? We divide data into 80% for training and 20% for testing. random_state=42 makes the split reproducible.
What is scaling? Standardization rescales every feature so mean=0, std=1. This matters because attendance might range from 0–1 (percentage) while age ranges from 15–25 — without scaling, features with larger ranges would dominate.
What is data leakage? When information from the test set accidentally influences training. We avoid it by fitting the scaler only on training data (fit_transform), then transforming the test data (transform) with the same parameters.
from sklearn.preprocessing import LabelEncoder, StandardScaler
from sklearn.model_selection import train_test_split
data = df.copy()
data = data.dropna()
# 3.1 Encode categorical features
le = LabelEncoder()
for col in categorical_cols:
data[col] = le.fit_transform(data[col].astype(str))
# 3.2 Create classification target
target_median = data[target_col].median()
data['high_performer'] = (data[target_col] >= target_median).astype(int)
# 3.3 Define features (X) and targets (y)
X_reg = data.drop(columns=[target_col, 'high_performer'])
y_reg = data[target_col]
X_clf = data.drop(columns=[target_col, 'high_performer'])
y_clf = data['high_performer']
# 3.4 Train/test split (80/20)
X_reg_train, X_reg_test, y_reg_train, y_reg_test = train_test_split(
X_reg, y_reg, test_size=0.2, random_state=42)
X_clf_train, X_clf_test, y_clf_train, y_clf_test = train_test_split(
X_clf, y_clf, test_size=0.2, random_state=42)
# 3.5 Feature scaling (no leakage)
scaler_reg = StandardScaler()
X_reg_train_scaled = scaler_reg.fit_transform(X_reg_train)
X_reg_test_scaled = scaler_reg.transform(X_reg_test)
scaler_clf = StandardScaler()
X_clf_train_scaled = scaler_clf.fit_transform(X_clf_train)
X_clf_test_scaled = scaler_clf.transform(X_clf_test)
print(f"Target median: {target_median}")
print(f"Train shape: {X_reg_train.shape}, Test shape: {X_reg_test.shape}")
print(f"Class distribution:\n{y_clf.value_counts(normalize=True)}")file_path from Step 1 and the target_col from Step 2 must be correct. Visit the Kaggle dataset page to verify the CSV filename, and check df.head() output to confirm the exact column name for academic performance.random_state=42 = reproducible.fit_transform on train (learns mean/std), transform on test (applies same parameters). No data leakage.Five scikit-learn models,
two tasks.
For each task — predict academic score (regression) and classify high vs. low performer (classification) — we train five models.
What is supervised learning? Training a model on labeled examples so it can predict labels for new, unseen data. Two types:
- Regression: predict a continuous number — e.g., “this student will have a GPA of 3.2”
- Classification: predict a category — e.g., “this student is a high performer (1) or at risk (0)”
The five models, explained simply:
- Linear Regression: draws the best-fitting straight line through the data. Simple, fast baseline.
- Logistic Regression: for classification — estimates the probability of being a high performer.
- Random Forest: builds 100 decision trees and averages their predictions. “Wisdom of the crowd.”
- Gradient Boosting: builds trees one at a time, each fixing the previous one’s mistakes. Often the strongest on tabular data.
- SVM: finds the best boundary between classes. Good for small datasets, sensitive to scaling.
- KNN: for a new student, finds the 5 most similar students and averages their scores. Simple but scaling-sensitive.
Regression metrics:
- RMSE: “off by X points on average” (in GPA/score units).
- R²: variance explained. 1.0 is perfect, 0 = no better than guessing the mean.
Classification metrics:
- Accuracy: fraction of all predictions that were correct.
- Precision: of all predicted “high performers,” how many actually were?
- Recall: of all actual “high performers,” how many did we catch? Crucial for dropout prevention — missing an at-risk student is worse than a false alarm.
- F1: harmonic mean of precision and recall.
4.1 · Regression — Predict Academic Score
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.svm import SVR
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
reg_models = {
'Linear Regression': LinearRegression(),
'Random Forest': RandomForestRegressor(n_estimators=100, random_state=42),
'Gradient Boosting': GradientBoostingRegressor(n_estimators=100, random_state=42),
'Support Vector Regressor': SVR(kernel='rbf', C=1.0),
'K-Nearest Neighbors': KNeighborsRegressor(n_neighbors=5),
}
reg_results = []
for name, model in reg_models.items():
model.fit(X_reg_train_scaled, y_reg_train)
y_pred = model.predict(X_reg_test_scaled)
mse = mean_squared_error(y_reg_test, y_pred)
rmse = np.sqrt(mse)
mae = mean_absolute_error(y_reg_test, y_pred)
r2 = r2_score(y_reg_test, y_pred)
reg_results.append({'Model':name, 'MSE':round(mse,2), 'RMSE':round(rmse,2), 'MAE':round(mae,2), 'R²':round(r2,4)})
reg_results_df = pd.DataFrame(reg_results).sort_values('R²', ascending=False)
print(reg_results_df.to_string(index=False))4.2 · Classification — Predict High vs. Low Performer
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix
clf_models = {
'Logistic Regression': LogisticRegression(random_state=42, max_iter=1000),
'Random Forest': RandomForestClassifier(n_estimators=100, random_state=42),
'Gradient Boosting': GradientBoostingClassifier(n_estimators=100, random_state=42),
'Support Vector Classifier': SVC(kernel='rbf', random_state=42, probability=True),
'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=5),
}
clf_results = []
for name, model in clf_models.items():
model.fit(X_clf_train_scaled, y_clf_train)
y_pred = model.predict(X_clf_test_scaled)
clf_results.append({'Model':name,'Accuracy':round(accuracy_score(y_clf_test,y_pred),4),'Precision':round(precision_score(y_clf_test,y_pred),4),'Recall':round(recall_score(y_clf_test,y_pred),4),'F1 Score':round(f1_score(y_clf_test,y_pred),4)})
clf_results_df = pd.DataFrame(clf_results).sort_values('Accuracy', ascending=False)
print(clf_results_df.to_string(index=False))
best_clf = clf_models[clf_results_df.iloc[0]['Model']]
y_pred_best = best_clf.predict(X_clf_test_scaled)
cm = confusion_matrix(y_clf_test, y_pred_best)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=['Low','High'], yticklabels=['Low','High'])
plt.show()Two TensorFlow neural networks
with modern regularization.
Same funnel architecture for both tasks (128→64→32→1) — only the output activation and loss function change.
What is a neural network? A stack of layers, each containing “neurons” (math units). Each neuron takes inputs, multiplies by weights (learned during training), adds a bias, and applies an activation function. “Deep” = many layers.
The architecture, explained:
Dense(128): 128 neurons, each connected to every neuron in the previous layer. First layer takes our features.Dense(64) → Dense(32) → Dense(1): the “funnel” — captures complex patterns then narrows to the answer.ReLU: if input is negative, output 0; if positive, output as-is. Lets the network learn non-linear patterns.
Regularization — preventing overfitting:
BatchNormalization: normalizes layer outputs during training. Stabilizes learning.Dropout(0.3): randomly disables 30% of neurons each step. Forces the network to learn redundant representations.
Training configuration:
Adam: adjusts learning rate automatically. The default choice for most projects.MSE loss(regression): penalizes large errors more than small ones.Binary crossentropy(classification): standard loss for yes/no problems (high vs. low performer).Epoch: one full pass through training data. We train for up to 200.Batch size (32): process 32 students at a time before updating weights.Validation split (0.2): 20% of training data held out to monitor overfitting.
Callbacks — automatic training controls:
EarlyStopping: if val loss doesn’t improve for 20 epochs, stop.restore_best_weights=Truerolls back to the best epoch.ReduceLROnPlateau: when val loss plateaus, halve the learning rate.
Regression vs classification — the only difference:
- Regression: final layer = LINEAR activation (outputs any number — the predicted GPA/score).
- Classification: final layer = SIGMOID activation (outputs 0–1 = probability of “high performer”). Threshold at 0.5.
5.1 · DL Regression — Predict Academic Score
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau
tf.random.set_seed(42)
np.random.seed(42)
def build_regression_model(input_dim):
model = Sequential([
Dense(128, activation='relu', input_shape=(input_dim,)),
BatchNormalization(), Dropout(0.3),
Dense(64, activation='relu'), BatchNormalization(), Dropout(0.2),
Dense(32, activation='relu'), Dropout(0.1),
Dense(1, activation='linear')
])
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
return model
dl_reg_model = build_regression_model(X_reg_train_scaled.shape[1])
early_stop = EarlyStopping(monitor='val_loss', patience=20, restore_best_weights=True, verbose=1)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, min_lr=1e-6, verbose=1)
history_reg = dl_reg_model.fit(X_reg_train_scaled, y_reg_train, validation_split=0.2, epochs=200, batch_size=32, callbacks=[early_stop, reduce_lr], verbose=1)
dl_reg_pred = dl_reg_model.predict(X_reg_test_scaled).flatten()
dl_reg_rmse = np.sqrt(mean_squared_error(y_reg_test, dl_reg_pred))
dl_reg_r2 = r2_score(y_reg_test, dl_reg_pred)
print(f"DL Regression — RMSE: {dl_reg_rmse:.2f}, R²: {dl_reg_r2:.4f}")5.2 · DL Classification — Predict High vs. Low Performer
tf.random.set_seed(42)
np.random.seed(42)
def build_classification_model(input_dim):
model = Sequential([
Dense(128, activation='relu', input_shape=(input_dim,)),
BatchNormalization(), Dropout(0.3),
Dense(64, activation='relu'), BatchNormalization(), Dropout(0.2),
Dense(32, activation='relu'), Dropout(0.1),
# KEY DIFFERENCE: sigmoid + binary_crossentropy
Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
return model
dl_clf_model = build_classification_model(X_clf_train_scaled.shape[1])
early_stop_clf = EarlyStopping(monitor='val_loss', patience=20, restore_best_weights=True, verbose=1)
reduce_lr_clf = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=10, min_lr=1e-6, verbose=1)
history_clf = dl_clf_model.fit(X_clf_train_scaled, y_clf_train, validation_split=0.2, epochs=200, batch_size=32, callbacks=[early_stop_clf, reduce_lr_clf], verbose=1)
dl_clf_prob = dl_clf_model.predict(X_clf_test_scaled).flatten()
dl_clf_pred = (dl_clf_prob >= 0.5).astype(int)
dl_clf_acc = accuracy_score(y_clf_test, dl_clf_pred)
dl_clf_f1 = f1_score(y_clf_test, dl_clf_pred)
print(f"DL Classification — Accuracy: {dl_clf_acc:.4f}, F1: {dl_clf_f1:.4f}")The final leaderboard.
Concatenate the ML results with the DL metrics and sort.
How to read the results table:
- For regression: higher R² is better. Lower RMSE is better (in GPA/score units — RMSE of 0.4 means “off by ~0.4 GPA points on average”).
- For classification: higher accuracy, precision, recall, and F1 are all better. Recall is especially important for dropout prevention — missing an at-risk student is worse than a false alarm.
The pattern you’ll see: On this education dataset, Gradient Boosting and Random Forest typically match or beat the neural network. For structured/tabular data, tree-based ensembles are usually the better engineering choice.
Key lesson: The R² value tells you how much of the variance in academic performance is explained by attendance, study hours, and other features. If R² is 0.5, it means 50% of the variance is explained — the rest is due to factors not in the dataset (motivation, mental health, family support, peer influence).
# Regression comparison
combined_reg = pd.concat([reg_results_df, pd.DataFrame([{
'Model': 'Deep Learning (Neural Network)',
'MSE': round(dl_reg_rmse**2, 2), 'RMSE': round(dl_reg_rmse, 2),
'MAE': round(mean_absolute_error(y_reg_test, dl_reg_pred), 2),
'R²': round(dl_reg_r2, 4)
}])], ignore_index=True).sort_values('R²', ascending=False)
print(combined_reg.to_string(index=False))
# Classification comparison
combined_clf = pd.concat([clf_results_df, pd.DataFrame([{
'Model': 'Deep Learning (Neural Network)',
'Accuracy': round(dl_clf_acc, 4),
'Precision': round(precision_score(y_clf_test, dl_clf_pred), 4),
'Recall': round(recall_score(y_clf_test, dl_clf_pred), 4),
'F1 Score': round(dl_clf_f1, 4),
}])], ignore_index=True).sort_values('Accuracy', ascending=False)
print(combined_clf.to_string(index=False))What the project
actually tells us.
Six findings from running the full pipeline on the Student Academic Performance & Dropout Risk dataset.
// attendance ↔ GPA positive · recall is keyAttendance strongly correlates with academic performance
The scatter plot and heatmap typically show that students with higher attendance tend to have higher GPAs. This is the strongest predictor in most education datasets.
Low performers are the dropout risk group
Students below the median GPA are the “at-risk” group. The classification model identifies these students — enabling early intervention before they drop out.
Multiple factors contribute to performance
Attendance, study hours, previous grades, and demographics all contribute. The correlation heatmap reveals which features matter most — and which don’t.
Tree ensembles match or beat DL on education data
Gradient Boosting and Random Forest typically outperform the neural network on this structured dataset. Deep learning isn’t always the answer — for tabular education data, tree-based ensembles win.
Recall is more important than precision for dropout prevention
Missing an at-risk student (false negative) is worse than a false alarm (false positive). When choosing a model, prioritize recall — the fraction of at-risk students you actually catch.
The real value is early warning, not perfect prediction
For schools, the goal isn’t to predict GPA perfectly — it’s to flag at-risk students early so interventions can happen. A model with 80% recall that catches 8 out of 10 at-risk students is genuinely useful for dropout prevention.
Frequently Asked Questions
What is the Student Academic Performance and Dropout Risk dataset? +
The Student Academic Performance and Dropout Risk dataset on Kaggle contains data about students’ academic records, attendance, demographics, and risk factors associated with dropping out of school.
Can machine learning predict student dropout risk? +
Yes. By training a model on attendance, grades, demographics, and behavioral features, ML learns the relationship between these factors and dropout risk, enabling early intervention for at-risk students.
Is deep learning better than ML for dropout prediction? +
Not necessarily. On this structured dataset, traditional ML models like Gradient Boosting typically match or outperform a 4-layer neural network. Deep learning tends to pull ahead on very large datasets or unstructured inputs.
Why is the R² not higher? +
Because academic performance is influenced by many factors not in the dataset — motivation, mental health, family dynamics, peer influence. A model that explains 30-60% of the variance is genuinely useful for early warning systems.
Run the project
yourself.
Grab the Student Academic Performance & Dropout Risk dataset from Kaggle, install the stack with one pip command, and run every cell end-to-end. Or click “Copy Full Project Code” at the top of this page.