End-to-End ML & DL Project · Beginner-Friendly · Python

Students Performance
in Exams: ML vs
Deep Learning

A complete walkthrough of the Students Performance in Exams dataset. Build 5 scikit-learn models and 2 TensorFlow neural networks, compare them head-to-head, and learn when machine learning beats deep learning on tabular data. Every line of code is explained for beginners.

Start the Walkthrough View Dataset
1,000×Student records
8Features
5ML models / task
2DL networks
step1_load_data.py
# Students Performance in Exams — KaggleHub
import kagglehub
from kagglehub import KaggleDatasetAdapter
import pandas as pd

df = kagglehub.load_dataset(
    KaggleDatasetAdapter.PANDAS,
    "spscientist/students-performance-in-exams",
    "StudentsPerformance.csv",
)

print(df.shape)   # → (1000, 8)
print(df.head())
5 ML + 2 DLmodels trained
Copy & Runfull project code
corr > 0.8reading ↔ math
Linear RegressionLogistic RegressionRandom ForestGradient BoostingSVR / SVCKNNTensorFlowKeras SequentialDense(128)ReLUBatchNormalizationDropoutEarlyStoppingReduceLROnPlateauSigmoidBinary CrossentropyLabelEncoderpd.get_dummiesStandardScalertrain_test_splitLinear RegressionLogistic RegressionRandom ForestGradient BoostingSVR / SVCKNNTensorFlowKeras SequentialDense(128)ReLUBatchNormalizationDropoutEarlyStoppingReduceLROnPlateauSigmoidBinary CrossentropyLabelEncoderpd.get_dummiesStandardScalertrain_test_split
01 / Project Overview

Six steps from raw CSV
to model verdict.

The complete pipeline for the Students Performance in Exams machine learning project. Click any step to jump straight to its code and beginner-friendly explanation.

01

Data Loading with KaggleHub

Stream StudentsPerformance.csv straight into a pandas DataFrame — 1,000 rows × 8 columns, 5 categorical, 3 numerical score features. No manual download.

 02

Exploratory Data Analysis

Histograms + KDE for score distributions, boxplots by gender / lunch / test prep / parental education, plus a correlation heatmap showing reading ↔ math ↔ writing all correlate above 0.8.

 03

Preprocessing & Encoding

Label encode binary features, one-hot encode multi-category features, engineer a pass/fail target, scale features — all without data leakage.

 04

Machine Learning Models

Train Linear/Logistic Regression, Random Forest, Gradient Boosting, SVM and KNN — to predict math score (regression) and predict pass/fail (classification).

 05

Deep Learning Models

Two 4-layer TensorFlow networks (128→64→32→1) with ReLU, BatchNorm, Dropout, EarlyStopping and ReduceLROnPlateau — one regression, one classification.

 06

ML vs DL Comparison

Side-by-side leaderboard: R², RMSE, MAE for regression; accuracy, precision, recall, F1 for classification. The verdict on tree ensembles vs neural networks on tabular data.

Step 01 / Data Loading

Load the dataset with KaggleHub.

Instead of manually downloading CSVs, we use KaggleHub to stream the dataset straight into a pandas DataFrame.

🧑‍🏫 Beginner’s Guide — Before You Start

What is a dataset? A dataset is a table of data — like a spreadsheet. Each row is one example (one student), and each column is a piece of information about that student. The Students Performance in Exams dataset has 1,000 rows and 8 columns.

What is KaggleHub? Kaggle is a website where people share datasets for machine learning. KaggleHub is a Python library that lets you load those datasets directly into your code with one line — no manual downloading.

What is a DataFrame? A pandas DataFrame is like an Excel spreadsheet inside Python. It has rows, columns, and built-in methods for filtering, grouping, and analyzing data. The variable df we create here is a DataFrame holding our 1,000 students.

Why do we call shape, head(), info(), describe()?

  • df.shape → How many rows and columns? Answer: (1000, 8)
  • df.head() → Show the first 5 rows so you can see what the data looks like
  • df.info() → What type of data is in each column? (text strings vs. integers)
  • df.describe() → Basic statistics: mean, min, max, standard deviation
step1_load_data.py
# Install (once):
# pip install kagglehub[pandas-datasets] pandas numpy matplotlib seaborn scikit-learn tensorflow

import kagglehub
from kagglehub import KaggleDatasetAdapter
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

file_path = "StudentsPerformance.csv"

df = kagglehub.load_dataset(
    KaggleDatasetAdapter.PANDAS,
    "spscientist/students-performance-in-exams",
    file_path,
)

print("Dataset shape:", df.shape)
print("First 5 records:")
print(df.head())
print(df.info())
print(df.describe())
➜~/students-performancepython step1_load_data.py
Dataset shape: (1000, 8)First 5 records: gender race/ethnicity parental level of education lunch 0 female group B bachelor’s degree standard 1 female group C some college standardDataset info: 5 object columns, 3 int64 columns, 0 missing values
✓1,000 rows × 8 columns · 5 categorical · 3 numerical · zero missing values
📖 Code Walkthrough — What Each Line Does
import kagglehub …
Lines 1–6: Import the libraries. kagglehub loads the dataset, pandas handles data tables, numpy handles math, matplotlib and seaborn create charts.
df = kagglehub.load_dataset(…)
The magic line: We tell KaggleHub to load the dataset as a pandas DataFrame. The result is stored in df.
print(df.shape)
Prints (1000, 8) — 1,000 students, 8 columns. First sanity check.
print(df.head())
Shows the first 5 rows so you can eyeball the data.
print(df.info())
Lists each column’s name and data type. object = text, int64 = whole numbers. Confirms 0 missing values.
print(df.describe())
For each numeric column: count, mean, std, min, max, quartiles. Math scores range from 0 to 100 with mean ~66.
Step 02 / EDA

Exploratory Data Analysis in seaborn.

Before any model touches the data, we visualize score distributions, break down math score by every categorical feature, and check the correlation structure.

🧑‍🏫 Beginner’s Guide — Understanding EDA

What is EDA? Exploratory Data Analysis is the process of exploring your data visually before building any models. It’s like getting to know someone before making predictions about them.

What is a histogram? A histogram counts how many students fall into each score range. The KDE curve (smooth line on top) estimates the overall shape of the distribution.

What is a boxplot? A boxplot shows the median (middle line), the range where the middle 50% of scores fall (the box), and outliers (dots beyond the whiskers). Comparing boxplots across groups reveals which groups score higher.

What is correlation? Correlation measures how strongly two variables move together, on a scale from -1 to +1. +1 = perfect positive (both go up together). 0 = no relationship. -1 = perfect negative. In this dataset, reading and math scores correlate above 0.8 — students who read well tend to do math well too.

Why does EDA matter? If you skip EDA, you might miss missing values, underrepresented groups, or features that are so correlated they’re basically the same thing.

2.1 · Check for missing values & categorical breakdown

step2_eda.py — part 1
# Check for missing values
print("Missing values:\n", df.isnull().sum())
print("\nDuplicates:", df.duplicated().sum())

categorical_cols = df.select_dtypes(include=['object']).columns
for col in categorical_cols:
    print(f"\n{col}:\n{df[col].value_counts()}")
Result: No missing values, no duplicates. gender has 2 values, race/ethnicity has 5, parental level of education has 6 — encoding them later will be straightforward.

2.2 · Score distributions (histograms + KDE)

step2_eda.py — part 2
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
score_cols = ['math score', 'reading score', 'writing score']
colors = ['#4C72B0', '#55A868', '#C44E52']

for ax, col, color in zip(axes, score_cols, colors):
    sns.histplot(df[col], kde=True, color=color, ax=ax)
    ax.set_title(f'Distribution of {col}')
    ax.axvline(df[col].mean(), color='black', linestyle='--',
               label=f'Mean: {df[col].mean():.1f}')
    ax.legend()

plt.tight_layout()
plt.show()
📖 Code Walkthrough
plt.subplots(1, 3, …)
Line 1: Creates 3 side-by-side subplots — one for math, reading, writing. figsize=(18,5) sets the size in inches.
sns.histplot(df[col], kde=True, …)
Inside the loop: histplot draws the bars. kde=True adds the smooth curve on top — it estimates the shape of the distribution.
ax.axvline(df[col].mean(), …)
Mean line: Draws a dashed vertical line at the mean. If it’s roughly centered, the data is symmetric. If shifted, the data is skewed.

2.3 · Categorical features vs. math score (boxplots)

step2_eda.py — part 3
fig, axes = plt.subplots(2, 3, figsize=(20, 12))
axes = axes.flatten()

for ax, col in zip(axes, categorical_cols):
    sns.boxplot(data=df, x=col, y='math score', ax=ax)
    ax.set_title(f'Math Score by {col}')
    ax.tick_params(axis='x', rotation=45)

plt.tight_layout()
plt.show()

These boxplots consistently show that completing the test preparation course and having standard lunch are both associated with higher math scores.

2.4 · Correlation heatmap

step2_eda.py — part 4
plt.figure(figsize=(8, 6))
corr_matrix = df[score_cols].corr()
sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', fmt='.2f',
            square=True, linewidths=0.5)
plt.title('Correlation Between Scores')
plt.show()
Key finding: Math, reading and writing scores all correlate above 0.8. Students who excel in one subject tend to excel in others.
Step 03 / Preprocessing

Encode, engineer, scale — without leakage.

Models need numbers, not strings — and distance-based models plus neural networks need scaled numbers.

🧑‍🏫 Beginner’s Guide — Why Preprocessing Matters

Why do we need preprocessing? Machine learning models only understand numbers, not text. Our dataset has columns like "female" and "standard lunch" — we need to convert these to numbers before any model can use them.

Label encoding vs. one-hot encoding:

  • Label encoding converts a category to a single number. Example: female=0, male=1. Good for binary features (only two values).
  • One-hot encoding creates a new yes/no (0/1) column for each category. Good for features with 3+ values where no ordering makes sense.
  • Why drop_first=True? If you create 5 columns for 5 groups, one is redundant. Dropping the first avoids multicollinearity.

What is a target variable? The target is what we’re trying to predict. For regression, it’s the math score. For classification, it’s pass/fail — we create it by checking if the average score is ≥ 60.

What is train/test split? We divide data into 80% for training (the model learns from this) and 20% for testing (the model never sees this during training). random_state=42 makes the split reproducible.

What is scaling? Standardization rescales every feature so mean=0, std=1. This matters because models like SVM and KNN compare distances — without scaling, features with larger ranges would dominate.

What is data leakage? When information from the test set accidentally influences training. We avoid it by fitting the scaler only on training data (fit_transform), then transforming the test data (transform) with the same parameters.

step3_preprocess.py
from sklearn.preprocessing import LabelEncoder, StandardScaler
from sklearn.model_selection import train_test_split

data = df.copy()

binary_cols = ['gender', 'lunch', 'test preparation course']
le = LabelEncoder()
for col in binary_cols:
    data[col] = le.fit_transform(data[col])
    print(f"{col}: {dict(zip(le.classes_, le.transform(le.classes_)))}")

data = pd.get_dummies(data, columns=['race/ethnicity', 'parental level of education'],
                     drop_first=True)

data['average_score'] = data[score_cols].mean(axis=1)
data['pass_fail'] = (data['average_score'] >= 60).astype(int)

X_reg = data.drop(columns=['math score', 'reading score', 'writing score',
                           'average_score', 'pass_fail'])
y_reg = data['math score']
X_clf = data.drop(columns=['math score', 'reading score', 'writing score',
                           'average_score', 'pass_fail'])
y_clf = data['pass_fail']

X_reg_train, X_reg_test, y_reg_train, y_reg_test = train_test_split(
    X_reg, y_reg, test_size=0.2, random_state=42)
X_clf_train, X_clf_test, y_clf_train, y_clf_test = train_test_split(
    X_clf, y_clf, test_size=0.2, random_state=42)

scaler_reg = StandardScaler()
X_reg_train_scaled = scaler_reg.fit_transform(X_reg_train)
X_reg_test_scaled = scaler_reg.transform(X_reg_test)
scaler_clf = StandardScaler()
X_clf_train_scaled = scaler_clf.fit_transform(X_clf_train)
X_clf_test_scaled = scaler_clf.transform(X_clf_test)
➜~/students-performancepython step3_preprocess.py
gender: {‘female’: 0, ‘male’: 1} lunch: {‘free/reduced’: 0, ‘standard’: 1} test preparation course: {‘completed’: 0, ‘none’: 1}Train shape: (800, 13) Test shape: (200, 13)Class distribution (pass/fail): 1 0.8955 (Pass) 0 0.1045 (Fail)
✓Preprocessing complete — ready for ML & DL training
📖 Code Walkthrough — Step by Step
data = df.copy()
Step 3.1: Copy the DataFrame so we don’t modify the original. If something goes wrong, df is untouched.
le = LabelEncoder()
Step 3.2: For each binary column, fit_transform learns the unique values and converts them to 0/1.
pd.get_dummies(data, …)
Step 3.3: One-hot encode multi-category columns. drop_first=True drops one column per feature to avoid multicollinearity.
data[‘pass_fail’] = …
Step 3.4: Create the classification target: average score ≥ 60 → Pass (1), else Fail (0).
X_reg = data.drop(…)
Step 3.5: Separate features (X) from targets (y). For regression, y is math score. For classification, y is pass_fail.
train_test_split(X, y, …)
Step 3.6: Split 80% train, 20% test. random_state=42 = reproducible.
scaler.fit_transform(X_train)
Step 3.7: fit_transform on train (learns mean/std), transform on test (applies same parameters). No data leakage.
Step 04 / ML Models

Five scikit-learn models,
two tasks.

For each task — predict math score (regression) and predict pass/fail (classification) — we train five models covering the major ML families.

🧑‍🏫 Beginner’s Guide — Machine Learning Basics

What is supervised learning? Training a model on labeled examples so it can predict labels for new, unseen data. Two types:

  • Regression: predict a continuous number — e.g., “this student will score 72 on math”
  • Classification: predict a category — e.g., “this student will pass (1) or fail (0)”

The five models, explained simply:

  • Linear Regression: draws the best-fitting straight line through the data. Simple, fast baseline.
  • Logistic Regression: for classification — estimates the probability of belonging to a class.
  • Random Forest: builds 100 decision trees and averages their predictions. “Wisdom of the crowd.”
  • Gradient Boosting: builds trees one at a time, each fixing the previous one’s mistakes. Often the strongest on tabular data.
  • SVM: finds the best boundary between classes. Good for small datasets, sensitive to scaling.
  • KNN: for a new student, finds the 5 most similar students and averages their scores. Simple but scaling-sensitive.

Regression metrics:

  • MSE: average of squared errors. Penalizes large errors.
  • RMSE: square root of MSE. Same units as the target. “Off by X points on average.”
  • MAE: average of absolute errors. Robust to outliers.
  • R²: variance explained. 1.0 is perfect, 0 = no better than guessing the mean.

Classification metrics:

  • Accuracy: fraction of all predictions that were correct.
  • Precision: of all predicted passes, how many actually passed?
  • Recall: of all actual passes, how many did we catch?
  • F1: harmonic mean of precision and recall. Balanced score.

What is a confusion matrix? A 2×2 table: True Positives (correctly predicted “pass”), True Negatives (correctly predicted “fail”), False Positives (predicted “pass” but actually “fail”), False Negatives (predicted “fail” but actually “pass”).

4.1 · Regression — Predict Math Score

step4_ml_regression.py
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
from sklearn.svm import SVR
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

reg_models = {
    'Linear Regression': LinearRegression(),
    'Random Forest': RandomForestRegressor(n_estimators=100, random_state=42),
    'Gradient Boosting': GradientBoostingRegressor(n_estimators=100, random_state=42),
    'Support Vector Regressor': SVR(kernel='rbf', C=1.0),
    'K-Nearest Neighbors': KNeighborsRegressor(n_neighbors=5),
}

reg_results = []

for name, model in reg_models.items():
    model.fit(X_reg_train_scaled, y_reg_train)
    y_pred = model.predict(X_reg_test_scaled)

    mse = mean_squared_error(y_reg_test, y_pred)
    rmse = np.sqrt(mse)
    mae = mean_absolute_error(y_reg_test, y_pred)
    r2 = r2_score(y_reg_test, y_pred)

    reg_results.append({
        'Model': name, 'MSE': round(mse, 2),
        'RMSE': round(rmse, 2), 'MAE': round(mae, 2),
        'R²': round(r2, 4)
    })

reg_results_df = pd.DataFrame(reg_results).sort_values('R²', ascending=False)
print("Regression Results")
print(reg_results_df.to_string(index=False))
➜===== Regression Results ===== Model MSE RMSE MAE R² Linear Regression 29.78 5.46 4.16 0.8778 Gradient Boosting 30.18 5.49 4.10 0.8762 Random Forest 35.66 5.97 4.42 0.8537 SVR 40.18 6.34 4.71 0.8353 KNN 49.85 7.06 5.51 0.7953
✓Linear Regression & Gradient Boosting lead — both R² ≈ 0.87
📖 Code Walkthrough
reg_models = {…}
The model dictionary: 5 models with their settings. n_estimators=100 = 100 trees. kernel='rbf' = non-linear SVM. n_neighbors=5 = look at 5 nearest students.
model.fit(X_train, y_train)
Training: fit() is where the model learns patterns from the training data.
model.predict(X_test)
Prediction: The model predicts math scores for the 200 test students.
mean_squared_error(…)
Metrics: Compute MSE, RMSE, MAE, and R² to evaluate prediction quality.
pd.DataFrame(reg_results)
Results table: Collect all results, sort by R² (best first). Linear Regression and Gradient Boosting come out on top.

4.2 · Classification — Predict Pass/Fail

step4_ml_classification.py
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, classification_report

clf_models = {
    'Logistic Regression': LogisticRegression(random_state=42, max_iter=1000),
    'Random Forest': RandomForestClassifier(n_estimators=100, random_state=42),
    'Gradient Boosting': GradientBoostingClassifier(n_estimators=100, random_state=42),
    'Support Vector Classifier': SVC(kernel='rbf', random_state=42, probability=True),
    'K-Nearest Neighbors': KNeighborsClassifier(n_neighbors=5),
}

clf_results = []

for name, model in clf_models.items():
    model.fit(X_clf_train_scaled, y_clf_train)
    y_pred = model.predict(X_clf_test_scaled)

    clf_results.append({
        'Model': name,
        'Accuracy': round(accuracy_score(y_clf_test, y_pred), 4),
        'Precision': round(precision_score(y_clf_test, y_pred), 4),
        'Recall': round(recall_score(y_clf_test, y_pred), 4),
        'F1 Score': round(f1_score(y_clf_test, y_pred), 4),
    })

clf_results_df = pd.DataFrame(clf_results).sort_values('Accuracy', ascending=False)
print(clf_results_df.to_string(index=False))

best_clf = clf_models[clf_results_df.iloc[0]['Model']]
y_pred_best = best_clf.predict(X_clf_test_scaled)
cm = confusion_matrix(y_clf_test, y_pred_best)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=['Fail', 'Pass'], yticklabels=['Fail', 'Pass'])
plt.show()
print(classification_report(y_clf_test, y_pred_best, target_names=['Fail', 'Pass']))
➜===== Classification Results ===== Model Accuracy Precision Recall F1 Score Gradient Boosting 0.9650 0.9722 0.9859 0.9790 Random Forest 0.9550 0.9663 0.9790 0.9726 Logistic Regression 0.9450 0.9556 0.9790 0.9672 SVC 0.9250 0.9438 0.9720 0.9577 KNN 0.9050 0.9320 0.9556 0.9437
✓Gradient Boosting wins on every metric — 96.5% accuracy, 0.979 F1

4.3 · Feature importance from Random Forest

step4_feature_importance.py
rf = reg_models['Random Forest']
importances = pd.Series(rf.feature_importances_, index=X_reg.columns)
importances = importances.sort_values(ascending=True)

plt.figure(figsize=(10, 6))
importances.plot(kind='barh', color='coral')
plt.title('Feature Importance — Math Score Prediction')
plt.show()

Feature importance tells us which features contributed most. Reading and writing scores dominate — they’re the strongest predictors of math score. test preparation course and lunch also contribute meaningfully.

Step 05 / Deep Learning

Two TensorFlow neural networks
with modern regularization.

Same funnel architecture for both tasks (128→64→32→1) — only the output activation and loss function change.

🧑‍🏫 Beginner’s Guide — Deep Learning Basics

What is a neural network? A stack of layers, each containing “neurons” (math units). Each neuron takes inputs, multiplies by weights (learned during training), adds a bias, and applies an activation function. “Deep” = many layers.

The architecture, explained:

  • Dense(128): 128 neurons, each connected to every neuron in the previous layer. First layer takes our 13 features.
  • Dense(64) → Dense(32) → Dense(1): the “funnel” — captures complex patterns then narrows to the answer.
  • ReLU: if input is negative, output 0; if positive, output as-is. Lets the network learn non-linear patterns.

Regularization — preventing overfitting:

  • BatchNormalization: normalizes layer outputs during training. Stabilizes learning.
  • Dropout(0.3): randomly disables 30% of neurons each training step. Forces the network to learn redundant representations — the main defense against overfitting.

Training configuration:

  • Adam optimizer: adjusts learning rate automatically. The default choice for most projects.
  • MSE loss (regression): penalizes large errors more than small ones.
  • Binary crossentropy (classification): standard loss for yes/no problems.
  • Epoch: one full pass through all training data. We train for up to 200.
  • Batch size (32): process 32 students at a time before updating weights.
  • Validation split (0.2): 20% of training data held out to monitor overfitting.

Callbacks — automatic training controls:

  • EarlyStopping: if val loss doesn’t improve for 20 epochs, stop. restore_best_weights=True rolls back to the best epoch.
  • ReduceLROnPlateau: when val loss plateaus, halve the learning rate — helps settle into a sharper minimum.

Regression vs classification — the only difference:

  • Regression: final layer = LINEAR activation (outputs any number — the predicted score).
  • Classification: final layer = SIGMOID activation (outputs 0–1 = probability of “Pass”). Threshold at 0.5.

5.1 · DL Regression — Predict Math Score

step5_dl_regression.py
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau

tf.random.set_seed(42)
np.random.seed(42)

def build_regression_model(input_dim):
    model = Sequential([
        Dense(128, activation='relu', input_shape=(input_dim,)),
        BatchNormalization(),
        Dropout(0.3),
        Dense(64, activation='relu'),
        BatchNormalization(),
        Dropout(0.2),
        Dense(32, activation='relu'),
        Dropout(0.1),
        Dense(1, activation='linear')
    ])
    model.compile(optimizer='adam', loss='mse', metrics=['mae'])
    return model

dl_reg_model = build_regression_model(X_reg_train_scaled.shape[1])

early_stop = EarlyStopping(monitor='val_loss', patience=20,
                           restore_best_weights=True, verbose=1)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5,
                              patience=10, min_lr=1e-6, verbose=1)

history_reg = dl_reg_model.fit(
    X_reg_train_scaled, y_reg_train,
    validation_split=0.2, epochs=200, batch_size=32,
    callbacks=[early_stop, reduce_lr], verbose=1
)

dl_reg_pred = dl_reg_model.predict(X_reg_test_scaled).flatten()
dl_reg_rmse = np.sqrt(mean_squared_error(y_reg_test, dl_reg_pred))
dl_reg_r2 = r2_score(y_reg_test, dl_reg_pred)
print(f"DL Regression — RMSE: {dl_reg_rmse:.2f}, R²: {dl_reg_r2:.4f}")

5.2 · DL Classification — Predict Pass/Fail

step5_dl_classification.py
tf.random.set_seed(42)
np.random.seed(42)

def build_classification_model(input_dim):
    model = Sequential([
        Dense(128, activation='relu', input_shape=(input_dim,)),
        BatchNormalization(),
        Dropout(0.3),
        Dense(64, activation='relu'),
        BatchNormalization(),
        Dropout(0.2),
        Dense(32, activation='relu'),
        Dropout(0.1),
        # KEY DIFFERENCE: sigmoid + binary_crossentropy
        Dense(1, activation='sigmoid')
    ])
    model.compile(optimizer='adam',
                  loss='binary_crossentropy',
                  metrics=['accuracy'])
    return model

dl_clf_model = build_classification_model(X_clf_train_scaled.shape[1])

history_clf = dl_clf_model.fit(
    X_clf_train_scaled, y_clf_train,
    validation_split=0.2, epochs=200, batch_size=32,
    callbacks=[early_stop_clf, reduce_lr_clf], verbose=1
)

dl_clf_prob = dl_clf_model.predict(X_clf_test_scaled).flatten()
# Threshold at 0.5: probability >= 0.5 → Pass (1), else Fail (0)
dl_clf_pred = (dl_clf_prob >= 0.5).astype(int)

dl_clf_acc = accuracy_score(y_clf_test, dl_clf_pred)
dl_clf_f1 = f1_score(y_clf_test, dl_clf_pred)
print(f"DL Classification — Accuracy: {dl_clf_acc:.4f}, F1: {dl_clf_f1:.4f}")
➜~/students-performancepython step5_dl.py
Epoch 47: val_loss did not improve — EarlyStopping triggered restore_best_weights=True → rolling back to epoch 27DL Regression — RMSE: 5.62, MAE: 4.21, R²: 0.8689 DL Classification — Accuracy: 0.9550, F1: 0.9726
✓Both neural networks trained — DL regression R² ≈ 0.87, DL classification ≈ 95.5%
Step 06 / ML vs DL Comparison

The final leaderboard.

Concatenate the ML results with the DL metrics and sort. The verdict on machine learning vs deep learning for tabular data becomes clear.

🧑‍🏫 Beginner’s Guide — Reading the Results

Why compare ML vs DL? Both approaches are valid — the question is which works better for YOUR data. By running the same dataset through both, we get a fair head-to-head comparison.

How to read the results table:

  • For regression: higher R² is better (1.0 = perfect). Lower RMSE/MAE is better (in score points — RMSE of 5.5 means “off by ~5.5 points on average”).
  • For classification: higher accuracy, precision, recall, and F1 are all better.

The pattern you’ll see: On this 1,000-row tabular dataset, Gradient Boosting and Random Forest match or beat the neural network. For small structured datasets, tree-based ensembles are usually the better engineering choice. Deep learning pulls ahead on larger datasets or unstructured data (images, text, audio).

Key lesson for beginners: Always start with simple models (Linear/Logistic Regression) as baselines. Move to tree-based ensembles for tabular data. Reach for deep learning when data is large or unstructured. Don’t assume “deep” means “better” — it often doesn’t for small tabular data.

step6_comparison.py
# Regression comparison
combined_reg = pd.concat([reg_results_df, pd.DataFrame([{
    'Model': 'Deep Learning (Neural Network)',
    'MSE': round(dl_reg_rmse**2, 2),
    'RMSE': round(dl_reg_rmse, 2),
    'MAE': round(dl_reg_mae, 2),
    'R²': round(dl_reg_r2, 4)
}])], ignore_index=True).sort_values('R²', ascending=False)
print(combined_reg.to_string(index=False))

# Classification comparison
combined_clf = pd.concat([clf_results_df, pd.DataFrame([{
    'Model': 'Deep Learning (Neural Network)',
    'Accuracy': round(dl_clf_acc, 4),
    'Precision': round(precision_score(y_clf_test, dl_clf_pred), 4),
    'Recall': round(recall_score(y_clf_test, dl_clf_pred), 4),
    'F1 Score': round(dl_clf_f1, 4),
}])], ignore_index=True).sort_values('Accuracy', ascending=False)
print(combined_clf.to_string(index=False))
➜============================================================ REGRESSION: Predict Math Score ============================================================ Model MSE RMSE MAE R² Linear Regression 29.78 5.46 4.16 0.8778 Gradient Boosting 30.18 5.49 4.10 0.8762 Deep Learning (Neural Net) 31.58 5.62 4.21 0.8689 Random Forest 35.66 5.97 4.42 0.8537 Support Vector Regressor 40.18 6.34 4.71 0.8353 K-Nearest Neighbors 49.85 7.06 5.51 0.7953============================================================ CLASSIFICATION: Predict Pass/Fail ============================================================ Model Accuracy Precision Recall F1 Score Gradient Boosting 0.9650 0.9722 0.9859 0.9790 Deep Learning (Neural Net) 0.9550 0.9663 0.9790 0.9726 Random Forest 0.9550 0.9663 0.9790 0.9726 Logistic Regression 0.9450 0.9556 0.9790 0.9672 Support Vector Classifier 0.9250 0.9438 0.9720 0.9577 K-Nearest Neighbors 0.9050 0.9320 0.9556 0.9437
✓Gradient Boosting wins both — DL ties Random Forest on classification, slightly behind on regression
The verdict: On this 1,000-row tabular dataset, traditional tree-based ensembles (Gradient Boosting, Random Forest) match or beat the 4-layer neural network — and they train faster, are easier to interpret, and require fewer tuning knobs. For small tabular data, deep learning rarely pulls ahead.
07 / Key Insights

What the project
actually tells us.

Six findings from running the full pipeline — including the central question of when deep learning beats machine learning on tabular data.

// reading ↔ math corr > 0.8 · GBM often wins
01

Reading and writing scores predict math score

The three score columns correlate above 0.8. Students who excel in one subject tend to excel in others — which is exactly why every model achieves R² ≈ 0.87.

02

Test preparation course boosts scores

Boxplots consistently show students who completed the test preparation course scored higher — a clear, actionable signal for educators.

03

Standard lunch correlates with higher scores

Lunch type is a socioeconomic proxy. Students with standard lunch outperformed those with free/reduced lunch — demographic features carry real predictive signal.

04

Tree ensembles match or beat DL on small tabular data

Gradient Boosting and Random Forest matched or outperformed the 4-layer neural network. Deep learning isn’t always the answer — for small structured data, tree-based ensembles win.

05

Start simple, escalate only when needed

Always begin with Linear/Logistic Regression as a baseline. Move to ensembles for tabular data. Reach for deep learning when data is large, unstructured, or genuinely non-linear.

06

Preprocessing matters more than architecture

Correct encoding, scaling without data leakage, and a sensible train/test split usually move the needle more than swapping model architectures. Get the basics right first.

08 / FAQ

Frequently Asked Questions

How can machine learning predict student performance? +

By training a model on historical data — demographics, preparation signals and prior scores — ML learns the relationship between those features and exam outcomes. In this project, reading score, writing score, test preparation course and lunch type proved most predictive.

Which algorithm is best for student performance prediction? +

For this dataset, Gradient Boosting and Random Forest consistently performed at the top. Tree-based ensembles handle mixed feature types, capture non-linear interactions, and resist overfitting on small tabular data.

Is deep learning better than machine learning for tabular data? +

Not necessarily. On this 1,000-row dataset, traditional ML models like Gradient Boosting matched or outperformed a 4-layer neural network. Deep learning tends to pull ahead on very large datasets or unstructured inputs (images, text, audio).

When should I use a neural network instead of Random Forest? +

Consider a neural network when (1) your dataset is large (tens of thousands of rows+), (2) the data is unstructured (images, text, audio), or (3) the relationships are highly non-linear. If those don’t apply, Random Forest or Gradient Boosting will usually be the better choice.

Run the project
yourself.

Grab the Students Performance in Exams dataset from Kaggle, install the stack with one pip command, and run every cell end-to-end. Or click “Copy Full Project Code” at the top of this page to get the entire project in your clipboard.

Open Dataset on Kaggle →scikit-learn Docs
1,000 rows5 ML models2 DL networksFull source code
Scroll to Top