Operations support • Delayed = ≥ 1 minute late

Predicting Bus Delays in Luxembourg

Find when and where delays are most likely, so operators can adjust service and improve reliability.

Plan • What you will see • Main score: F1

Agenda

A simple path: problem → data → patterns → model → what to do with it.

1) Problem and goal Why delays matter, who benefits, and what “delayed” means (≥ 1 minute)
2) Data and checks Where the data comes from and how I checked if it is usable
3) What I learned (EDA) When and where delays happen more often (time, routes, stops, weather)
4) Modeling How I built the model and how I tested it fairly
5) Results and next steps Best model, trade-offs, and practical recommendations
Target: Delayed vs On-time (≥ 1 minute) Main score: F1 Also shown: Precision / Recall / ROC AUC

Quick facts

  • Target: is_delayed (Delayed ≥ 1 minute)
  • Delay rate: about 36% delayed overall
  • Dataset size:7.36M stop events
  • Data window: Jul 25 – Sep 16, 2025 (collected_at)

Time plan (20 minutes)

Problem
2m
Data
4m
EDA
7m
Model
6m
Wrap-up
1m
Problem • Impact • Objective

Why bus delays matter (and what I am trying to solve)

Even small delays can break connections and reduce trust. The goal is to support operations with an early warning signal.

The problem

  • Buses do not always arrive on time.
  • Passengers feel it quickly: missed connections, longer trips, and less confidence in the service.
  • Operations teams often learn about issues after delays already build up.

I want a simple, useful way to spot “higher risk” situations early.

Why this is important

  • Passenger experience: reliability matters as much as speed.
  • Operations efficiency: better signals help prioritize attention.
  • Planning: knowing patterns helps adjust schedules and resources.

Project objective

  • Predict whether a stop event will be Delayed or On-time.
  • Delayed means ≥ 1 minute late.
  • Output is a probability so it can be used as a risk score, not just “yes/no”.

Think of it as a “heads up” for dispatch and planning teams.

What success looks like

  • Main score: F1 (balances false alarms and missed delays).
  • I also show Precision, Recall, and ROC AUC.
  • Practical value: clear patterns and a simple threshold policy that ops can use.

The goal is not “perfect prediction”. The goal is a model that helps make better decisions.

Data • Where it comes from • How I prepared it

Data source and workflow

One cleaned dataset is used for both analysis and modeling, so the numbers always match.

What data I use

  • Stop events: scheduled time vs actual time → delay in minutes
  • Extra context: signals like weather to capture “bad conditions”

Important rule: I keep one cleaned dataset so EDA and modeling are consistent.

Why this matters

  • It prevents “EDA says one thing, model uses another dataset”.
  • It’s easier to explain and defend: every number has a clear source.
  • It makes the project repeatable (anyone can re-run the same steps).
Same data everywhere
Easy to reproduce
Ready for ops use
Workflow at a glance

Get the data

Extract with SQL and export a CSV for analysis.

Clean and check

Fix types, check missing values, and review extreme delays.

Explore patterns (EDA)

Look at delays by hour, day, route/stop, and conditions.

Create features

Build useful columns (time bands, route context, weather flags).

Train and evaluate

Compare models using F1, plus Precision/Recall and ROC AUC.

Dataset overview • One row = one stop event

Dataset schema and scale

Each row is one bus stop event with a scheduled time and an observed time. With millions of rows, I can spot patterns and train models with confidence.

Column Type Meaning
stop_id categorical Stop identifier (where the bus event happened)
trip_id categorical Trip identifier (specific run of a bus line)
planned_time datetime Scheduled arrival/departure time at the stop
real_time datetime Observed arrival/departure time at the stop
delay_minutes numeric Difference between real and planned time (minutes)
collected_at datetime Timestamp when the observation was recorded
operator categorical Operator/company running the service (some missing values)
transport_type categorical Transport mode label (bus category in this dataset)
route_id categorical Internal route identifier
route_short_name categorical Public-facing line number/name
Modeling label: Delayed = (delay_minutes ≥ 1), otherwise On-time.
Scale 7,356,700 stop events Enough volume for stable EDA and modeling
Columns 10 core fields Time + route/stop + operator context
Missingness Operator: 6,059 missing (~0.08%) Filled as “Unknown” so I keep all rows

Why this dataset is useful for operations

  • Time coverage (collected_at): July 25, 2025 → September 16, 2025 (about 7–8 weeks)
  • I can compare reliability by time (hour, weekday, peak periods)
  • I can compare reliability by network (route and stop)
  • I can compare reliability by operator (different service providers)
Target • Class balance • Why F1 matters

What I predict: delayed vs on-time (and how common each one is)

For each stop event, the model predicts one of two outcomes: Delayed or On-time.

Target (label) definition

  • Delayed = bus is 1 minute late or more
  • On-time = bus is less than 1 minute late

I use 1 minute as a clear and simple rule that matches how I talk about reliability.

Why the class balance matters

  • Most events are On-time (~64%).
  • A large part are still Delayed (~36%).
  • If I only used accuracy, I could get a “good” score by guessing On-time too often.
  • So I use F1 as the main score: it balances catching delays and avoiding too many false alarms.

I also report Precision, Recall, and ROC AUC to give extra context.

Baseline delayed share 35.98% delayed On-time: 64.02%
Chart source: value counts of is_delayed (Delayed ≥ 1 minute).
Data quality • Delay sizes • Rare extreme events

Most delays are small, but a few are very large

Before modeling, I check missing values and I look at the delay distribution to understand what “normal” looks like.

Main messages

  • Data is mostly complete (no major missing-value problem).
  • Typical delays are short: most are close to 0–2 minutes.
  • Percentiles help summarize this safely: P95 ≈ 5 min and P99 ≈ 9 min.
  • There is also a rare tail (15+ minutes). These look like disruptions and should be monitored separately.
Check What I saw Why it matters
Missing values Very small I can compare routes/operators without big data gaps.
Typical delay size P95 ≈ 5 min, P99 ≈ 9 min Most delays are minor, so small operational fixes can help many trips.
Extreme delays Rare but can be huge (chart capped at 75+) Averages can be misleading; treat rare extremes as disruption cases.
Target rule Delayed = ≥ 1 minute Keeps EDA and modeling aligned and easy to explain.
On the right: the top chart shows the common range (0–15 min). The bottom chart zooms into rare long delays (15+ min).
Top: 0–15 min with P95/P99 markers. Bottom: >15 min tail, capped for readability.
EDA • Time of day • When delays happen more

Some hours have more delays than others

This chart shows the share of stop events that are delayed for each planned hour.

How to read the chart

  • Each point = one hour of the day (00 to 23).
  • Y-axis = % of events delayed (Delayed = ≥ 1 minute).
  • The dashed line is the baseline (overall average).
  • Points above the dashed line = higher-than-usual risk.

What I learn

  • Delay risk is higher in the morning and late afternoon (commute hours).
  • The chart marks the top risk hours (the red dots).
  • This is useful for operations: focus attention on these hours first.

Think of this as a simple “when should I watch more closely?” signal.

Metric shown: delayed share = mean(is_delayed) by planned hour.
The dashed line is the baseline delayed share (overall average). Red points mark the highest-risk hours.
EDA • Weekday vs weekend • Time bands

Weekends are slightly better, and evening peak is the riskiest band

Same metric as before: delayed share (Delayed = ≥ 1 minute). Dashed line = overall baseline.

What I see

  • Weekdays are stable: Mon–Fri stay close to the baseline (around mid-30%).
  • Weekend is lower: Saturday and especially Sunday have fewer delays.
  • Time bands show bigger gaps: night is the lowest, and evening peak is the highest.

Why this matters

  • If you can only focus on one window, focus on evening peak first.
  • Weekday differences are small, so the bigger lever is time-of-day, not “which weekday”.
  • These patterns become useful features later: weekday and time band.
Charts show delayed share = mean(is_delayed).
Dashed line = baseline delayed share (overall).
EDA • Routes • Where delays happen more

Some routes have more delays than others

I rank routes by delayed share (Delayed = ≥ 1 minute). I only keep routes with many events so the ranking is fair.

What this shows

  • Worst routes: routes with the highest % of delayed events.
  • Best routes: routes with the lowest % of delayed events (among high-volume routes).
  • The dashed line is the baseline (overall system average).

How to use this

  • Start with the worst routes: they are good candidates for a quick operational check.
  • Possible reasons: traffic, long boarding time, difficult segments, or peak-hour pressure.
  • Next slide (stops) helps pinpoint where the route problem is happening.

This creates a short “priority list” instead of trying to fix everything at once.

Ranking metric: delayed share = mean(is_delayed). I filter to high-volume routes to avoid noisy results.
Tip: low-volume routes are excluded so the “top/bottom” results are not driven by small samples.
EDA • Stop hotspots • Where delays happen most

Some stops are much “more likely to be delayed” than others

I rank stops by delayed share (Delayed = ≥ 1 minute). I only keep stops with many events so the ranking is fair.

What this shows

  • Worst stops: stops with the highest % of delayed events.
  • Best stops: stops with the lowest % of delayed events (among high-volume stops).
  • The dashed line is the baseline (overall system average).

How an operator can use this

  • Start with the worst stops: they are good candidates for a quick field check.
  • Look for common causes: heavy boarding, traffic lights, intersections, road works, school zones.
  • Compare worst vs best to learn what “good performance” looks like.

This slide is meant to create a short, practical “investigation list”.

Ranking metric: delayed share = mean(is_delayed). I filter to high-volume stops to avoid noisy results.
Dashed line = overall baseline delayed share (system average).
EDA • Operator view • Useful signal (with caution)

Delay rates vary by operator — but this is not a “scoreboard”

This is helpful for monitoring. But operators run different routes and areas, so I must compare carefully.

What this chart answers

  • Are some operators above or below the overall baseline?
  • Which operators should I look at first for deeper investigation?

How to interpret it safely

  • Use it as a monitoring signal, not a final judgement.
  • Differences may happen because an operator serves harder routes or busier areas.
  • Best follow-up: compare operators on similar routes and time bands.
  • “Unknown” usually means the operator value was missing in the data.
Metric: delayed share = mean(is_delayed) (Delayed ≥ 1 min). Dashed line = overall baseline.
Bars show delayed share; hover shows stop-event volume (n).
EDA • Severity • How big delays get

When delays happen, some time bands have bigger delays

Earlier I looked at how often delays happen. Here I look at how many minutes the delay is (only for delayed events).

How to read this chart

  • Each box shows the “typical” delay minutes for that time band.
  • The line inside the box is the middle delay value.
  • Longer whiskers and more dots mean more extreme delays can happen.
  • I only include delay ≥ 1 minute (so I focus on real delays).

Why it matters

  • Operations care about both: how often delays happen and how big they get.
  • If a time band has bigger delays, it may need more attention (buffer time, faster response).
  • Rare big delays can come from disruptions (traffic, incidents, road works).
Metric: delay_minutes for delayed events only (≥ 1). Y-axis is capped for readability.
Boxplots help us see typical delays and rare extreme delays.
EDA • Correlation • Quick overlap check

Correlation heatmap: what moves together (and a quick leakage reminder)

Correlation is a simple way to see which variables “travel together”. It helps spot overlap and obvious mistakes.

How to read this

  • Each square compares two variables.
  • Values go from -1 to +1.
  • +1 = they rise together, -1 = one goes up while the other goes down.
  • Near 0 = no strong linear relationship.
  • Blue = positive, red = negative, gray ≈ near zero.

Key takeaways

  • Some features overlap a lot (example: the rain variables).
  • Calendar features also overlap (weekday vs weekend).
  • The target (is_delayed) has very small correlations here, so delays are not explained by one single numeric feature.
  • Leakage reminder: I only use information known before the bus arrives (so not actual arrival timestamps like real_time).
Note: low correlation does not mean “not useful”. It can miss non-linear patterns.
Blue = positive correlation, red = negative, gray ≈ near zero.
Modeling • What the model sees • Fair evaluation

Modeling setup: what I use as inputs (and how I test fairly)

I predict Delayed vs On-time (Delayed ≥ 1 min) using only information known before the bus reaches the stop.

Inputs to the model (13 features)

Group Columns How I handle it
Numeric planned_hour, planned_weekday, rain, route_volume, stop_volume LR: scale numbers (StandardScaler)
Trees (RF/GB): use as-is
Binary is_weekend, is_rainy, is_heavy_rain use as-is
Small categories operator (≈5), time_band (≈5) One-hot (safe with new values)
Large IDs stop_id (≈2494), route_id (≈589), route_short_name (≈568) LR baseline: not used
RF: OrdinalEncoder (unknown = -1)
Target: is_delayed (Delayed ≥ 1 minute). Dataset size ≈ 7.36M rows.
Why this setup: I keep a simple baseline (Logistic Regression) and also test a stronger model that can use route/stop IDs carefully.

Train/test split (fair check)

  • I split data into train (learn) and test (final check).
  • I keep the same delayed rate in both sets (so the test is realistic).
  • Train: 5,885,360 rows • Test: 1,471,340 rows.

Because delayed is ~36%

  • Accuracy alone can be misleading, so our main score is F1.
  • I also report Precision, Recall, and ROC AUC.
  • I use class_weight so the model pays enough attention to delayed cases.

Leakage rules (must-follow)

  • Leakage = cheating by accident. It happens when the model uses information that is only known after the stop happens.
  • So I never use “answer” columns like delay_minutes or real_time.
  • I only use schedule + context that I know in advance (time, operator, weather, demand proxies, IDs).
Modeling • Compare models • Main score: F1

Which model performs best for delay alerts?

I compare a simple baseline to stronger tree models, and I choose the one with the best balance between missed delays and false alarms.

How I compare (same test set)

  • Target: Delayed vs On-time (Delayed ≥ 1 minute).
  • Split: 80/20 stratified (same real delay rate in train and test).
  • Main score: F1 (balance of Precision and Recall).
  • Also shown: Precision, Recall, and ROC AUC.

What the scores mean (in plain words)

  • Recall: out of all real delays, how many I catch.
  • Precision: out of all alerts I raise, how many are real delays.
  • F1: one number that balances Precision + Recall.
  • ROC AUC: how well the model ranks “more risky” vs “less risky”.

Main takeaway

  • Random Forest is the top performer (best overall F1 and ROC AUC).
  • Undersampled train vs class_weight: results are close. Undersampling can catch slightly more delays, but may add more false alarms.
  • The Logistic Regression baseline is much lower, so the pattern is not just a simple linear rule.
Highlighted row = best F1 on the test set.
Modeling • Threshold choice • Confusion matrix

Turning model probability into a clear “Delay alert”

The model gives a probability. I choose a threshold to decide when to flag Delayed vs On-time (Delayed ≥ 1 min).

What the top chart means

  • Lower threshold → I catch more delays (higher Recall), but I also create more false alarms.
  • Higher threshold → fewer false alarms (higher Precision), but I miss more real delays.
  • I choose the threshold that gives the best balance using F1.

Chosen point (best F1)

  • Best F1 threshold ≈ 0.46 (from the chart).
  • Recall ≈ 0.83 → I catch about 83% of real delays.
  • Precision ≈ 0.51 → about half of our alerts are true delays.
  • This is adjustable later, depending on how costly false alarms vs missed delays are.
Simple idea: choose a threshold that matches operational reality (how much I hate false alarms vs missed delays).
Practical note: I can tune the threshold later based on the cost of false alarms vs missed delays.
Modeling • Validation • ROC & PR curves

Do the predictions separate delayed vs on-time well?

These curves check if the model is truly useful, not just at one threshold. (Delayed = ≥ 1 minute.)

ROC curve (top) — “ranking test”

  • ROC asks: can the model rank delayed trips above on-time trips?
  • The dashed diagonal is random guessing.
  • Curves closer to the top-left are better.
  • AUC is a summary score: 0.5 = random, 1.0 = perfect.

Precision–Recall (bottom) — better for imbalance

  • Because delays are ~36% of events, PR is often more realistic than ROC.
  • The dashed line is the baseline (what you get by guessing “delay” at the same rate as the data).
  • PR shows the trade-off: if I try to catch more delays (higher recall), precision usually drops (more false alarms).
Simple takeaway: both curves sit clearly above the baseline → the model can prioritize “riskier” stop events.
Modeling • Interpretability • What the model uses most

What information does the model rely on most?

This chart ranks features by importance inside the Random Forest. Bigger bars mean the model used that feature more.

How to read this

  • Importance is a relative score: features add up to 100%.
  • Higher importance means the feature helped the model make better splits/predictions.
  • This is not proof of cause. It only tells us what is useful for prediction.
  • If two features are similar, they can share importance.

Simple takeaway from our results

  • Demand/location signals (like stop volume and stop/route identifiers) matter the most.
  • Time features (hour/weekday/time band) matter, but less than the location signals.
  • Weather features are present, but they contribute a smaller share here.
  • Operational meaning: focus first on busy stops and specific routes when looking for improvements.
Tip: ID features help prediction, but they are “where” signals. For action, I still need to inspect the real causes on those routes/stops.
Recommendation • How to use the model

Recommendation: use the model as a “risk list” for daily operations

The model does not “fix” delays by itself. It helps teams focus on the trips that are more likely to run late.

What the model gives you

  • For each stop event, the model outputs a risk score (0 to 1).
  • Higher score = more likely to be delayed.
  • I use it to create a short priority list for dispatch and planning.

Three simple ways to use it

  • Monitor: keep an eye on high-risk trips (especially during peak hours).
  • Prepare: add small buffer or extra attention on risky routes/stops.
  • Communicate: if risk is high, send earlier passenger updates (when possible).
Simple idea: use the model to decide “where to look first,” not “who to blame.”

Traffic-light alerts (easy version)

Green
Low risk → do nothing.
Avoid alert fatigue.
Yellow
Medium risk → watchlist.
Good for awareness.
Red
High risk → act early.
Dispatch attention + optional message.
I can tune the cutoffs later (fewer false alarms vs fewer missed delays).

Rollout plan (3 steps)

  1. Shadow mode: show alerts, but don’t act yet. Check daily if they make sense.
  2. Small pilot: enable on a few routes/time bands first.
  3. Scale: expand after the team trusts the alerts.
Track: number of alerts, how many were correct, and whether performance changes over time.
Limitations • Risks • Next steps

Limitations and next steps

The model is useful, but some delays are hard to predict without real-time information.

Main limitations (today)

  • No real-time signals: I don’t have live traffic, incidents, or bus telemetry (GPS speed, headway).
  • Short time window: July 25 → Sept 16, 2025 (about 7–8 weeks), so I don’t capture full seasons.
  • Unexpected events: accidents, road works, and vehicle issues are hard to predict from static data.
  • Stops/routes change: ID patterns may not transfer well to new or changed routes.
  • Not causality: features help prediction, but they don’t prove the real cause.

What I do about it (practical)

  • Time-based testing: train on earlier weeks, test on later weeks.
  • Monitoring: track precision/recall and drift by route/time band.
  • Retrain: update the model regularly when patterns change.
Takeaway: this is a decision helper that needs monitoring, not a one-time model.

Best data to add next

  • Traffic + incidents: congestion, road works, accidents, major events.
  • Bus telemetry (AVL/GPS): speed, dwell time, headway, bunching signals.
  • Network history: what happened at previous stops on the same trip/line.

Simple go-live checklist

  • Alert volume: keep alerts manageable (avoid alert fatigue).
  • Quality: weekly precision/recall at the chosen threshold.
  • Stability: watch for sudden changes by route/time band.
End

Thank you

This concludes my presentation.