Unsupervised learning · clickstream · telecom self-care app

Turning 250 million unlabeled clicks into journeys people can name

A production pipeline that pulls app events from Kafka every day, cuts each session into goal-level journeys, clusters them with no labels, and tells the product team what users were trying to do and where they struggled.

One iOS session, segmented into three journeys

Illustrative session with placeholder screen names. Times are seconds from session start.

250 M+raw events in one quarter

~17 Mjourneys scored

~2,800journey archetypes across Android and iOS

~500business intents in the 3-level taxonomy

DailyAirflow run, Kafka to named journeys

The problem

Click logs say what happened. They don't say why.

The product team wanted to know which tasks customers come to the app for, which flows cause friction, and how iOS and Android differ. Nobody had labels, and the raw logs fight back in five ways.

No ground truth

Hand-labeling millions of sessions is impossible. Clusters have to emerge from the data; people only name them afterwards.

A session is not a goal

One session_id can last over 30 hours and mix a bill payment, a Wi-Fi reset and a support chat.

Two schemas in one table

View and Action events store the screen name in different columns, so a naive join puts them in separate vocabularies.

Two platforms, two dialects

iOS DeviceListVC and Android network_protect/device_management can be the same feature.

An exploding vocabulary

Contract codes, UUIDs and URL parameters turn every event into a near-unique token unless they are masked.

Scale

A quarter of data is 250 M+ rows. Every stage has to stream with bounded memory and rerun safely.

Part 1 · Data pipeline

MongoDB → Kafka → SQL, with a daily Airflow job on top

The app writes events to MongoDB, and they are published to a Kafka cluster. I built the sink connector that pulls those topics into SQL, and an Airflow DAG that runs every day to summarise and cluster the previous day's behaviour.

Mobile appAndroid · iOS click events

MongoDBoperational event store

Kafka clusterclickstream topics

Sink connectorbuilt for this project · pulls topics

SQLevent tables by day

Airflow DAG · daily cron

Each task runs one CLI in the same Docker image, so it can retry on its own.

ExtractSQL export
Yesterday's events, after the sink has committed the full day.
Partitionjourney-partition
Hashes (platform, session_id) so a session never splits across files.
Cluster & scorejourney-infer
Android and iOS in parallel on the frozen champion model.
Namejourney-name
Joins scores with the reviewed taxonomy mapping.
Summariseprepare_dashboard_data
Daily KPIs, cluster mix, friction and next-action tables.

Out-of-core

PyArrow streaming and DuckDB. Training on hundreds of thousands of journeys per platform stays within a few GB of memory.

Idempotent reruns

Data partitions and model runs carry separate IDs, so rerunning a day reuses what already exists.

Traceable rows

Every scored journey stores model_version and source_partition.

Part 2 · ML pipeline

Data processing → model development → monitoring

Data processing

From raw rows to clean journeys

Canonicalise. Fix the View/Action column swap, namespace screens by OS, mask {id}, {uuid}, {code}.
Semantic ladder. Map each event to family / module / object / operation so both platforms meet.
Segment. Cut on idle gap > 90 s (p97.5), return to home after ≥ 6 events, login or logout, or 80 events.
Clean. Collapse repeats and A→B→A→B loops, but keep the counts as friction signals.

Model development

Embed, cluster, explain

Embed. N-gram TF-IDF → SVD, three semantic channels and 11 behaviour metrics, each L2-normalised, then PCA to 48 dims.
Cluster. HDBSCAN finds archetypes of any shape and leaves irregular journeys as noise. K-means is the baseline.
Markov bank. One transition chain per cluster for likelihood and next-action prediction.
Name. Cluster n-grams are matched to a ~500-intent taxonomy with evidence thresholds and a human review queue.

Monitoring

Know when the model goes stale

Data drift. Schema changes, row volume, new-token rate, boundary-reason mix, journey length and timing distributions.
Model drift. Unassigned rate, centroid distance, Markov log-prob, cluster population PSI, anomaly and friction rates.
Response. Drift that persists triggers a challenger. It replaces the champion only after passing holdout and stability gates, and the old model stays for rollback.

The semantic ladder

One tap on the modem toggle, seen at five resolutions. Coarser levels let iOS and Android meet in the same space; tokens seen in fewer than 3 journeys back off one level.

Level	Token	Granularity
Exact	`action@ModemSettingsVC#toggle_power`	Verbatim event
L3 intent	`internet/modem/modem/toggle`	family / module / object / operation
L2 coarse	`internet/modem`	family / module
L1 family	`internet`	business domain
Operation	`configure:toggle`	stage : operation

Two ways a journey can look wrong

Every scored journey gets a geometric check (distance to its cluster centre) and a generative check (how likely its screen transitions are under that cluster's Markov chain).

Transitions likely

Transitions unlikely (< p05)

Close to centre

Typical journeyThe expected flow for this task.

Odd path, familiar screensRight screens, strange order. Often a confusing navigation step.

Far from centre (> p95)

Odd mix, normal stepsPlausible moves but an unusual overall shape, such as very long or very loopy.

Anomalous on bothHighest priority for UX review.

Results · one quarter of data

What ~17 million journeys say about the app

Share of journeys by business family

Rounded to whole percent. Unclassified includes noise (about 5 %) and clusters without enough naming evidence.

AndroidiOS

Show as table

Family	Android	iOS

Friction map: every discovered cluster

Each dot is one cluster. Longer journeys with more backtracking sit up and to the right. Dot area shows journey count. Values are rounded.

Highlight

HDBSCAN, per platform	Clusters	Silhouette	Davies–Bouldin	Calinski–Harabasz	Fit-time noise
Android	~1,400	0.36	1.3	~8,700	~43 %
iOS	~1,400	0.36	1.2	~7,600	~45 %

HDBSCAN leaves irregular journeys out at fit time on purpose. At inference, nearest-centroid assignment and noise post-processing bring the unassigned share down to about 5 %.

Beyond the warehouse

The same model runs on the phone

Android · Kotlin + ONNX Runtime

Replays real clickstreams at 0.25× to 10× and classifies each journey on-device, with friction flags and next-action prediction.

iOS · Swift package

A port with no external dependencies: float16 projection weights, centroid matrix and Markov graph shipped as compact binaries.

One contract

Python, Kotlin and Swift share tokenisation, masking and model files, so phone and warehouse agree on every journey.

What's next

Stable cluster identities

Match clusters across retrains by medoid and n-gram overlap, so "cluster 7" keeps its meaning.

Entropy-based segmentation

Promote branching-entropy cuts from opt-in to default once validated against the rules.

Neural sequence encoders

Add a learned sequence embedding as one more channel next to TF-IDF.