generate buyer journeys that seem easy and interesting, however evaluating whether or not these journeys are structurally sound stays difficult for present strategies.
This text introduces Continuity, Deepening, and Development (CDP) — three deterministic, content-structure-based metrics for evaluating multi-step journeys utilizing a predefined taxonomy moderately than stylistic judgment.
Historically, optimizing customer-engagement methods has concerned fine-tuning supply mechanics equivalent to timing, channel, and frequency to attain engagement and enterprise outcomes.
In apply, this meant you educated the mannequin to know guidelines and preferences, equivalent to “Don’t contact clients too usually”, “Consumer Alfa responds higher to telephone calls”, and “Consumer Beta opens emails principally within the night.”
To handle this, you constructed a cool-off matrix to stability timing, channel constraints, and enterprise guidelines to control buyer communication.
To this point, so good. The mechanics of supply are optimized.
At this level, the core problem arises when the LLM generates the journey itself. The problem is not only about channel or timing, however whether or not the sequence of messages kinds a coherent, efficient narrative that meets enterprise aims.
And all of a sudden you understand:
There isn’t a normal metric to find out if an AI-generated journey is coherent, significant, or advances enterprise targets.
What We Count on From a Profitable Buyer Journey
From a enterprise perspective, the sequence of contents per journey step can’t be random: it have to be a guided expertise that feels coherent, strikes the client ahead by significant levels, and deepens the connection over time.
Whereas this instinct is frequent, it is usually supported by customer-engagement analysis. Brodie et al. (2011) describe engagement as “a dynamic, iterative course of” that varies in depth and complexity as worth is co-created over time.
In apply, we consider journey high quality alongside three complementary dimensions:
Continuity — whether or not every message suits the context established by prior interactions.
Deepening — whether or not content material turns into extra particular, related, or personalised moderately than remaining generic.
Development — whether or not the journey advances by levels (e.g., from exploration to motion) with out pointless backtracking.
Why Current LLM Analysis Metrics Fall Quick
If we take a look at normal analysis strategies for LLMs, equivalent to accuracy metrics, similarity metrics, human-evaluation standards, and even LLM-as-a-judge, it turns into clear that none present a dependable, unambiguous strategy to consider buyer journeys generated as multi-step sequences.
Let’s look at what normal buyer journey metrics can and may’t present.
Accuracy Metrics (Perplexity, Cross-Entropy Loss)
These metrics measure confidence degree in predicting the subsequent token given the coaching knowledge. They don’t seize whether or not a generated sequence kinds a coherent or significant journey.
Similarity Metrics (BLEU, ROUGE, METEOR, BERTScore, MoveScore)
These metrics evaluate the generated end result to a reference textual content. Nevertheless, buyer journeys hardly ever have a single appropriate reference, as they adapt to context, personalization, and prior interactions. Structurally legitimate journeys could differ considerably whereas remaining efficient.
Undoubtedly, semantic similarity has its benefits, and we’ll use cosine similarity, however extra on that later.
Human Analysis (Fluency, Relevance, Coherence)
Human judgment usually outperforms automated metrics in assessing language high quality, however it’s poorly suited to steady journey analysis. It’s costly, suffers from cultural bias and ambiguity, and doesn’t perform as a everlasting a part of the workflow however moderately as a one-time effort to bootstrap a fine-tuning stage.
LLM-as-a-Decide (AI suggestions scoring)
Utilizing LLMs to guage outputs from different LLM methods is a formidable course of.
This strategy tends to focus extra on fashion, readability, and tone moderately than structural analysis.
LLM-as-a-Decide might be utilized in multi-stage use instances, however outcomes are sometimes much less exact as a result of elevated danger of context overload. Moreover, fine-grained analysis scores from this technique are sometimes unreliable. Like human evaluators, LAAJ additionally carries biases and ambiguities.
A Structural Method to Evaluating Buyer Journeys
Finally, the first lacking ingredient in evaluating really useful content material sequences throughout the buyer journey is construction.
Probably the most pure strategy to symbolize content material construction is as a taxonomic tree, a hierarchical mannequin consisting of levels, content material themes, and ranges of element.
As soon as buyer journeys are mapped onto this tree, CDP metrics might be outlined as structural variations:
- Continuity: easy motion throughout branches
- Deepening: transferring into extra particular nodes
- Development: transferring ahead by buyer journey levels
The answer is to symbolize a journey as a path by a hierarchical taxonomy derived from the content material house. As soon as this illustration is established, CDP metrics might be computed deterministically from the trail. The diagram under summarizes your entire pipeline.
Developing the Taxonomy Tree
To judge buyer journeys structurally, we first require a structured illustration of content material. We assemble this illustration as a multi-level taxonomy derived straight from customer-journey textual content utilizing semantic embeddings.
The taxonomy is anchored by a small set of high-level levels (e.g., motivation, buy, supply, possession, and loyalty). Each anchors and journey messages are embedded into the identical semantic vector house, permitting content material to be organized by semantic proximity.
Inside every anchor, messages are grouped into progressively extra particular themes, forming deeper ranges of the taxonomy. Every degree refines the earlier one, capturing growing topical specificity with out counting on handbook labeling.
The result’s a hierarchical construction that teams semantically associated journey messages and gives a steady basis for evaluating how journeys move, deepen, and progress over time.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is established, particular person buyer journeys are mapped onto it as ordered sequences of messages. Every step is embedded in the identical semantic house and matched to the closest taxonomy node utilizing cosine similarity.
This mapping converts a temporal sequence of messages right into a path by the taxonomy, enabling the structural evaluation of journey evolution moderately than treating the journey as a flat record of texts.
Defining the CDP Metrics
The CDP framework consists of three complementary metrics: Continuity, Deepening, and Development. Every captures a definite side of journey high quality. We describe these metrics conceptually earlier than defining them formally primarily based on the taxonomy-mapped journey.
Setup and Computation
Earlier than analyzing actual journeys, we make clear two elements of the setup.
(1) how journey content material is structurally represented, and
(2) how CDP metrics are derived from that construction.
Buyer-journey content material is organized right into a hierarchical taxonomy consisting of anchors (L1 journey levels), thematic heads (L2 subjects), and deeper nodes that symbolize growing specificity:
Anchor (L1)
└── Head (L2)
└── Baby (L3)
└── Grandchild (L4+)As soon as a journey is mapped onto this hierarchy, Continuity, Deepening, and Development are computed deterministically from the journey’s path by the tree.
Let a buyer journey be an ordered sequence of steps:
J = (x₁, x₂, …, xₙ)
Every step xᵢ is assigned:
- aᵢ — anchor (L1 journey stage)
- tᵢ — thematic head (L2 subject), the place tᵢ = 0 means “unknown”
- ℓᵢ — taxonomy depth degree (L1 = 0, L2 = 1, L3 = 2, …)
Continuity (C)
Continuity evaluates whether or not consecutive messages stay contextually and thematically coherent.
For every transition (xᵢ →xᵢ₊₁), a step-level continuity rating cᵢ ∈ [0, 1] is assigned primarily based on taxonomy alignment, with greater weights given to transitions that keep throughout the similar subject or intently associated branches.
Transitions are ranked from strongest to weakest (e.g., similar subject, associated subject, ahead stage transfer, backward transfer), and
assigned lowering weights:
1 ≥ α₁ > α₂ > α₃ > α₄ > α₅ > α₆ ≥ 0
The general continuity rating is computed as:
C(J) = (1 / (n − 1)) · Σ cᵢ for i = 1 … n−1
Deepening (D)
Deepening measures whether or not a journey accumulates worth by transferring from common content material towards extra particular or detailed
interactions. It’s computed utilizing two complementary parts.
Journey-based deepening captures how depth adjustments alongside the noticed path:
Δᵢᵈᵉᵖᵗʰ = ℓᵢ₊₁ − ℓᵢ, dᵢ = max(Δᵢᵈᵉᵖᵗʰ, 0)
D_journey(J) = (1 / (n − 1)) · Σ dᵢ
Taxonomy-aware deepening measures how deeply a journey explores the precise taxonomy tree, primarily based on the heads it visits.
It evaluates how most of the attainable deeper content material objects (kids, sub-children, and so on.) underneath every visited head are later seen
throughout the journey.
D_taxonomy(J) = |D_seen(J)| / |D_exist(J)|
The ultimate deepening rating is a weighted mixture:
D(J) = λ₁ · D_taxonomy(J) + λ₂ · D_journey(J), λ₁ + λ₂ = 1.
Deepening lies in [0, 1].
Development (P)
Development measures directional motion by journey levels. For every transition, we compute:
Δᵢ = aᵢ₊₁ − aᵢ.
Solely transferring steps (Δᵢ ≠ 0) are thought of. Let wᵢ denote the relative significance of the present stage.
If Δᵢ > 0 (ahead motion):
cᵢ = wᵢ / Δᵢ
If Δᵢ < 0 (backward motion):
cᵢ = Δᵢ · wᵢ
The uncooked development rating is:
P_raw(J) = Σ cᵢ for all i the place Δᵢ ≠ 0
To certain the rating to[−1, +1], we apply a tanh normalization:
P(J) = (e^(P_raw) − e^(−P_raw)) / (e^(P_raw) + e^(−P_raw))
Making use of CDP Metrics to an Automotive Buyer Journey
To show how structured analysis works on lifelike journeys, we generated an artificial automotive customer-journey dataset masking the primary levels of the client lifecycle.
Enter Information: Anchors and Journey Content material
The CDP framework makes use of two important inputs: anchors, which outline journey levels, and customer-journey content material, which gives the messages to guage.
Anchors symbolize significant phases within the lifecycle, equivalent to motivation, buy, supply, possession, and loyalty. Every anchor is augmented with a small set of consultant key phrases to floor it semantically. Anchors serve each as reference factors for taxonomy development and because the anticipated directional move used later within the Development metric.
anchor Phrases:
motivation exploration analysis discovery curiosity check drive wants evaluation expertise
buy financing comparability quotes mortgage negotiation credit score pre-approval deposit
supply paperwork signing deposit logistics handover activation
possession upkeep guarantee restore vendor assist service inspections
loyalty suggestions satisfaction survey referral improve retention advocacyBuyer-journey content material consists of quick, action-oriented CRM-style messages (emails, calls, chats, in-person interactions) with various ranges of specificity and spanning a number of levels. Though this dataset is synthetically generated, anchor data will not be used throughout taxonomy development or CDP scoring.
CJ messages:
Discover fashions that match your life-style and private targets.
Take a digital tour to find key options and trims.
Evaluate physique types to evaluate house, consolation, and utility.
E book a check drive to expertise dealing with and visibility.
Use the wants evaluation to rank must-have options.
Filter fashions by vary, mpg, or towing to slender decisions.Taxonomy Development Outcomes
Right here, we utilized the taxonomy development course of to the automotive customer-engagement dataset. The determine under reveals the ensuing customer-journey taxonomy, constructed from message content material and anchor semantics.
Every top-level department corresponds to a journey anchor (L1), which represents main journey levels equivalent to Motivation, Buy, Supply, Possession, and Loyalty.
Deeper ranges (L2, L3+) group messages by thematic similarity and growing specificity.
What the Taxonomy Reveals
Even on this compact dataset, the taxonomy highlights a number of purposeful patterns:
- Early-stage messages cluster round exploration and comparability, progressively narrowing towards concrete actions equivalent to reserving a check drive.
- Buy-related content material separates naturally into monetary planning, doc dealing with, and finalization.
- Possession content material reveals a transparent development from upkeep scheduling to diagnostics, price estimation, and guarantee analysis.
- Loyalty content material shifts from transactional actions towards suggestions, upgrades, and advocacy.
Whereas these patterns align with how practitioners usually motive about journeys, they come up straight from the information moderately than from predefined guidelines.
Why This Issues for Analysis
This taxonomy now gives a shared structural reference:
- Any buyer journey might be mapped as a path by the tree.
- Motion throughout branches, depth ranges, and anchors turns into measurable.
- Continuity, Deepening, and Development are not summary ideas; they now correspond to concrete structural adjustments.
Within the subsequent part, we use this taxonomy to map actual journey examples and compute CDP scores in steps.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is constructed, evaluating a buyer journey turns into a structural drawback.
Every journey is represented as an ordered sequence of customer-facing messages.
As a substitute of judging these messages in isolation, we mission them onto the taxonomy and analyze the ensuing path.
Formally, a journey J = (x₁, x₂, …, xₙ) is mapped to a sequence of taxonomy nodes: (x₁→v₁),(x₂→v₂),…,(xₙ→vₙ) the place every vᵢ is the closest taxonomy node primarily based on embedding similarity.
A Step-by-Step Walkthrough: From Journey Textual content to CDP Scores
To make the CDP framework concrete, let’s stroll by a single buyer journey instance and present how it’s evaluated step-by-step.
Step 1 — The Buyer Journey Enter
We start with an ordered sequence of customer-facing messages generated by an LLM.
Every message represents a touchpoint in a practical automotive buyer journey:
journey = ['Take a virtual tour to discover key features and trims.';
'We found a time slot for a test drive that fits your schedule.';
'Upload your income verification and ID to finalize the pre-approval decision.';
'Estimate costs for upcoming maintenance items.';
'Track retention offers as your lease end nears.';
'Add plates and registration info before handover.']Step 2 — Mapping the Journey into the Taxonomy
For structural analysis, every journey step is mapped into the customer-journey taxonomy. Utilizing textual content embeddings, every message is matched to its closest taxonomy node. This produces a journey map (jmap), a structured illustration of how the journey traverses the taxonomy.
Desk 2: Every message is assigned to an anchor (stage), a thematic head, and a depth degree within the taxonomy primarily based on semantic similarity within the shared embedding house. This desk acts as the muse for all future evaluations.
Step 3 — Making use of CDP Metrics to This Journey
As soon as the journey is mapped, we compute Continuity, Deepening, and Development deterministically from step-to-step transitions.
Desk 3: Every row represents a transition between consecutive journey steps, annotated with indicators for continuity, deepening, and development.
Ultimate CDP scores (this journey):
Taken collectively, the CDP indicators point out a journey that’s largely coherent and forward-moving, with one clear second of
deepening and one seen structural regression. Importantly, these insights are derived solely from construction, not from
stylistic judgments in regards to the textual content.
Conclusion: From Scores to Profitable Journeys
Continuity, Deepening, and Development are decided by construction and might be utilized wherever LLMs generate multi-step
content material:
- to check different journeys generated by completely different prompts or fashions,
- to offer automated suggestions for bettering journey era over time.
On this means, CDP scores provide structural suggestions for LLMs. They complement, moderately than change, stylistic or fluency-based analysis by offering indicators that mirror enterprise logic and buyer expertise.
Though this text focuses on automotive commerce, the idea is broadly relevant. Any system that generates ordered, goal-oriented content material requires sturdy structural foundations.
Massive language fashions are already able to producing fluent, persuasive textual content.
The higher problem is making certain that textual content sequences type coherent narratives that align with enterprise logic and consumer expertise.
CDP gives a strategy to make construction specific, measurable, and actionable.
Thanks for staying with me by this journey. Hopefully, this idea helps you assume in another way about evaluating AI-generated sequences and evokes you to deal with construction as a major sign in your personal methods. All logic introduced on this article is applied within the accompanying Python code on GitHub. You probably have any questions or feedback, please depart them within the feedback part or attain out through LinkedIn.
References
Brodie, R. J., et al. (2011). Buyer engagement: Conceptual area, elementary propositions, and implications for analysis.
