generate buyer journeys that seem clean and fascinating, however evaluating whether or not these journeys are structurally sound stays difficult for present strategies.
This text introduces Continuity, Deepening, and Development (CDP) — three deterministic, content-structure-based metrics for evaluating multi-step journeys utilizing a predefined taxonomy reasonably than stylistic judgment.
Historically, optimizing customer-engagement methods has concerned fine-tuning supply mechanics reminiscent of timing, channel, and frequency to attain engagement and enterprise outcomes.
In observe, this meant you educated the mannequin to know guidelines and preferences, reminiscent of “Don’t contact clients too usually”, “Shopper Alfa responds higher to cellphone calls”, and “Shopper Beta opens emails largely within the night.”
To handle this, you constructed a cool-off matrix to stability timing, channel constraints, and enterprise guidelines to control buyer communication.
Thus far, so good. The mechanics of supply are optimized.
At this level, the core problem arises when the LLM generates the journey itself. The difficulty is not only about channel or timing, however whether or not the sequence of messages varieties a coherent, efficient narrative that meets enterprise aims.
And all of the sudden you understand:
There isn’t a normal metric to find out if an AI-generated journey is coherent, significant, or advances enterprise objectives.
What We Anticipate From a Profitable Buyer Journey
From a enterprise perspective, the sequence of contents per journey step can’t be random: it should be a guided expertise that feels coherent, strikes the shopper ahead by way of significant phases, and deepens the connection over time.
Whereas this instinct is frequent, it is usually supported by customer-engagement analysis. Brodie et al. (2011) describe engagement as “a dynamic, iterative course of” that varies in depth and complexity as worth is co-created over time.
In observe, we consider journey high quality alongside three complementary dimensions:
Continuity — whether or not every message suits the context established by prior interactions.
Deepening — whether or not content material turns into extra particular, related, or personalised reasonably than remaining generic.
Development — whether or not the journey advances by way of phases (e.g., from exploration to motion) with out pointless backtracking.
Why Current LLM Analysis Metrics Fall Quick
If we take a look at normal analysis strategies for LLMs, reminiscent of accuracy metrics, similarity metrics, human-evaluation standards, and even LLM-as-a-judge, it turns into clear that none present a dependable, unambiguous option to consider buyer journeys generated as multi-step sequences.
Let’s look at what normal buyer journey metrics can and may’t present.
Accuracy Metrics (Perplexity, Cross-Entropy Loss)
These metrics measure confidence stage in predicting the subsequent token given the coaching knowledge. They don’t seize whether or not a generated sequence varieties a coherent or significant journey.
Similarity Metrics (BLEU, ROUGE, METEOR, BERTScore, MoveScore)
These metrics examine the generated outcome to a reference textual content. Nonetheless, buyer journeys hardly ever have a single appropriate reference, as they adapt to context, personalization, and prior interactions. Structurally legitimate journeys could differ considerably whereas remaining efficient.
Undoubtedly, semantic similarity has its benefits, and we’ll use cosine similarity, however extra on that later.
Human Analysis (Fluency, Relevance, Coherence)
Human judgment usually outperforms automated metrics in assessing language high quality, however it’s poorly suited to steady journey analysis. It’s costly, suffers from cultural bias and ambiguity, and doesn’t operate as a everlasting a part of the workflow however reasonably as a one-time effort to bootstrap a fine-tuning stage.
LLM-as-a-Choose (AI suggestions scoring)
Utilizing LLMs to judge outputs from different LLM methods is a powerful course of.
This strategy tends to focus extra on fashion, readability, and tone reasonably than structural analysis.
LLM-as-a-Choose will be utilized in multi-stage use instances, however outcomes are sometimes much less exact because of the elevated danger of context overload. Moreover, fine-grained analysis scores from this technique are sometimes unreliable. Like human evaluators, LAAJ additionally carries biases and ambiguities.
A Structural Method to Evaluating Buyer Journeys
Finally, the first lacking component in evaluating really useful content material sequences throughout the buyer journey is construction.
Probably the most pure option to signify content material construction is as a taxonomic tree, a hierarchical mannequin consisting of phases, content material themes, and ranges of element.
As soon as buyer journeys are mapped onto this tree, CDP metrics will be outlined as structural variations:
- Continuity: clean motion throughout branches
- Deepening: shifting into extra particular nodes
- Development: shifting ahead by way of buyer journey phases
The answer is to signify a journey as a path by way of a hierarchical taxonomy derived from the content material house. As soon as this illustration is established, CDP metrics will be computed deterministically from the trail. The diagram beneath summarizes your entire pipeline.
Establishing the Taxonomy Tree
To guage buyer journeys structurally, we first require a structured illustration of content material. We assemble this illustration as a multi-level taxonomy derived instantly from customer-journey textual content utilizing semantic embeddings.
The taxonomy is anchored by a small set of high-level phases (e.g., motivation, buy, supply, possession, and loyalty). Each anchors and journey messages are embedded into the identical semantic vector house, permitting content material to be organized by semantic proximity.
Inside every anchor, messages are grouped into progressively extra particular themes, forming deeper ranges of the taxonomy. Every stage refines the earlier one, capturing rising topical specificity with out counting on handbook labeling.
The result’s a hierarchical construction that teams semantically associated journey messages and offers a secure basis for evaluating how journeys move, deepen, and progress over time.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is established, particular person buyer journeys are mapped onto it as ordered sequences of messages. Every step is embedded in the identical semantic house and matched to the closest taxonomy node utilizing cosine similarity.
This mapping converts a temporal sequence of messages right into a path by way of the taxonomy, enabling the structural evaluation of journey evolution reasonably than treating the journey as a flat listing of texts.
Defining the CDP Metrics
The CDP framework consists of three complementary metrics: Continuity, Deepening, and Development. Every captures a definite side of journey high quality. We describe these metrics conceptually earlier than defining them formally primarily based on the taxonomy-mapped journey.
Setup and Computation
Earlier than analyzing actual journeys, we make clear two facets of the setup.
(1) how journey content material is structurally represented, and
(2) how CDP metrics are derived from that construction.
Buyer-journey content material is organized right into a hierarchical taxonomy consisting of anchors (L1 journey phases), thematic heads (L2 subjects), and deeper nodes that signify rising specificity:
Anchor (L1)
└── Head (L2)
└── Baby (L3)
└── Grandchild (L4+)As soon as a journey is mapped onto this hierarchy, Continuity, Deepening, and Development are computed deterministically from the journey’s path by way of the tree.
Let a buyer journey be an ordered sequence of steps:
J = (x₁, x₂, …, xₙ)
Every step xᵢ is assigned:
- aᵢ — anchor (L1 journey stage)
- tᵢ — thematic head (L2 matter), the place tᵢ = 0 means “unknown”
- ℓᵢ — taxonomy depth stage (L1 = 0, L2 = 1, L3 = 2, …)
Continuity (C)
Continuity evaluates whether or not consecutive messages stay contextually and thematically coherent.
For every transition (xᵢ →xᵢ₊₁), a step-level continuity rating cᵢ ∈ [0, 1] is assigned primarily based on taxonomy alignment, with greater weights given to transitions that keep throughout the similar matter or intently associated branches.
Transitions are ranked from strongest to weakest (e.g., similar matter, associated matter, ahead stage transfer, backward transfer), and
assigned lowering weights:
1 ≥ α₁ > α₂ > α₃ > α₄ > α₅ > α₆ ≥ 0
The general continuity rating is computed as:
C(J) = (1 / (n − 1)) · Σ cᵢ for i = 1 … n−1
Deepening (D)
Deepening measures whether or not a journey accumulates worth by shifting from basic content material towards extra particular or detailed
interactions. It’s computed utilizing two complementary elements.
Journey-based deepening captures how depth adjustments alongside the noticed path:
Δᵢᵈᵉᵖᵗʰ = ℓᵢ₊₁ − ℓᵢ, dᵢ = max(Δᵢᵈᵉᵖᵗʰ, 0)
D_journey(J) = (1 / (n − 1)) · Σ dᵢ
Taxonomy-aware deepening measures how deeply a journey explores the precise taxonomy tree, primarily based on the heads it visits.
It evaluates how most of the attainable deeper content material gadgets (youngsters, sub-children, and many others.) beneath every visited head are later seen
throughout the journey.
D_taxonomy(J) = |D_seen(J)| / |D_exist(J)|
The ultimate deepening rating is a weighted mixture:
D(J) = λ₁ · D_taxonomy(J) + λ₂ · D_journey(J), λ₁ + λ₂ = 1.
Deepening lies in [0, 1].
Development (P)
Development measures directional motion by way of journey phases. For every transition, we compute:
Δᵢ = aᵢ₊₁ − aᵢ.
Solely shifting steps (Δᵢ ≠ 0) are thought of. Let wᵢ denote the relative significance of the present stage.
If Δᵢ > 0 (ahead motion):
cᵢ = wᵢ / Δᵢ
If Δᵢ < 0 (backward motion):
cᵢ = Δᵢ · wᵢ
The uncooked development rating is:
P_raw(J) = Σ cᵢ for all i the place Δᵢ ≠ 0
To sure the rating to[−1, +1], we apply a tanh normalization:
P(J) = (e^(P_raw) − e^(−P_raw)) / (e^(P_raw) + e^(−P_raw))
Making use of CDP Metrics to an Automotive Buyer Journey
To show how structured analysis works on real looking journeys, we generated an artificial automotive customer-journey dataset masking the principle phases of the shopper lifecycle.
Enter Information: Anchors and Journey Content material
The CDP framework makes use of two principal inputs: anchors, which outline journey phases, and customer-journey content material, which offers the messages to judge.
Anchors signify significant phases within the lifecycle, reminiscent of motivation, buy, supply, possession, and loyalty. Every anchor is augmented with a small set of consultant key phrases to floor it semantically. Anchors serve each as reference factors for taxonomy building and because the anticipated directional move used later within the Development metric.
anchor Phrases:
motivation exploration analysis discovery curiosity check drive wants evaluation expertise
buy financing comparability quotes mortgage negotiation credit score pre-approval deposit
supply paperwork signing deposit logistics handover activation
possession upkeep guarantee restore vendor help service inspections
loyalty suggestions satisfaction survey referral improve retention advocacyBuyer-journey content material consists of brief, action-oriented CRM-style messages (emails, calls, chats, in-person interactions) with various ranges of specificity and spanning a number of phases. Though this dataset is synthetically generated, anchor info isn’t used throughout taxonomy building or CDP scoring.
CJ messages:
Discover fashions that match your way of life and private objectives.
Take a digital tour to find key options and trims.
Evaluate physique kinds to evaluate house, consolation, and utility.
E book a check drive to expertise dealing with and visibility.
Use the wants evaluation to rank must-have options.
Filter fashions by vary, mpg, or towing to slim decisions.Taxonomy Building Outcomes
Right here, we utilized the taxonomy building course of to the automotive customer-engagement dataset. The determine beneath reveals the ensuing customer-journey taxonomy, constructed from message content material and anchor semantics.
Every top-level department corresponds to a journey anchor (L1), which represents main journey phases reminiscent of Motivation, Buy, Supply, Possession, and Loyalty.
Deeper ranges (L2, L3+) group messages by thematic similarity and rising specificity.
What the Taxonomy Reveals
Even on this compact dataset, the taxonomy highlights a number of useful patterns:
- Early-stage messages cluster round exploration and comparability, steadily narrowing towards concrete actions reminiscent of reserving a check drive.
- Buy-related content material separates naturally into monetary planning, doc dealing with, and finalization.
- Possession content material reveals a transparent development from upkeep scheduling to diagnostics, value estimation, and guarantee analysis.
- Loyalty content material shifts from transactional actions towards suggestions, upgrades, and advocacy.
Whereas these patterns align with how practitioners usually purpose about journeys, they come up instantly from the information reasonably than from predefined guidelines.
Why This Issues for Analysis
This taxonomy now offers a shared structural reference:
- Any buyer journey will be mapped as a path by way of the tree.
- Motion throughout branches, depth ranges, and anchors turns into measurable.
- Continuity, Deepening, and Development are not summary ideas; they now correspond to concrete structural adjustments.
Within the subsequent part, we use this taxonomy to map actual journey examples and compute CDP scores in steps.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is constructed, evaluating a buyer journey turns into a structural drawback.
Every journey is represented as an ordered sequence of customer-facing messages.
As an alternative of judging these messages in isolation, we mission them onto the taxonomy and analyze the ensuing path.
Formally, a journey J = (x₁, x₂, …, xₙ) is mapped to a sequence of taxonomy nodes: (x₁→v₁),(x₂→v₂),…,(xₙ→vₙ) the place every vᵢ is the closest taxonomy node primarily based on embedding similarity.
A Step-by-Step Walkthrough: From Journey Textual content to CDP Scores
To make the CDP framework concrete, let’s stroll by way of a single buyer journey instance and present how it’s evaluated step-by-step.
Step 1 — The Buyer Journey Enter
We start with an ordered sequence of customer-facing messages generated by an LLM.
Every message represents a touchpoint in a sensible automotive buyer journey:
journey = ['Take a virtual tour to discover key features and trims.';
'We found a time slot for a test drive that fits your schedule.';
'Upload your income verification and ID to finalize the pre-approval decision.';
'Estimate costs for upcoming maintenance items.';
'Track retention offers as your lease end nears.';
'Add plates and registration info before handover.']Step 2 — Mapping the Journey into the Taxonomy
For structural analysis, every journey step is mapped into the customer-journey taxonomy. Utilizing textual content embeddings, every message is matched to its closest taxonomy node. This produces a journey map (jmap), a structured illustration of how the journey traverses the taxonomy.
Desk 2: Every message is assigned to an anchor (stage), a thematic head, and a depth stage within the taxonomy primarily based on semantic similarity within the shared embedding house. This desk acts as the muse for all future evaluations.
Step 3 — Making use of CDP Metrics to This Journey
As soon as the journey is mapped, we compute Continuity, Deepening, and Development deterministically from step-to-step transitions.
Desk 3: Every row represents a transition between consecutive journey steps, annotated with alerts for continuity, deepening, and development.
Ultimate CDP scores (this journey):
Taken collectively, the CDP alerts point out a journey that’s largely coherent and forward-moving, with one clear second of
deepening and one seen structural regression. Importantly, these insights are derived solely from construction, not from
stylistic judgments in regards to the textual content.
Conclusion: From Scores to Profitable Journeys
Continuity, Deepening, and Development are decided by construction and will be utilized wherever LLMs generate multi-step
content material:
- to match various journeys generated by totally different prompts or fashions,
- to supply automated suggestions for bettering journey era over time.
On this approach, CDP scores provide structural suggestions for LLMs. They complement, reasonably than exchange, stylistic or fluency-based analysis by offering alerts that mirror enterprise logic and buyer expertise.
Though this text focuses on automotive commerce, the idea is broadly relevant. Any system that generates ordered, goal-oriented content material requires robust structural foundations.
Massive language fashions are already able to producing fluent, persuasive textual content.
The higher problem is guaranteeing that textual content sequences kind coherent narratives that align with enterprise logic and person expertise.
CDP offers a option to make construction specific, measurable, and actionable.
Thanks for staying with me by way of this journey. Hopefully, this idea helps you assume otherwise about evaluating AI-generated sequences and conjures up you to deal with construction as a main sign in your individual methods. All logic offered on this article is carried out within the accompanying Python code on GitHub. You probably have any questions or feedback, please go away them within the feedback part or attain out by way of LinkedIn.
References
Brodie, R. J., et al. (2011). Buyer engagement: Conceptual area, basic propositions, and implications for analysis.
