---
title: LLM Drift, Prompt Drift & Cascading
description: "LLM Drift, Prompt Drift & Cascading\n\nPrompt Chaining can be performed manually or automatically; manual entails crafting chains by hand, via a GUI chain building tool. Autonomous Agents create chains on the fly as they execute while making use of the tools at their disposal. Both these approaches are susceptible to cascading, LLM & prompt Drift."
image: https://blog.kore.ai/hubfs/2024-09-19_07-27-11.jpg
---

[![Kore.ai_logo-latest](https://blog.kore.ai/hubfs/Kore.ai_logo-latest.svg "Kore.ai_logo-latest")](https://blog.kore.ai/)

- AI Offerings 
    - [AI for Work](https://kore.ai/ai-for-work/)
    - [AI for Service](https://kore.ai/ai-for-service/)
    - [AI for Process](https://kore.ai/ai-for-process/)
- Agent Platform 
    - [Agentic Platform](https://kore.ai/agentic-platform/)
    - Capabilities 
          - [Multi Agent Orchestration](https://kore.ai/agent-platform/multi-agent-orchestration/)
          - [AI Engineering Tools](https://kore.ai/agent-platform/ai-engineering-tools/)
          - [Search and Data AI](https://kore.ai/agent-platform/search-data-ai/)
          - [AI Security, Compliance,
& Governance](https://kore.ai/agent-platform/ai-security-compliance-governance/)
          - [No-Code & Pro-Code Tools](https://kore.ai/agent-platform/no-code-development-tools/)
          - [Integrations](https://kore.ai/agent-platform/integrations/)
    - AI Insights 
          - [Agentic Continuum](https://kore.ai/beyond-ai-agents-the-rise-of-agentic-x-and-the-future-of-intelligent-automation/)
          - [Intelligent AI Agent Networks](https://kore.ai/the-future-of-enterprise-ai-why-you-need-to-start-thinking-about-agent-networks-today/)
    - Whitepaper 
          - [Beyond AI Islands](https://kore.ai/gtd-bcg-beyond-ai-islands/)
- [Agent Marketplace](https://kore.ai/marketplace/)
- Resources 
    - Knowledgebase 
          - [Blog](https://blog.kore.ai/)
          - [Whitepapers](https://kore.ai/white-papers/)
          - [Research Analysis](https://kore.ai/research-analysis/)
          - [Webinars](https://kore.ai/webinars/)
    - Technical 
          - [Documentation](https://developer.kore.ai/virtual-assistants/)
    - Introductions 
          - [Generative AI 101](https://kore.ai/generative-ai-101-chapters/?utm_source=website&utm_medium=content&utm_campaign=main-nav)
          - [Responsible AI Framework](https://kore.ai/responsible-ai-framework/)
- Company 
    - Company 
          - [About Us](https://kore.ai/about-us/)
          - [Contact Us](https://kore.ai/contact-us/)
          - [Leadership](https://kore.ai/leadership/)
          - [Careers](https://kore.ai/careers/)
          - [Newsroom](https://kore.ai/press/)
          - [Events](https://kore.ai/events/)
    - Reviews 
          - [Analyst Recognition](https://kore.ai/analyst-recognition/)
          - [Customer Stories](https://kore.ai/customer-stories/)
    - Partners 
          - [Partner Program](https://kore.ai/partners/)
          - [Become a Partner](https://info.kore.ai/partner-si)
          - [Find a Partner](https://kore.ai/find-a-partner/)
- [Sign In](https://kore.ai/sign-in/)

[![Kore.ai_logo-latest](https://blog.kore.ai/hubfs/Kore.ai_logo-latest.svg "Kore.ai_logo-latest")](https://blog.kore.ai/)

<https://blog.kore.ai/cobus-greyling/llm-drift-prompt-drift-cascading-1#growfast-header__mobile-nav__mmenu>

[![Kore.ai_logo-latest](https://blog.kore.ai/hubfs/Kore.ai_logo-latest.svg "Kore.ai_logo-latest")](https://blog.kore.ai/)

- AI Offerings 
    - [AI for Work](https://kore.ai/ai-for-work/)
    - [AI for Service](https://kore.ai/ai-for-service/)
    - [AI for Process](https://kore.ai/ai-for-process/)
- Agent Platform 
    - [Agentic Platform](https://kore.ai/agentic-platform/)
    - Capabilities 
          - [Multi Agent Orchestration](https://kore.ai/agent-platform/multi-agent-orchestration/)
          - [AI Engineering Tools](https://kore.ai/agent-platform/ai-engineering-tools/)
          - [Search and Data AI](https://kore.ai/agent-platform/search-data-ai/)
          - [AI Security, Compliance,
& Governance](https://kore.ai/agent-platform/ai-security-compliance-governance/)
          - [No-Code & Pro-Code Tools](https://kore.ai/agent-platform/no-code-development-tools/)
          - [Integrations](https://kore.ai/agent-platform/integrations/)
    - AI Insights 
          - [Agentic Continuum](https://kore.ai/beyond-ai-agents-the-rise-of-agentic-x-and-the-future-of-intelligent-automation/)
          - [Intelligent AI Agent Networks](https://kore.ai/the-future-of-enterprise-ai-why-you-need-to-start-thinking-about-agent-networks-today/)
    - Whitepaper 
          - [Beyond AI Islands](https://kore.ai/gtd-bcg-beyond-ai-islands/)
- [Agent Marketplace](https://kore.ai/marketplace/)
- Resources 
    - Knowledgebase 
          - [Blog](https://blog.kore.ai/)
          - [Whitepapers](https://kore.ai/white-papers/)
          - [Research Analysis](https://kore.ai/research-analysis/)
          - [Webinars](https://kore.ai/webinars/)
    - Technical 
          - [Documentation](https://developer.kore.ai/virtual-assistants/)
    - Introductions 
          - [Generative AI 101](https://kore.ai/generative-ai-101-chapters/?utm_source=website&utm_medium=content&utm_campaign=main-nav)
          - [Responsible AI Framework](https://kore.ai/responsible-ai-framework/)
- Company 
    - Company 
          - [About Us](https://kore.ai/about-us/)
          - [Contact Us](https://kore.ai/contact-us/)
          - [Leadership](https://kore.ai/leadership/)
          - [Careers](https://kore.ai/careers/)
          - [Newsroom](https://kore.ai/press/)
          - [Events](https://kore.ai/events/)
    - Reviews 
          - [Analyst Recognition](https://kore.ai/analyst-recognition/)
          - [Customer Stories](https://kore.ai/customer-stories/)
    - Partners 
          - [Partner Program](https://kore.ai/partners/)
          - [Become a Partner](https://info.kore.ai/partner-si)
          - [Find a Partner](https://kore.ai/find-a-partner/)
- [Sign In](https://kore.ai/sign-in/)

[![Kore.ai_logo-latest](https://blog.kore.ai/hubfs/Kore.ai_logo-latest.svg "Kore.ai_logo-latest")](https://blog.kore.ai/)

<https://blog.kore.ai/cobus-greyling/llm-drift-prompt-drift-cascading-1#growfast-header__mobile-nav__mmenu>

- AI Offerings 
    - [AI for Work](https://kore.ai/ai-for-work/)
    - [AI for Service](https://kore.ai/ai-for-service/)
    - [AI for Process](https://kore.ai/ai-for-process/)
- Agent Platform 
    - [Agentic Platform](https://kore.ai/agentic-platform/)
    - Capabilities 
          - [Multi Agent Orchestration](https://kore.ai/agent-platform/multi-agent-orchestration/)
          - [AI Engineering Tools](https://kore.ai/agent-platform/ai-engineering-tools/)
          - [Search and Data AI](https://kore.ai/agent-platform/search-data-ai/)
          - [AI Security, Compliance,
& Governance](https://kore.ai/agent-platform/ai-security-compliance-governance/)
          - [No-Code & Pro-Code Tools](https://kore.ai/agent-platform/no-code-development-tools/)
          - [Integrations](https://kore.ai/agent-platform/integrations/)
    - AI Insights 
          - [Agentic Continuum](https://kore.ai/beyond-ai-agents-the-rise-of-agentic-x-and-the-future-of-intelligent-automation/)
          - [Intelligent AI Agent Networks](https://kore.ai/the-future-of-enterprise-ai-why-you-need-to-start-thinking-about-agent-networks-today/)
    - Whitepaper 
          - [Beyond AI Islands](https://kore.ai/gtd-bcg-beyond-ai-islands/)
- [Agent Marketplace](https://kore.ai/marketplace/)
- Resources 
    - Knowledgebase 
          - [Blog](https://blog.kore.ai/)
          - [Whitepapers](https://kore.ai/white-papers/)
          - [Research Analysis](https://kore.ai/research-analysis/)
          - [Webinars](https://kore.ai/webinars/)
    - Technical 
          - [Documentation](https://developer.kore.ai/virtual-assistants/)
    - Introductions 
          - [Generative AI 101](https://kore.ai/generative-ai-101-chapters/?utm_source=website&utm_medium=content&utm_campaign=main-nav)
          - [Responsible AI Framework](https://kore.ai/responsible-ai-framework/)
- Company 
    - Company 
          - [About Us](https://kore.ai/about-us/)
          - [Contact Us](https://kore.ai/contact-us/)
          - [Leadership](https://kore.ai/leadership/)
          - [Careers](https://kore.ai/careers/)
          - [Newsroom](https://kore.ai/press/)
          - [Events](https://kore.ai/events/)
    - Reviews 
          - [Analyst Recognition](https://kore.ai/analyst-recognition/)
          - [Customer Stories](https://kore.ai/customer-stories/)
    - Partners 
          - [Partner Program](https://kore.ai/partners/)
          - [Become a Partner](https://info.kore.ai/partner-si)
          - [Find a Partner](https://kore.ai/find-a-partner/)
- [Sign In](https://kore.ai/sign-in/)

[Try for free](https://kore.ai/try-kore-platform-for-free/)

# LLM Drift, Prompt Drift & Cascading

 Author: [Cobus Greyling](https://blog.kore.ai/cobus-greyling/author/cobus-greyling)

- [Tweet](https://twitter.com/share)

###### Prompt Chaining can be performed manually or automatically; manual entails crafting chains by hand, via a GUI chain building tool. Autonomous Agents create chains on the fly as they execute while making use of the tools at their disposal. Both these approaches are susceptible to cascading, LLM & prompt Drift.

 

# LLM Drift

LLM Drift is definite changes in LLM responses over a relatively short period of time. This is not related to LLMs being in essence non-deterministic or related to slight prompt engineering *wording* changes; but rather fundamental changes to the LLM.

A recent [study](https://arxiv.org/pdf/2307.09009.pdf) found that over a period of *four months*, the response accuracy of GPT-4 and GPT-3.5 fluctuates considerably in the positive but more alarming…*negatively*.

The study found that both GPT-3.5 and GPT-4 varied significantly and that there was performance degradation on some tasks.

> *Our findings highlight the need to continuously monitor LLMs behaviour over time. —* [*Source*](https://arxiv.org/pdf/2307.09009.pdf)

The schematic below shows the fluctuation in model accuracy over a period of four months. it some cases the deprecation is quite stark, being more than 60% loss in accuracy.

![](https://miro.medium.com/v2/resize:fit:2000/0*QaeCeH2vPaE6xm2_.png)

[Source](https://arxiv.org/pdf/2307.09009.pdf)

# Prompt Drift

The output of LLMs are non-deterministic, this means that the exact input, to the same LLM, at different times, will most probably yield different responses over time.

In essence this is not a problem, and wording can differ while the ground truth remains the same.

However, there are instances where there are aberrations in the response of the LLM. For instance, LLMs are deprecated and migration is often necessitated, as we saw recently with OpenAI deprecating a number of models. Hence the prompt remains the same, but the underlying model referenced change.

The data which is injected into the prompt at inference might also be different at times. Suffice to say all of these factors contribute to a phenomenon known as ***prompt drift***.

> *Prompt Drift is the phenomenon where a prompt yields different responses over time due to model changes, model migration or changes in prompt-injection data at inference.*

# In short, prompt drift can be caused by:

- Model-inspired tangents,
- Incorrect problem extraction,
- LLM randomness and creative surprises.

There has been the emergence of Prompt Management and testing interfaces, like for instance GALE introduced by Kore.ai.

There is a definite market-need to ensure generative applications (Gen-Apps) can be tested prior to large language model migration/deprecation.

And if a model could be largely agnostic to the underlying LLM, so much the better. One avenue to move closer to achieving this, is leveraging in-context learning (ICL) capabilities of Large Language Models.

# Cascading

Cascading is when an aberration or deviation is introduced by one of the nodes in a chain, and this unexpected *exception* is carried over to the next node, where the exception will most probably be exacerbated.

With each node output deviating further-and-further from the intended outcome.

This phenomenon is commonly referred to as cascading.

# Considering the image below:

1. The user input can be unexpected or unplanned in the chained application, hence producing an unforeseen output from the node.
2. The previous node’s output can be inaccurate or produce a degree of deviation which is exacerbated in the current node.
3. The LLM Response can also be unexpected, due to the fact that LLMs are non-deterministic. Point three is where prompt drift or LLM drift can be introduced.
4. And the output from Node 2 is then carried over and cascading of the deviation is caused.

![](https://miro.medium.com/v2/resize:fit:2000/1*adD31ltQkpe0d68cTC9ONw.png)

# In Closing

Prompt chaining should not viewed in isolation, but rather consider [Prompt Engineering](https://medium.com/@cobusgreyling/eight-prompt-engineering-implementations-fc361fdc87b) as a discipline which consists of several legs.

The wording or technique followed when prompting the LLM is also important and has a demonstrable effect on the quality of the output.

Prompt Engineering is the foundation of *Chaining* and the discipline of Prompt Engineering is very simple and accessible.

However, as the LLM landscape develops, prompts are becoming programable (templates and context injection via RAG) and incorporated into increasing complex structures.

Hence chaining are being supported by elements like [Agents](https://medium.com/@cobusgreyling/agents-f444d165024), [Pipelines](https://medium.com/@cobusgreyling/generative-ai-prompt-pipelines-de82d803ffc2), [Chain-of-Thought Reasoning](https://medium.com/@cobusgreyling/chain-of-thought-prompting-in-llms-1077164edf97), etc.

 

---

<https://cobusgreyling.medium.com/subscribe?source=post_page-----5a2ea2a5c455-------------------------------->

##### [How is ChatGPT's behavior changing over time? arxiv.org ---](https://arxiv.org/abs/2307.09009?source=post_page-----5a2ea2a5c455--------------------------------)

[PromptChainer arxiv.org ---  ](https://arxiv.org/abs/2203.06566?source=post_page-----5a2ea2a5c455--------------------------------)

##### [Language Model Cascades arxiv.org ---  ](https://arxiv.org/abs/2207.10342?source=post_page-----5a2ea2a5c455--------------------------------)

##### [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning arxiv.org  ](https://arxiv.org/abs/2308.08747?source=post_page-----5a2ea2a5c455--------------------------------)

[![forrester-blog-img-1](https://blog.kore.ai/hubfs/forrester-blog-img-1.jpg "forrester-blog-img-1")](https://info.kore.ai/forrester-wave-report-conversational-ai-2024#access-the-report)

### Subscribe to Blog Updates

[![Kore.ai named a leader in Gartner Magic Quadrant](https://no-cache.hubspot.com/cta/default/3458671/43a5219e-9a26-4203-b313-6eaaa9dbd2e7.png)](https://cta-redirect.hubspot.com/cta/redirect/3458671/43a5219e-9a26-4203-b313-6eaaa9dbd2e7)



### Recent Posts

### Follow us on LinkedIn

## Recent Posts

[![](https://blog.kore.ai/hubfs/2025-05-20_07-56-17.jpg)](https://blog.kore.ai/cobus-greyling/llms-training-slms-2) [LLMs Training SLMs](https://blog.kore.ai/cobus-greyling/llms-training-slms-2)

[![](https://blog.kore.ai/hubfs/2025-05-15_08-50-52.png)](https://blog.kore.ai/cobus-greyling/large-impact-the-rise-of-small-language-models-1) [Large Impact: The Rise of Small Language Models](https://blog.kore.ai/cobus-greyling/large-impact-the-rise-of-small-language-models-1)

[![](https://blog.kore.ai/hubfs/2025-05-12_07-26-19%20SDG%20CG.png)](https://blog.kore.ai/cobus-greyling/rag-data-privacy-attack-methods-safe-prompts-1) [RAG, Data Privacy, Attack Methods & Safe-Prompts](https://blog.kore.ai/cobus-greyling/rag-data-privacy-attack-methods-safe-prompts-1)

[![](https://blog.kore.ai/hubfs/2025-05-09_06-38-12.png)](https://blog.kore.ai/cobus-greyling/multihop-rag-1) [MultiHop-RAG](https://blog.kore.ai/cobus-greyling/multihop-rag-1)

[Try for Free](https://kore.ai/try-kore-platform-for-free/)

![leftangle](https://blog.kore.ai/hubfs/Imported_Blog_Media/1608504-64.png)

![rightangle](https://blog.kore.ai/hubfs/Imported_Blog_Media/1608505-64.png) 

[![Request a Demo](https://no-cache.hubspot.com/cta/default/3458671/59161269-5385-4359-92bf-34a88643c829.png)](https://cta-redirect.hubspot.com/cta/redirect/3458671/59161269-5385-4359-92bf-34a88643c829) [![Build a Virtual Assistant](https://no-cache.hubspot.com/cta/default/3458671/e87d004f-e996-44e0-925f-ef6f76e50b30.png)](https://cta-redirect.hubspot.com/cta/redirect/3458671/e87d004f-e996-44e0-925f-ef6f76e50b30) [![Resources](https://no-cache.hubspot.com/cta/default/3458671/63c59009-df8c-42d7-91dd-4c7111a920ce.png)](https://cta-redirect.hubspot.com/cta/redirect/3458671/63c59009-df8c-42d7-91dd-4c7111a920ce)

![Kore.ai_Logo_Black (12)](https://blog.kore.ai/hs-fs/hubfs/Kore.ai_Logo_Black%20(12).png?width=132&height=32&name=Kore.ai_Logo_Black%20(12).png)

 

##### Let’s work together

Get answers and a customized quote for your projects

[Submit RFP](https://info.kore.ai/request-quote)

##### AI OFFERINGS

- [AI for Work](https://kore.ai/ai-for-work/)
- [AI for Process](https://kore.ai/ai-for-process/)
- [AI for Service](https://kore.ai/ai-for-service/)

##### DEVELOPERS

- [Community](https://community.kore.ai/)
- [Docs](https://docs.kore.ai/)
- [Kore.ai Academy](https://academy.kore.ai/)
- [Status](https://status.kore.com/?__hstc=59894770.fe7b8349b8ca5b1ecd3df202af95e5a3.1752466536142.1752466536142.1752820250645.2&__hssc=59894770.5.1752820250645&__hsfp=945770823)
- [Changelog](https://docs.kore.ai/xo/getting-started/whats-new-in-xo-platform/)

##### KORE.AI AGENT PLATFORM

- [Agent Platform](https://kore.ai/agent-platform/)
- [Multi Agent Orchestration](https://kore.ai/agent-platform/multi-agent-orchestration/)
- [AI Engineering Tools](https://kore.ai/agent-platform/ai-engineering-tools/)
- [Search and Data AI](https://kore.ai/agent-platform/search-data-ai/)
- [AI Security, Compliance,
& Governance](https://kore.ai/agent-platform/ai-security-compliance-governance/)  
  <https://kore.ai/agent-platform/search-data-ai/>
- [No-Code & Pro-Code Tools](https://kore.ai/agent-platform/no-code-development-tools/)
- [Observability](https://kore.ai/agent-platform/observability/)
- [Integrations](https://kore.ai/agent-platform/observability/)

##### RESOURCES

- [Blog](https://blog.kore.ai/)
- [Webinars](https://kore.ai/webinars/)
- [Case Studies](https://kore.ai/customer-stories/)

##### SOLUTIONS

- [Banking](https://kore.ai/ai-for-service/banking/)
- [Retail](https://kore.ai/ai-for-service/retail/)
- [Healthcare](https://kore.ai/ai-for-service/healthcare/)
- [HR](https://kore.ai/ai-for-work/hr/)
- [IT](https://kore.ai/ai-for-work/it/)
- [Recruiting](https://kore.ai/ai-for-work/recruiting/)

##### Support

- [Partner Program](https://kore.ai/partners/)
- [Our Services](https://kore.ai/services/)
- [Get Support](https://kore.ai/support/)

- New York, USA
- Orlando, USA
- London, UK
- Frankfurt, Germany
- Hyderabad, India
- Seoul, South Korea
- Tokyo, Japan

[Privacy Policy](https://kore.ai/privacy-policy/)    [Cookie Policy    ](https://kore.ai/cookie-policy/)[Terms of Service    ](https://kore.ai/terms-of-service/)[Acceptable Use Policy    ](https://kore.ai/acceptable-use-policy/)

© 2026 Kore.ai Inc. All trademarks are property of their respective owners.

[](https://www.facebook.com/KoreDotAI)

Facebook

[](https://twitter.com/koredotai)

Twitter

[](https://www.linkedin.com/company/kore-inc/)

LinkedIn

[](https://www.youtube.com/Koreai)

YouTube

<http://t.me/koredotai>

Telegram

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Cobus Greyling",
    "url" : "https://blog.kore.ai/cobus-greyling/author/cobus-greyling"
  },
  "dateModified" : "2024-09-19T05:38:08.705Z",
  "datePublished" : "2024-09-19T05:26:18.000Z",
  "headline" : "LLM Drift, Prompt Drift & Cascading",
  "image" : [ "https://blog.kore.ai/hubfs/2024-09-19_07-27-11.jpg" ],
  "mainEntityOfPage" : {
    "@id" : "https://blog.kore.ai/cobus-greyling/llm-drift-prompt-drift-cascading-1",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://blog.kore.ai/hubfs/New%20Logo_Kore.png"
    },
    "name" : "Kore.ai, Inc."
  }
}
```

```json
{
 
     "@context": "http://schema.org",
     "@type": "BlogPosting",
     "headline": "LLM Drift, Prompt Drift &amp; Cascading",
     "image": {
          "@type": "ImageObject",
          "url": "https://info.kore.ai/hubfs/2024-09-19_07-27-11.jpg"
     },
     "datePublished": "2024-09-19 05:26:18",
     "dateModified": "September 19, 2024, 5:38:08 AM",
     "author": {
         "@type": "Person",
         "name": "Cobus Greyling"
     },
     "publisher": {
         "@type": "Organization",
         "name": "Kore.ai",
         "logo": {
             "@type": "ImageObject",
             "url": "https://info.kore.ai/hubfs/kore.ai_logo.svg"
         }
     },
     "description": "LLM Drift, Prompt Drift & Cascading

Prompt Chaining can be performed manually or automatically; manual entails crafting chains by hand, via a GUI chain building tool. Autonomous Agents create chains on the fly as they execute while making use of the tools at their disposal. Both these approaches are susceptible to cascading, LLM & prompt Drift."
    
 }
```