AI chatbots have made such a splash because they can be interacted with using natural language. Instead of having to match programming syntax, you just straight up tell it what you want to do. At first glance, this might seem like a simple process: you write or upload something, submit it to the AI, and it analyzes what you sent and comes back with a response. And as an extremely generalized example, this is true.
But what happens to that data after you hit send?
There's a lot of things that happen in the background that are more complicated than a simple send and receive. The process involves multiple steps where data is processed and retained, and understanding what's happening can help organizations decide what type of data is appropriate to use for different AI tools and what safeguards should be put in place to protect sensitive data.
How does AI turn prompts into responses?
When you type a message or upload a file into AI, it analyzes and processes that data to give you what you ask for. But how exactly does that work? These machines don’t “understand” words and context the same way that humans do. So what are they doing instead?
Here’s a simplified version of what’s happening:
-
The user writes and submits a prompt.
With your typical tools, there’s no restrictions on what this can be. It can be as simple as asking if a tomato is a vegetable or a fruit out of curiosity or submitting confidential financial information to try and see if the company will still exist in a year.
By default, there's no control over what goes into the LLM, except in the case of abusive/dangerous requests like homemade weapon making. Usually to restrict data that gets ingested, you'll need to pair the AI with a tool that puts guardrails around prompts so that certain queries are flagged or denied. -
Prompt data passes through third-party infrastructure and is converted to “tokens” along the way.
AI doesn't read words like humans, it turns prompts into tokens that it can understand instead. For your natural language prompts, one token is usually around 4 characters. These tokens are how paid models are priced, and are used both for the prompt and for the generated response, which is why you see things like Caveman syntax to reduce token usage, and consequently, the cost.
The converted tokens that are sent to the chatbot are what it actually uses to generate the response. It's the information of the prompt itself and other context like project settings it pulls from. So you typing a short sentence “How do I make liquid money” will extrapolate to include the context (that in this project it would be referring to liquidity instead of coins made out of water, for instance) and give an appropriate response. With this final version of the prompt, LLMs compare the prompt tokens to the training dataset—estimated to be “trillions of tokens” for OpenAI—to determine context. -
The training data set gives context.
Context is given by showing which words/numbers/characters are associated to other ones. The more association it finds, the higher the “weight” of that word to be the best choice. The context changes depending on the individual tokens and the order they're in.
Because of the huge amount of data being crunched, patterns emerge. The more data the model is trained, the greater the accuracy, but the more compute that goes into it training and as a result, the “larger” the model. This is why you’ll see some people comparing them to “glorified autocomplete” and can force wrong answers in some models with loaded questions like “A surgeon says ‘I can’t operate on him—he’s my son.’ How can that be?” and get a confident ‘The surgeon is his mother!’ because the framing resembles the classic riddle. -
The response is generated.
Responses are based on weights in context. So “The best dinner food is” will have all tokens “weighted” in this context, with weights in the training data based on how likely the next response is. So “Pizza” may be the most weighted and show up here. This varies widely by the model type and how the companies handle prompts. Claude, for instance, applies probabilities to the weights and does a "random roll" so if pizza was 80% and tacos was 10%, tacos will show up sometimes, which is why the exact same prompt gives different results.
Reasoning goes a step beyond this. “Thinking” modes break down the prompt into smaller pieces.“Lite” or “Simple” modes of LLMs will typically generate a response in a single pass. In these basic modes, they simply compare the contextual prompt to the weights in the training data and respond with the most likely option. "Thinking" modes do multiple passes. These are checked multiple times to weed out biases and assumptions, so the results are often better. A “thinking” mode is likely to catch simple mistakes like the riddle one (nowhere did it say the boy’s father was dead) so you get better accuracy at the cost of more tokens. -
The conversation is retained for later reference.
Once the prompt information is processed and the AI generates a response, the entire conversation is retained on the company servers. For the user, this allows the chatbot to do things like reference past conversations and files that were uploaded. For the AI company, it gives them more data and protects their interests if legal issues or newsorthy stories come up. It's convenience with a tradeoff for privacy.
In general, conversations are isolated to the user account where other humans can't access the data, but this isn't guaranteed. AI companies all have provisions for their staff to be able access data in a user's account for cases of safety or abuse. This can be turned off for higher tiers in certain products, but most don't have an opt-out even for higher tiers unless you have a ZDR agreement in place, and even then it's not a guarantee.
Different companies handle this process in different ways, but in general, turning a prompt into a response follows a similar framework, which has impacts on the security and privacy of your data.
The prompt process and data privacy
From the first step of the process of getting a response, there are risks for data privacy. One of the most important ones is that the data leaves your environment, and is then processed by the company that developed the AI tool that you're using. From there, they can do use that data for the reasons outlined in their terms and conditions.
These reasons vary vastly depending on the company, tool type, and account tier. Free tiers of course have the least amount of privacy and protections, and AI companies train models off your conversations unless you disable certain settings. Higher tiers offer more protections, especially with a DPA, access controls, and detailed logs.
However, even Enterprise tiers don't necessarily guarantee complete data privacy and control. They can still have issues in areas such as data sovereignty and when third-party plugins are used.
AI and Data Sovereignty
Enterprise agreements for all of the major AI clients have strong data privacy clauses (except when voluntarily opting into the training/improvement programs) that make using first-party data far less risky with them. One concern is where data processing activities happen. This important for strict compliance standards where cross-border processing isn't allowed, which can apply to specific specific datasets like Australian health records, where no processing or handling or records can occur outside of Australia.
Some AI companies can't guarantee these types of restrictions can be met. Other times, they can with a separate addition on top of the standard enterprise detail. A paper by Quillo, a legal company is Australia, explains this in detail in their paper about Enterprise AI and confidential client material. While the focus is on tightly-controlled legal field, their hypothesis applies across industries: make sure to check the T's and C's for data processing and retention instead of assuming Enterprise has you covered.
But even with an agreement in place that covers everything, that only pertains to the company you have the agreement with, and doesn't cover third-party add-ons like plugins and connectors, which are becoming more common to conveniently share data.
Plugins and connectors
Adding Plugins can significantly change the default data processing. Many plugins work as integrations that let you make use of other products and services directly within a chatbot. These aren’t necessarily AI themselves—often they aren’t—but instead hook in to other products, which then can have access to the data submitted to the LLM.
When these have too much access, the organization could be unknowingly sending sensitive data to these SaaS providers and break compliance. This can include sensitive or confidential information, and if these plug-ins don’t have the same level of data governance and are used in the chat, they can potentially get access to that information. And even if they aren't accessing the data directly, they can retain it somewhere outside of your control.
A study on data privacy on LLMs found that this is an issue for many plugins. The study found that some plugins collected much more user data then they needed, including sensitive and personal information. Some also didn't give details on how they use and process user data provided to them. So even if the AI tool itself meets your data privacy requirements, be sure to double-check the details for any plug-ins before adding them.
Ensure your team uses AI safely with Oveka
A lot goes into turning prompts into responses, including multiple processing and refinement points where data leaves your environment. Protecting sensitive data from going to AI is important, but it's not easy to do. It requires a proper AI policy that the team chooses to follow because they understand the risks.
One method to help strengthen your security is to use a dedicated AI governance tool like Oveka. Oveka finds and monitors AI use throughout the organization and lets you know which tools are being used and with what data. From there, you can decide to restrict what kinds of data can go into specific tools, ban unauthorized AI completely, or simply observe how AI is being used. It can also checks for sensitive data where it shouldn't be, including third-party SaaS platforms that may be connected through AI integrations and plugins.
Oveka is a simple, holistic approach to data security and AI governance. To find out if Oveka is right for you, get in touch with us and book your demo now.