AI chatbots have made such a splash because they can be interacted with using natural language. Instead of having to match programming syntax, you just straight up tell it what you want to do. At first glance, this might seem like a simple process: you write or upload something, submit it to the AI, and it analyzes what you sent and comes back with a response. And as an extremely generalized example, this is true.
But what happens to that data after you hit send?
There's a lot of things that happen in the background that are more complicated than a simple send and receive. The process involves multiple steps where data is processed and retained, and understanding what's happening can help organizations decide what type of data is appropriate to use for different AI tools and what safeguards should be put in place to protect sensitive data.
When you type a message or upload a file into AI, it analyzes and processes that data to give you what you ask for. But how exactly does that work? These machines don’t “understand” words and context the same way that humans do. So what are they doing instead?
Here’s a simplified version of what’s happening:
Different companies handle this process in different ways, but in general, turning a prompt into a response follows a similar framework, which has impacts on the security and privacy of your data.
From the first step of the process of getting a response, there are risks for data privacy. One of the most important ones is that the data leaves your environment, and is then processed by the company that developed the AI tool that you're using. From there, they can do use that data for the reasons outlined in their terms and conditions.
These reasons vary vastly depending on the company, tool type, and account tier. Free tiers of course have the least amount of privacy and protections, and AI companies train models off your conversations unless you disable certain settings. Higher tiers offer more protections, especially with a DPA, access controls, and detailed logs.
However, even Enterprise tiers don't necessarily guarantee complete data privacy and control. They can still have issues in areas such as data sovereignty and when third-party plugins are used.
Enterprise agreements for all of the major AI clients have strong data privacy clauses (except when voluntarily opting into the training/improvement programs) that make using first-party data far less risky with them. One concern is where data processing activities happen. This important for strict compliance standards where cross-border processing isn't allowed, which can apply to specific specific datasets like Australian health records, where no processing or handling or records can occur outside of Australia.
Some AI companies can't guarantee these types of restrictions can be met. Other times, they can with a separate addition on top of the standard enterprise detail. A paper by Quillo, a legal company is Australia, explains this in detail in their paper about Enterprise AI and confidential client material. While the focus is on tightly-controlled legal field, their hypothesis applies across industries: make sure to check the T's and C's for data processing and retention instead of assuming Enterprise has you covered.
But even with an agreement in place that covers everything, that only pertains to the company you have the agreement with, and doesn't cover third-party add-ons like plugins and connectors, which are becoming more common to conveniently share data.
Adding Plugins can significantly change the default data processing. Many plugins work as integrations that let you make use of other products and services directly within a chatbot. These aren’t necessarily AI themselves—often they aren’t—but instead hook in to other products, which then can have access to the data submitted to the LLM.
When these have too much access, the organization could be unknowingly sending sensitive data to these SaaS providers and break compliance. This can include sensitive or confidential information, and if these plug-ins don’t have the same level of data governance and are used in the chat, they can potentially get access to that information. And even if they aren't accessing the data directly, they can retain it somewhere outside of your control.
A study on data privacy on LLMs found that this is an issue for many plugins. The study found that some plugins collected much more user data then they needed, including sensitive and personal information. Some also didn't give details on how they use and process user data provided to them. So even if the AI tool itself meets your data privacy requirements, be sure to double-check the details for any plug-ins before adding them.
A lot goes into turning prompts into responses, including multiple processing and refinement points where data leaves your environment. Protecting sensitive data from going to AI is important, but it's not easy to do. It requires a proper AI policy that the team chooses to follow because they understand the risks.
One method to help strengthen your security is to use a dedicated AI governance tool like Oveka. Oveka finds and monitors AI use throughout the organization and lets you know which tools are being used and with what data. From there, you can decide to restrict what kinds of data can go into specific tools, ban unauthorized AI completely, or simply observe how AI is being used. It can also checks for sensitive data where it shouldn't be, including third-party SaaS platforms that may be connected through AI integrations and plugins.
Oveka is a simple, holistic approach to data security and AI governance. To find out if Oveka is right for you, get in touch with us and book your demo now.