Head of Strategy and Innovation at Sydney Water

Luther Uthayakumaran, Head of Strategy and Innovation at Sydney Water

One of the earliest episodes of the television series X Files aired in the 90s was called ‘The Ghost in the Machine’ and was on runaway artificial intelligence (AI). When maverick ‘special agent’ Fox Mulder suspects that computer software, rather than its developer, was responsible for a killing, his skeptical partner Dana Scully responds, "But that sort of AI is several years away.” This has been the industry adage for several years; that ‘true AI is always three years away.’  However, the arrival of Language Learning Models (LLMs) has caused tremor waves, to say the least. Although, whether LLMs are actually artificial intelligence remains largely a matter of interpretation. While governments and policy experts are more concerned about things like new modes of warfare, threats to democracy from misinformation, behavior modifications, reinforced learning, etc., organizations are dealing with a more mundane immediate concern of balancing the productivity gains of LLMs against potential information security breaches.

Productivity gains from LLMs are undisputable and range from automating mundane tasks (e.g., first- not final - drafts of run-of-the-mill emails, information sheets, internal communications, newsletters, FAQs, etc.). Far from leading to job losses, they enable refocusing the time of valuable employees on producing much more polished finished products. LLMs are also very useful in doing tasks such as converting computer code from one language to another and writing basic pieces of analytical code such as SQL, and developing simple mathematical models, freeing up the time of your highly skilled programmers and modelers to do more value-added – interesting - work – leading to greater productivity but also leading to a happier and more satisfied workforce.

But there is a very particular problem with the way LLMs in open platforms work. It is that every time a task is performed, the model is retrained, using data provided by the user and that which is publicly available. (e.g., if you request an LLM to generate FAQs as part of an internal communication on an organizational policy developed in response to a new piece of legislation, in order to get the best results, you would typically feed the LLM with the new policy and other internal documents related to it, the LLM would then use this along with publicly available information on the new legislation, including how other organizations have responded to it, to come up with an answer. The model is not only retrained when performing this task but it would now be available, publicly, for future use. The problem with this is that the newly trained model - in the public domain - has embedded in it the internal data used to train it. However, this is not as simple as it sounds. The data is usually not readily available for all to see, and though this possibility could not be excluded, it is more likely that it exists in a subtler way, embedded in the logic of the newly trained model. Nevertheless, it is true that the internal data is outside the organization's firewall. Which, obviously, is disconcerting to most organizations.

“Productivity gains from LLMs are undisputable.”

While there are no easy ways out of this, banning access to LLMs does not seem to be something most organizations are doing; instead, they are taking a ‘soft-stick’ approach, providing employees with guidelines on how to use them in a secure way. Even organizations that have blocked LLMs, such as CharGPT, say it is only a temporary measure while they evaluate the situation. This indicates a great deal of ambiguity in the industry. But such situations present the best opportunity for innovation. I am inclined to think that the solution might lie in developing appropriate software solutions. Say, if we could introduce a design whereby the internal data is partitioned into what the organization considers sensitive and what it is willing to share, and if every time the model is used (trained), the user would have the flexibility to request two trained models; one trained using only non-sensitive internal data and external data, and another trained using all data; the later to remain behind the organizational firewall and the former to be released publicly. This way, the LLM could keep getting better for the common benefit while protecting sensitive information. Developing the technology capable of doing this might be the next challenge for Research and Development in information security.

In the early days of the web, several organizations restricted access to it based on need and (sometimes) seniority. This was driven by a fear of both sensitive information leaking and untoward things like pornography seeping in. But hardly any organization restricts access to the web anymore, and this is mainly due to sophisticated web security systems. The solution to the current apprehensions regarding LLMs might also lie in the same thing.