Anthropic CEO Dario Amodei recently published an open letter calling for a reduction of the pace of AI development by means of new guardrails and oversight. OpenAI CEO Sam Altman, Tesla and SpaceX CEO Elon Musk and Google DeepMind’s Demis Hassabis came out in agreement and asked that antitrust rules be relaxed to enable all AI companies to co-ordinate on slowing down, or “pacing the frontier”, as it’s described.
The Amodei open letter followed the rather public resignation of Jacob Coxon from Anthropic (he previously worked at OpenAI) because he believes leading AI labs are recklessly rushing toward self-improving superintelligence and risking human survival.
Coxon states that top labs are “gambling with our lives” by prioritising competitive pressure over safety guardrails. He warns that AI could cause human extinction before the end of the decade. Anthropic alignment science lead Evan Hubinger claims there is a greater than 10% chance that advanced AI could kill all humans within the next decade.
All this raises the question: What’s the appropriate way to regulate AI, and specifically the large language models (LLMs)?
There are fears, prompted by the heads of the largest US AI companies, that AI will recursively self-improve and develop some kind of undefined superintelligence.
There is much discussion about trying to develop global AI rules to get the models to align better (with whom?) and about implementing safety controls or guardrails. However, there is no common understanding of what “responsible AI” is, and reaching a global agreement seems a long way off.
Away from the hype, a project is being run from Columbia University in New York that instead proposes to treat AI as a normal technology. On this understanding, AI is not something otherworldly, it is a new technology and subject to regular product liability laws — both civil and criminal.
If you put a defective product onto the market and it causes harm, you are liable. If you perform an action using a tool, whether defective or not, and that action causes harm, you are liable. This should not be controversial. OpenAI has already developed models that hacked into other companies on several occasions. This would be a crime under an existing US statute, the Computer Fraud & Abuse Act, in place since 1986.
That being said, what is unique to AI is its speed, scale, opacity, autonomy and, critically, the evidentiary burden involved in applying existing laws.
In South Africa we have an extensive governance and regulatory framework covering companies, financial institutions, government departments and the processing of personal information. This is underpinned by the constitution, the Protection of Personal Information Act, the Promotion of Administrative Justice Act, the Companies Act and financial sector legislation. It’s also supported by regulators like the Reserve Bank, the Prudential Authority, the Financial Sector Conduct Authority, the Information Regulator, the Financial Intelligence Centre and the National Credit Regulator, along with the common law. The King V Code specifically recognises the governance implications of emerging technologies, including AI.
Who’s the boss?
AI introduces some complications when machines start making recommendations, influencing important decisions and increasingly acting on their own. The issue of accountability comes to mind. AI cannot of itself be accountable. In a traditional business process, it is generally possible to identify the person responsible for a decision. An AI model can produce a recommendation, another system can interpret it, an employee can approve it, an AI agent can initiate an action and a downstream system can execute it. When something goes wrong, saying that “AI did it” cannot be the answer to the question of who is to blame. The model developer may have built the technology, but the organisation chose to deploy it. The software provider may operate the platform, but the institution remains responsible for how it is used. Accountability cannot be handed to an autocomplete machine.
AI also exposes an important distinction between technical capability and organisational authority. A board may delegate authority to a CEO, who delegates financial authority to executives and managers, subject to defined limits. These rules are designed in relation to human actors. Now consider an AI procurement agent: the system may technically be capable of finding suppliers, negotiating terms, creating purchase orders, changing supplier information and communicating directly with an enterprise resource planning system. That does not mean it has the authority to do so. AI makes that authority more difficult to control because it may have to be checked and enforced automatically, in milliseconds, before the action occurs.
Many AI governance frameworks require a “human in the loop” for important decisions. However, simply inserting a person into a workflow does not necessarily create meaningful oversight. If an employee receives an AI recommendation and clicks “approve” two seconds later, a human may technically have participated. But did that person understand the recommendation, and did they see the relevant information? Was their approval genuinely independent or were they simply accepting what the system told them? AI does not create an oversight requirement — that already exists. Organisations still need to prove that oversight was real rather than perfunctory.
AI has become a powerful tool. It scales capability but also removes the evidence needed to prosecute (or defend) the actions it enables.
Before AI, a transaction could often be forensically reconstructed from applications, e-mails, workflow records, approvals and database entries. An AI-enabled action may involve a model version, data sources, prompts, retrieved information, external services, risk scores, software agents, human interventions and changing system configurations. It is impossible to explain how AI models produced an answer. They work probabilistically, not consequentially (or deterministically). An LLM will produce a different answer to the same prompt depending on the state of the LLM when prompted. It is not possible to recreate the exact system state in an AI model from six months ago.
And that is what is lacking in AI: traceability. Traceability makes it possible to attribute liability, but that depends on preserving the right evidence so that existing laws can still be applied. This becomes more important as AI changes from systems that recommend actions to systems capable of taking them.
Vanishing evidence
Traditionally, after something goes wrong, legal liability needs to be established. It requires organisations to rebuild events from logs, databases, policies and interviews. AI makes this increasingly difficult to do and therefore to respond to a regulator, customer or board — or without clear evidence of what occurred.
So, what to do? An obvious solution is to implement measures so that important actions create appropriate evidence when those actions occur. It requires preserving enough information about the authority, policy, system context, controls, material evidence and execution outcome to establish what governed the action. Consequential digital execution should be accompanied by independently verifiable evidence of what occurred.
Regulating AI is very hard, as there is little consensus about how best to do it, in South Africa and elsewhere. It can even be counterproductive. It would be better to apply existing laws and regulations while ensuring that we have the evidence to reconstruct events and fix liability where liability must be determined.
The existence of preserved evidence of what occurred will be a powerful motivator for responsible use of these new and powerful tools. After all, nobody, and especially not a company director, wants to face paying fines or damage claims or do jail time.
De Vos is a director of corporate finance and advisory firm QED Solutions
