AI Risks at Work: What Ungoverned AI Use Exposes

Staff at a mid-size firm open AI tools for ordinary reasons: a deadline, a long file, a clumsy system at work. The trouble starts when that use runs without agreed rules, a named owner or a record of who uses what. AI risks at work are what the firm exposes through that use. Five points along the path of a single request carry that exposure: who types it, what goes in with it, where it lands, what comes back and who acts on the answer. The use case on governing and securing AI use across the firm covers the wider subject.

AI Risks at Work Follow the Request, Not the Tool

Before going further, three words need a meaning. First, a request is one question or task a person gives an AI tool, including whatever they paste in. Next, ungoverned AI use is use with no agreed rules, no named owner and no record of who uses what. Finally, an answer is what comes back, which the person may then act on.

Published lists of harms are long. The MIT AI Risk Initiative lists on its home page more than 1,700 AI risks, drawn from 65 frameworks. A list of that size may serve as a research map rather than one firm’s to-do list.

News coverage is uneven, too. A study of news media from 27 countries by Allaham, Kieslich and Diakopoulos reports a skewed prioritisation, with societal risks first among the categories it names. Signisys reads this as a sign that a popular list may reflect what gets reported. This article stays with one request in one firm, point by point.

Before the Request: Who Is Typing and Why

The first point is the person. For instance, picture an associate at a mid-size accountancy practice, invented for this article, short of time before a client’s year-end deadline. She opens a chat window on her personal laptop to summarise the client’s file. She signs in with a free account under her personal email address.

The exposure here is use the firm has no record of. The firm holds no list of who uses which tool, and it may not be able to reach that account later if she leaves.

Her reason is sensible: a deadline is close, and the tool at work feels clumsy. The exposure lies in the missing record, not in the associate.

First to find out: the line manager, if anyone asks. The question for the firm:

Which AI tools have your staff opened this month, and which of those accounts belong to the firm?

What the Request Carries Out of the Building

The second point is what goes in with the request. The associate pastes the client’s trial balance into the window. She also adds a few client names and a partner’s note on the file. Once she presses enter, the firm may have no means to take the text back or to say where it now sits.

Some content raises the stakes, such as the client’s unpublished figures, the names in the file and the partner’s private note. Rules on personal data are a job for the firm’s own adviser, and this article is not legal advice. The use case on keeping sensitive business data protected covers that side in more depth.

First to find out: the client, if the client later asks where the data went. At that moment, the firm may have no answer to give. Where a firm wants to see what it holds on record, a sheet that rests on dated evidence is set out in nine checks on the AI tools staff use today.

Which sensitive item could a colleague paste today, and does anyone in the firm know?

Where the Request Lands

The third point is the receiving side. The supplier’s terms may set how long it keeps the text, who on its side can read it and whether it uses the text to improve the service. Those terms may differ between suppliers. If no one at the firm chose the tool, no one there may have read them.

That gap reaches the client relationship. For example, suppose a client asks where their data sits. The answer may lie in terms that no one at the firm has read. A partner may also have signed a client confidentiality clause that the associate’s habit breaks without meaning to. The compliance side of staff signing up on their own is covered in unapproved AI use and the data protection and intellectual property rules it touches.

First to find out: the partner who signs the client contract.

For each tool in use, who at the firm has read the supplier’s terms on keeping and reusing text?

What Comes Back, and Whether Anyone Checks It

The fourth point is the answer. The associate’s summary could be wrong, invented or one-sided, and it reads fluently either way. In this case, the summary mentions a deduction that the client’s file does not contain. A rushed reader may not notice without a check.

Checking takes a person who knows the subject, with the source open beside the answer. A reader who does not know the subject may miss what is wrong, and a reader without the source open has nothing to compare against.

First to find out: the reviewing partner, if a partner reviews the work. If none does, the client may be the one.

Which pieces of AI-assisted work leave the firm without a second person reading the source?

The Advice Letter Built on the Answer

The fifth point is the decision. Here someone files, advises or sends on the strength of the answer. The responsibility rests with that person and the firm, not with the tool or its supplier.

In the practice, the summary shapes the advice letter, and the letter goes to the client. The mistaken deduction then belongs to the practice. The client may ask how the advice was reached, and the firm may not remember that AI was involved, as the associate wrote nothing down at the first point.

First to find out: the client.

If a client asked how a piece of advice was produced, could the firm answer from its own records?

AI Risks Along the Path of One Request

In Signisys’ working description, the path has five points. The table below is that description, not a standard. For a leader, AI risks at work become five questions for staff.

Each row names a person or a record, not a product. The answers in the last column come from staff and may differ between firms. Once those answers are in, the order of work that follows is set out in five steps from finding the AI in use to a dated review.

Starting From the Firm’s Own Answers

When a leader sits down with staff and asks the five questions in the table, the firm can learn about its own use what a long list of harms does not show. Some answers may be uncomfortable to hear. For a wider look, the use case on taking stock of AI at your firm starts from the same place. After that, Signisys can walk through the answers with you and show where a rule or a control would help.

Signisys provides enterprise AI solutions and data and AI services, and is not a neutral party here. See its enterprise AI solution and its data and AI service. The box below is a Signisys offer, and no outcome is promised.

AI Risks at Work FAQ: Points of Doubt