Can AI really be your virtual CTO?

The UK Government's advice on AI adoption

The government's SME Digital Adoption Taskforce published its final report last year, looking at how small and medium-sized businesses can be encouraged to make better use of digital technology and AI.

One of its recommendations caught my attention. It proposes developing an AI-powered “CTO as a service”, which would provide SMEs with basic diagnostic tools, guidance and pointers to help them choose and adopt digital products.

I can certainly see the attraction. Most SMEs don't have a Chief Technology Officer, or a Chief Information Officer, in fact many don't have anyone with specialist IT knowledge at all.

But I'm not convinced that AI is the answer, simply because AI is only as useful as the conversation you have with it.

I hold my hands up and admit that I use AI myself, and I think it can be an extremely useful business tool. But there's an important difference between using AI to help you do something and asking AI to advise you about something you don't understand.

To get useful results from AI, you need to know what questions to ask.

Then, when it gives you an answer, you need enough understanding to know whether that answer makes sense. You need to recognise what's missing, challenge assumptions and know when you need to ask another question.

That's not particularly difficult when you're using AI within an area you already understand. It's much more difficult when you're asking it to provide the expertise you don't have.

Interestingly, the Taskforce's own report identifies lack of expertise and execution support as one of the main barriers to digital adoption, and at the same time it says that many SMEs overestimate their digital readiness.

So if a business doesn't have the expertise to make a technology decision in the first place, how does it know whether its virtual CTO is giving it good advice?

Writing the document isn't the same as understanding the process

Take something relatively straightforward like preparing a specification to send to potential IT suppliers.

AI could be very useful. Give it the right information and it can help organise your requirements, suggest questions, identify areas you might want to consider and turn a pile of notes into a professional-looking document.

But somebody still needs to understand the procurement process.

Before you can ask AI to write the specification, you need to establish what the organisation is trying to achieve, who will use the system, what constraints there are, what support will be needed and which requirements genuinely matter.

You also need to recognise when something important is missing from the document AI produces.

The finished specification your AI gives you may look excellent, but that doesn't necessarily mean it's asking suppliers to solve the right problem.

Sometimes the question you start with isn't the real problem

This is probably the biggest difficulty I have with the idea of AI acting as a virtual CTO.

When organisations come to me for advice, the question they initially ask isn't always the question we eventually need to answer.

For example, a client might ask whether its IT supplier is charging too much. Before I can answer that, I need to know what the organisation needs, what the supplier actually provides, what the agreement says, what alternatives exist and what changing supplier would involve.

Or another organisation might ask why it has to pay its IT company £100 to fix an email problem when it already has a support contract. Investigation might show that the company isn't actually contracted to provide the service the organisation thought it was paying for.

Or an organisation might think it needs a new website when the real problem is that its services and strategy have changed so all it needs is a reorganisation and some new content.

In each case, answering the original question without investigating the situation could produce perfectly sensible advice about the wrong problem.

Where I think AI can help

None of this means I think SMEs shouldn't use AI for technology decisions, because I don't.

Used well, it can help you understand terminology, explore possibilities, organise information, prepare questions for suppliers, structure documents and make complicated information easier to digest.

The Taskforce itself found that SMEs want guidance and impartial advice about technologies they could adopt. AI could certainly make some of that advice much more accessible.

But I would be wary of treating it as a substitute for expertise.

A CTO doesn't simply answer technology questions. They understand the organisation behind those questions. They know its priorities, people, systems, suppliers, constraints and risks. And sometimes their most valuable contribution is recognising that the question everyone is trying to answer isn't actually the question that needs answering.

AI can be a very useful tool in that process.

But I'm not ready to give it the CTO's chair just yet.