Fine-tuning to deliver business AI value

A few months ago, Microsoft introduced Copilot Tuning, offering its customers a way to use low-code tooling in Microsoft Copilot Studio to take advantage of highly automated fine-tuning “recipes” trained on enterprise data.

While generative artificial intelligence (GenAI) tools tend to be associated with AI models that are trained on vast swathes of public information on internet and social media platforms, businesses need models that understand their internal data and processes, and now the major AI providers have planted the power of GenAI into the mindset of business executives and IT chiefs, this is the current focus of commercial AI.

Such products aim to provide an AI system that is industry sector-specific compared with the highly generalised models that have been trained on freely available internet data. In theory, they should suffer less from hallucinations that afflict the more general AI models and more closely match the way a business works.

Ranveer Chandra, vice-president and group product manager of experiences and devices at Microsoft, posted in a blog: “AI tools powered by out-of-the box LLMs [large language models] and retrieval augmented generation may not always understand your business in terms of specific processes, terminology and style.” He claimed Microsoft’s approach to optimising AI models for business has been to reduce the complexity often associated with fine-tuning projects.

One of the customers in the Microsoft 365 Copilot (M365) Tuning early access programme is accountancy firm Ernst & Young. Marna Ricker, global vice-chair for tax at the firm, said the company was integrating a tax-domain fine-tuned LLM with its enterprise knowledge and the expertise of its tax advisors through M365 to deliver an enhanced tax service to the market. “This synergy improves service quality, and significantly advances tax and legal research with relevant knowledge and intelligence readily available in M365 where people are already working,” she added.

According to a forecast from Gartner, the market for specialised GenAI models will more than double to $2.5bn by 2026. While this is significantly smaller than the $23bn forecast by Gartner for general GenAI models, it shows there is demand in businesses for such technology.

Roberta Cozza, senior director analyst at Gartner, said the major AI providers are fine-tuning their models as this is where enterprises are moving. Enterprise buyers, she said, value working with a trusted technology provider, but they also need GenAI-based tools to respond to something that is specific to their domain. “What we are seeing actually is domain-specific models,” she said.

Cozza noted that many of these actually start from open source models as a base, and are often deployed as small language models (SLM), which offer efficiencies in terms of resourcing costs, but also provide better control since they can be trained on an enterprise’s own data. 

GenAI can deliver value, but the enterprise IT leaders she has spoken to say they want it to be trained with the issues, data and content of the specific industry they operate in. While the likes of Microsoft and the major IT consulting providers are ramping up their AI business offerings to cater for enterprises that are now looking to deliver business value with GenAI, IT leaders should consider alternative approaches. “They need to put their proprietary data in the hands of either a model builder or an IT service provider,” said Cozza.

“Barriers to entry using basic open source models have reduced a lot, so we’ve seen a lot of smaller AI providers helping large customers with their own small model,” she added. “They can distill either a proprietary model like ChatGPT, but many are starting with Meta’s Llama, and in Europe, we are seeing Mistral as a starting point.” 

While 90% of GenAI models are managed by a few major providers, Cozza said Gartner has been fielding inquiries from IT decision-makers who specifically need to deploy European AI technology as a safeguard that buffers the volatile geopolitical environment they need to operate in. They are also considering how to remain compliant under the EU AI Act.

“Those AI applications and technologies that are deemed to be high-risk are the ones that will need to be regulated and comply with the EU AI Act,” she said. “But this covers frontier models that are trained on internet data.”

Cozza said models built on a company’s internal data and SLMs, which are explainable, are less likely to require regulatory scrutiny. “Training an AI and creating something that is more domain-specific actually improves general compliance because you can make it comply with policies or regulations,” she added.

Tools like M365 Copilot Tuning will inevitably help to lower the barrier to entry for IT leaders who have been tasked with providing GenAI capabilities that can add business value, but SLMs offer an alternative approach that can deliver explainability and comply more easily with the EU AI Act.