Part II – Choosing your AI Agent’s training academy

Here at DMS Group we have recently embarked on a series of adventures with Agents, AI Agents. This is Part II of a series on that theme. We continue with a conundrum: what training academy (toolset) do we best use to train our Agents? Here goes…

So you want to build a custom AI Agent. Whatever the motivations behind that decision, your next concern will be deciding what tools to build it with. Your AI Agents are going to need training and you’re going to need to find the training academy (toolset) that enables that best. In this post we explore a common landscape: enterprise shops with a modest IT department and a heavy investment in Microsoft tooling. E.g. Entra ID, M365, SharePoint and splash of Azure. Sure, that’s not everyone’s IT real-estate but if its you, then you are not alone. What we will share in this and coming posts is a journey and approach that we’ve found to work for others.

Exploring Choices for the Agent Academy

Given the stated Microsoft footprint and early market feedback, we definitely encourage exploring Microsoft options. Their AI services are easy to switch on and integrate well with your landscape but which one to create and train your agents? A search for tools can list several options but broadly the choice distills into two camps: a code/developer heavy toolset like Azure AI Foundry or a low code option like Copilot Studio/Power Platform.

Diving into tools like Azure AI foundry is exciting. It certainly provides granular control. However before we get too carried away, we considered our clients situation. We asked if we could not do more with low code options before we leapt into the more complex but powerful toolsets like AI foundry. The latter may be great for our consulting budget but irresponsible when we assess client capabilities aligned with requirements.

We have found Copilot Studio’s Agent creation experience to be powerful, yet simple to use. A lot of our clients have repositories in SharePoint that they want to form the knowledge basis for their workflow support agents. Copilot Studio has excellent capabilities when it comes to training knowledge agents on client SharePoint data. That is great, but what if you need more control and granular integration into your line of business systems? Commodity connectors exist but they can only get you so far.

Can we have our AI Agent’s cake, and eat it?

We commonly find the need for agents to be able to interact with custom business systems e.g. web apps and SQL Server back ends that are core to a specific business. If we cannot solve for this then it is going to be a limiting journey with Copilot Studio. What if we could supplement the ease of Copilot Studio’s agent creation with more powerful capabilities?

Perhaps we could win the best of both worlds: an Agent development and management tool that is not as demanding on client knowledge but supplemented with custom connector super powers when needed. On the occasions the client needs it, those connectors capabilities can be written by their technology partners like DMS. This has a potential side benefit in that it allows a high degree of control to remain in the hands of the information workers closest to the business. This is the idea we grappled with at DMS. The good news is it does show potential. In our next post of this series ‘adventures with agents’ we’ll be sharing a ‘how-to’ on an approach that we’ve found to work. One that you may wish to adapt/reuse.