Without a doubt, AI will be a booster for companies using Customer Experience (CX) to grow their business. CX, with its masses of data and clearly defined tasks, is the perfect use case. And the willingness to invest in AI seems to be high at many places. Sometimes, even following a temptation to do “something with AI” to be part of the movement.
In such a situation, there are two main risks.
- First, the temptation that implementation follows technology availability rather than being driven by business outcomes and strategic business priorities.
- Second, numerous providers offer AI for almost everything, making it hard to separate the wheat from the chaff.
So, there are a couple of key initial questions you might want to consider during vendor pitches … or even to ask yourself beforehand. Let’s start right ahead …
How does the AI model support the intended business outcomes?
This question requires you to have a clear understanding of what is to be achieved (business outcomes), and there is less risk of being an ideal pitch for something that sounds exciting.
This will also help the vendor to understand your needs and customize their offer (if they are customer-centric), enabling a more valuable discussion. Where the “how” forces the vendor to show up with a well-thought-through proposal instead of just telling their standard story.
Having this discussion will also help you understand the exact purpose of the model offered and find the most suitable, customized AI model for your specific use case.
How does the model support the company’s strategic competitive positioning?
Sounds similar to the previous … but actually adds the strategic perspective. Getting to a clear understanding will support the overall investment rationale and ensure that an AI model/solution is selected that really supports strategic business growth.
Is the vendor willing to comply with our corporate data privacy, data security, CSV, and compliance guidelines?
In case the answer is not clearly yes, you might want to consider ending the conversation immediately. It doesn’t really make sense to invest more time in a potential collaboration that will obviously not work. At least for my industry – the pharmaceutical industry – it is critical that everybody involved meets data privacy, data security, GxP, computerized systems validation (CSV, Annex 11), pharmacovigilance requirements, your corporate AI governance framework, and more. This includes vendors, as regulatory authorities do not accept a delegation of the responsibility.
In case the answer is clearly yes, a valuable follow-up question would be to ask for examples of cases where the vendor has already solved this challenge or passed audits with other industry clients.
Is our task well-defined enough to be formulated for your AI?
You might realize that I generally prefer asking open questions over closed (yes/no) questions. It opens the vendor the opportunity to explain a bit more rather than making a blanket promise.
As it is a basic success factor for every AI use case that the task to be executed is well-defined and easy to formulate for the machine, I wouldn’t want to miss asking the question early on. It also puts some more ‘flesh to the bone’.
How does the machine learning component of the solution leverage the data?
Which includes the question ‘what data’ and leads to a discussion on data availability, data access, data requirements, data biases, RAG availability, etc.. This also includes the question about how the model has been trained (so far) and how it can continue to learn in the future. With the latter being vital for many AI models.
You will find out whether the machine has been trained with data relevant to the intended use case at all, whether there is (internally or externally) a sufficient quantity and quality of suitable data accessible at all, and what exactly you can expect the machine to be able to do … and what not.
Which success metrics (KPIs) and assessment criteria do you propose?
You do not stick to what you will hear, but I would be curious to know how the vendor itself assesses the performance and quality provided by its AI model. Asking the question is also a clear signal that measuring success against clear criteria matters to you. And it can be a starting point for defining success and impact metrics together early on.
Can we in-license the model?
Especially in sensitive areas, it may be preferable to run the AI model internally in a sandbox rather than in an external vendor’s cloud. This certainly increases the complexity, as well as the time, costs, and maintenance efforts, associated with an implementation. But in case data security is a critical factor, this would be an option worth exploring.
Bonus tip: Ask the vendor to refrain from wasting time with providing a detailed background on their company …
…, but to reserve at least 50% ot the meeting time for our questions and – for the other half – better invest in inspiring us how they can help to achieve our intended business outcome.
I generally ask this. The pitch should be about you and your company’s needs, and not about the vendor and how great they think they are. In case you are a vendor reading this, sorry guys, I hope it is OK to be honest and transparent. I have already wasted too much time in pitch meetings where vendors mostly talked about themselves instead of listening to the customer. And I think it is understandable that after such a bad early customer experience, I personally prefer not to invest any more of my precious lifetime in something that has already wasted my time.
There are certainly many more detailed questions I would ask to ensure a rock-solid approach. But this list of 7 initial key questions will ensure that you are speaking to the right people and may even help you gain more clarity and maturity for your initiative.
Feel free to leave a comment if you disagree with an aspect or have an additional valuable question.