Within the Data Science community, there’s a joke that goes like this:
Q: What’s the difference between AI and Machine Learning (ML)?
A: If it’s written in Python, it’s ML. If it’s written in PowerPoint, it’s AI.
There are a lot of companies out there that claim to do artificial intelligence (AI). You’ll find the omnipresent initials appended to domain names or within the first few words of a pitch deck, but, frankly, a lot of it is BS. Usually, it’s a set of simple regressions and/or heuristics that require a human monitor, making it more of a semi-automation system. A true, robust AI system enables in-the-moment decision-making with the influx of new data — all without human oversight (*some initial assembly required). More often than not, many solutions are labeled “AI” simply because it excites executives and VCs.
Despite the noise that crowds the market, there are, in fact, companies that offer true AI solutions. Take one of the many major autonomous vehicle companies that are dealing with very tight control tolerances: the risk of inaccuracy could mean a major accident.
So, if the legitimate AI companies are mixed in with vaporware like puffed cheese snacks in party mix, how can you identify the good ones?
Vet the executive team
When PE or VC firms evaluate startups, it’s not entirely the idea or product that determines an investment decision. A major factor is the executive team — how well they work together, and how their backgrounds prepare them for success. When evaluating an AI/machine learning product, adopt the same process.
Consider the backgrounds and expertise of the key contributors to the product. A majority of their backgrounds should fall within a technical domain: Computer Science, Mathematics, Engineering, etc. Another plus: team members with experience in your vertical, which can shape how a solution is deployed and ensure it’s a good fit for your company and processes.
Ask some basic technical questions
More often than not, companies with vaporware have not done their technical homework. If the AI product is totally bogus, you can often sniff that out with some basic questions that true AI experts can answer. If needed, bring someone with a machine learning and statistics background to spot BS.
I once spoke to a company selling AI for predictive automobile maintenance. A colleague of mine asked how many hidden layers were in their neural network, to which the startup responded “11,000.”
This answer provoked 15 more minutes of awkward conversation and backtracking, because that figure is roughly 500x more hidden layers than the best pre-trained neural networks geared towards object recognition. This one simple question flagged many alarms for our team and needless to say, we ended our discussions.
You should ask questions like:
- What types of models/techniques do you employ?
- How do you perform model validation?
- What would you check for in our data to ensure proper models are used?
- Can your solution integrate with our current systems?
Consider the fit within your company
If you’re evaluating a product for your business, it’s not enough to ask someone with a technical background to weigh in on the validity of an AI claim. There should be a team with experience in project management, technical product deployment and/or change management to gauge an AI product’s ability to adapt to your existing systems, processes, and even cultural norms.
Other areas of technical/managerial expertise may be needed depending on the problem you are hoping to solve with a particular AI product. For instance, if you’re a manufacturing company looking to integrate AI into your quality control processes, you should also involve Manufacturing and Quality engineers to examine and validate the predicted responses.
The next time you see “AI” on anything (which is inevitable these days), be skeptical by default. There are companies who are truly doing amazing work in this field, but do your homework. The above isn’t a complete checklist, but a good starting point to help you avoid buying into snake oil and creating a big mess at your company.
