Artificial Intelligence (AI) is booming. Today, 80% of enterprises invest in some form of AI. AI-enabled tools will generate $2.9 trillion in business value in 2021. But developing AI-based software is fraught with several challenges.
Developers of AI-based software have to create algorithms and train them with data sets. The algorithm learns on its own, getting better with every experience. Machine learning models, powered by such algorithms, deduce patterns and features from the input data and generate outputs. This tech blog lists the top strategies for building such AI-based software.
1. Co-opt Data in the Planning Process
The product management team of conventional software focuses on the features and capabilities of the product. Developers of AI-based projects regard data in the same light.
An Artificial Intelligence system is only as good as the data used to train it. The output of AI-based software depends not on the rule coded by the programmers, but on the training data fed to the algorithm. Feeding the algorithm with junk or poor data leads to poor decisions.
In AI-based software, rewriting code does not change the output. Developers rather retrain algorithms with new data. Thus identifying and curating the right data is the hardest task in developing AI-based software.
Co-opting data considerations to the software planning process entails:
- Defining the data required for the project
- Identifying the data required to train the model
- Laying down the acceptance criteria for the identified data
- Deciding how to gather and store the data
- Collecting, labelling, and cleaning the identified data
- Applying data to models, to train algorithms
- Evaluating how the AI models use the fed data and fine-tuning the data as required.
2. Decouple the AI Model From the Product
Speed up the development process by decoupling the AI model from the rest of the software development process.
Developing AI models takes much longer than developing conventional software. Gathering data samples, cleaning the data and tagging it takes days. The training process takes even longer. The number of days required multiplies for each training cycle. Even after such delays, the Artificial Intelligence model may remain unpredictable.
Build the AI model as a separate pipeline, and develop the main product independent of the AI model. Tying the entire application to the AI model limits the progress to the speed of the model’s development, and holds up time-to-market.
Decoupling allows releasing the primary product and the AI-model at different modulations. The developer can gather feedback and change product feedback, without waiting for changes on the AI model. Side-by-side, they may update the AI model without considering unrelated issues, such as a code change breaking the build. For in-premises software, users need to make only a one-time effort to install the software. They may apply updates to the AI model without an extensive upgrade process for every iteration.
3. Leverage Existing AI Models
Reinventing the Artificial Intelligence wheel is foolhardy. Rather, leverage existing resources and build on it.
Artificial Intelligence is already adept at solving several problems, especially speech and facial recognition. A lot of painstaking and time-consuming work has already gone into building and refining such AI models. Many cloud providers offer AI products for such tasks on a “pay as you use” model. Common examples include Amazon’s “Rekognition,” Microsoft’s Face API and Google’s Vision API. Developers of AI-based software may subscribe to such works, and hit the ground running to develop their custom suites.
Often, a combination of existing AI and non-AI services, with some custom development offers a prototype to solve a business need.
4. Integrate the AI Model with Applications
Good software solves real-world problems. Developers have to integrate the AI model into the product, to solve the underlying problem. This may not always be straightforward.
Load the AI models into containers. Several recent standards allow developers to load their AI models into applications.
Focus on the UI. The user interface is as important for AI-based software as it is for conventional software.
An AI model is an iterative process, which gets better with every transaction. Developers have to take a call on how often they will retrain the AI. The decision depends on the software and the maturity of the model. But when they retrain, they have to make sure the new generation is better than the old one, through extensive testing.
5. Build Cross-functional and Multi-disciplinary Teams
AI-based software is susceptible to unexpected deviance. In conventional software, debugging errors or unexpected results only requires a hard look at code. But with an AI model that develops its logic, developers cannot determine the logic leading to a decision.
Influencing the decision of AI-model decisions is not through debugging but through supervised trial and error. AI experts who understand how the model works may guide the learning and tuning process. But effective testing has to take place on the system as a whole, with all the components deployed and working. This requires expertise in both AI and software testing.
Cross-functional teams, involving data scientists, analysts, testers, data architects, and product managers, work best for the purpose. When data scientists who design AI models work side-by-side with coding and testing specialists, synergy sets in. AI algorithms learn from each transaction. As such, the fine-tuning will extend post-implementation and will involve the customer or end-user as well.
The AI revolution is on. Software vendors roll out new AI-powered solutions and add AI capabilities to existing products. Restructuring the software development process, to co-opt AI is the way to go. The above strategies and best practices allow developers to get a head-start in an otherwise challenging process.
