The 5 Questions#
Should I use AI/ML?#
Developing an AI application offers challenges in several areas not present in typical software engineering projects, for example the areas of data collection, management, testing, and validation. While there are areas in which the technology is beneficial it is important to evaluate the repsective costs and benefits of AI and Machine Learning for the proposed task. To address this, the first step in this guide is to provide a high level feasibility check of the key question: should I use AI for my project? We break this question down into three tasks:
What is the engineering case for using AI to solve the problem, over other approaches?
Does the problem involve any application or data areas that may prohibit development?
What is the risk that the project falls into the unacceptable AI Risk category?
What is the risk that the project makes use of problematic data?
Requirement |
Evidence |
Complete |
|---|---|---|
How do I define my AI project?#
In the previous step we developed an engineering case for using AI in our project. In this step, we start to realize our project by setting the goals and bounds of the project. There are three parts:
Defining goals and metrics for success
Defining limitations and boundaries on the project
Completing a risk assessment of the project as a whole
Requirement |
Evidence |
Complete |
|---|---|---|
How do I collect data for my AI/ML project?#
In the previous step, we defined the scope of our project. In this step, we move on to the first practical engineering task: collecting the data. AI/ML applications are fundamentally, data driven, so gathering or processing data is essential. It’s very important to get this right, as the quality of the data will directly impact the success of training and deploying the AI application. We set out a number of steps for this section, but our primary challenges are:
Making sure the data we collect is useful, truthful, and effective.
Transforming our raw data into a form that can effectively utilized by AI/ML algorithms
Setting up a robust infrastructure for data collection, storage, and access.
How do I train my AI/ML application?#
In the previous step, we collected the data for our AI project. In this step, we will use it to train an AI algorithm of our choice to meet the goals of our project. This is a critical step at which issues from earlier decisions may start to surface, so we recommend reviewing earlier decisions if problems arise. This step addresses 3 primary challenges:
Deciding which AI approach to use
Engineering a pipeline that optimizes the training
Ensuring that the training results in an AI algorithm that performs as expected and avoids unintended behavior.
How do I deploy my AI application?#
After training our AI application, we can finally deploy it and (hopefully) achieve the goals set out in our previous steps. For many professional engineers his step will feel familiar, as the process for deploying an AI application is similar to that of deploying any other software application. Our process has three steps:
Preparing our trained the model for a live environment
Engineering a process for deployment and model updating
Setting up continuous monitoring to ensure the model performs as expected over time
Requirement |
Evidence |
Complete |
|---|---|---|