3.1 How do I know when my problem can be solved with AI? - Video Tutorials & Practice Problems

<v ->So how do you know</v> when you have a problem that AI, and especially machine learning is gonna allow a solution for? How do you know that it's the right type of problem to use for machine learning? Well, let's take an example. So this is an example that a real client brought to me at one point where they had a committee that was deciding whether webpage changes actually met all of the standards that they had. And so did they meet the branding standards? Did it pass legal muster? Did it use the logo correctly? So all sorts of rules that they had, and they had written them down, and they had experts who were on a committee, and what they would do every time a new web page was being proposed or a change to a webpage was being proposed, the committee would inspect it, take a look, judge it and say, "You know you have to fix these things, go back to start and do it again. Or yes, it's okay. You can put that up on the website." And so this was kind of a very time consuming process for them. And you can imagine that the kinds of experts that they had, like the lawyers, the brand experts, these were actually fairly highly paid people. So it was fairly expensive to convene this committee for all this piddly stuff. And the problem was that it was actually the most boring part of their jobs. None of the people in the committee really liked doing this. It was really boring and dull, and they were running into the same problems all the time over and over again, and they said, "You know not only is it boring for people, not only is it something where these are highly paid people. So it's a costly process for us, but it's also slow." You couldn't have the committee look at a page every time was it ready, they would meet like once a week. So on Fridays, they would spend a couple of hours going through all the changes that came in. That means you couldn't change a page in less than a few days. You had to wait for the committee to meet in order to be able to put it in production. And maybe we'd like to move faster than that. So we're all sorts of reasons why they thought, "Hey, we'd like to have an automated process, but nobody could figure out how to automate that process with traditional methods. It was too complicated. It was too subjective in a lot of cases. And so what could they do to use AI? How could they have automation that actually employed human-like judgment, which is really what AI is supposed to do? Well, one of the problems was that they weren't actually collecting a lot of data. You know how AI loves data? Well, what they really needed was data about which pages had passed, and which pages failed and why? And they hadn't been collecting any of that data. So this problem really wasn't ready to be solved with AI, but then what they started doing is collecting the data. So instead of the lawyer just saying to the person who did the webpage, "Hey, you know you're not allowed to make that claim because that actually goes against the pharmaceutical law about making claims that haven't been validated with the FDA." And the person says, "Okay, I'll change it." Instead of that being verbal, instead, what they needed to do is to create a structure where they ticked off here are all the different standards that we have to comply with, and here is the exact phrase on the page that did not comply with this exact standard. And then after they spent a long time collecting all of that data about highly specific things that did not pass, then they got to the point where they could start to use AI to be able to solve this problem. But you have to really ask yourself for the problem that you're trying to solve on your project. Do you have the data? And if you don't have the data, where are you going to get the data? How can you put a process in for you to collect the data the way my client did here? 'Cause this is the first step to using AI to solve the problem, to actually have the data that the AI needs in order to put together the model that solves the problem. So if you don't have the data, your very first step is to create a process that collects that data, and collects it in a very structured form so that the AI will be able to then use that to solve the problem later. So how do we identify good AI opportunities? Well first, it should be an important problem. AI is going to be somewhat expensive to do no matter how you do it, it's gonna be complicated. It's gonna require experts. I mean think about how much money data scientists make, right? You don't wanna use them on a piddly little problem. You also wanna have a well-defined process. So in the previous example, they really didn't have a well-defined process. The committee got together, they looked at stuff. They told people that page passes. Here's why, here's the thing you have to do, but it wasn't really defined very well. It was more based on the expertise of the people then that there was an actual process that said, "Here's an example of something that passes the standard. Here's an example of something that doesn't." And as we saw in that previous example, it has to have outcome data. So if you're gonna do supervised machine learning, where you need labeled data, you have to have that outcome data in order for it to work. And so what is outcome data? How do we know when we have it? So it's really the data that you're gonna use to train the machine learning model. So training data means you know the input, but you also know the output. So what it means for example, is that you know what the condition is, but you also know what the label is. You know whether for example, in the example that we used, whether that webpage passed or failed, was it approved or was it not approved? And if it was not approved, what was the exact thing that caused it to fail? And so the input is here's the webpage, and here's all the words on the webpage. Here are the pictures on the webpage, but the output is exactly which standard did it fail on, and what exactly was the part of the webpage that failed? That is outcome data. Now there's two kinds of outcome data, there's human agreement. So it's basically an opinion where most people agree that this is really what the correct outcome is. The other kind of outcome data uses objective metrics. So let's look at some examples for each of those. So human agreement, we can go back to the example we used before of social sentiment. So if we wanna know which subjects are being discussed that are positive or negative, the input is the text from the social media conversations about your brand, but the outcome is actually a human opinion. Is it positive? Is it negative? Is it neutral? And it may have some kind of confidence associated with it. And so the approach to use here is human agreements. So how do you get human agreement? Well, you might have multiple people each look at the same tweet, and see if they agree with each other that this is positive, or if they agree with each other that it's negative or neutral. And so you might have to have two or three or even more people look at that tweet, and agree whether it's positive, negative, or neutral in order for you to get the outcome data you need, because you don't want to have just one person look at it because they might be wrong. And so it's expensive to have multiple people look at it, but you have to have that in order to make that human agreement data correct. If you train your machine learning model with data that's incorrect, when you guess what you're gonna have happen, you're going to have a model that's giving the wrong answers. So if you just use one person's opinion for human agreement, now you're gonna have a model that reflects that one person's opinion, but that one person may not be right. And so that's the thing for you to really understand is that we have this belief that human opinion is always correct, but it's actually not always correct. Humans actually make mistakes when they're actually deciding for things like social sentiment, and some problems are even harder for humans to get correct. And so it's even more important if you'd have multiple people agreeing with each other, that this is actually the right answer before you feed that to the model. You also might have objective metrics. They don't require human agreement. They're actually metrics from data you've already collected that are objective. They're not subjective the way human agreement must be. And so the contact management example that we gave previously of which contact should the salesperson follow up with? Well, there are objective outcomes there. Which of the recommendations for people to follow up on, have salespeople in the past, actually followed through and followed up with? Which are the ones that those people that they followed up with actually responded to them. And when did they complete sales, and what was the order size? So you are actually trying to get the recommendations for who to follow up with to be ones that sales people will actually do. The people follow it up with will actually respond to, and they'll actually go ahead, and complete a large sale, right? Those are the best things to be recommending for contacts to follow up with. And so those are all objective outcomes. You don't need human agreement. You've got the data that shows you what the objective outcomes are for those recommendations that the system is making, and that's another way of having outcome data. So whether it's with human beings that are agreeing with each other, because you don't want just one person's opinion, you want multiple or its objective metrics from data you can already collect, both of those are perfectly good as outcome data for you to be able to train your machine learning model with. Now let's go back to our original example, which one is this one? Well, it's human agreement. And what you'd want to do is you'd wanna have multiple experts in each area looking at these particular webpages, not just one expert, but multiple of them in each area. Multiple lawyers, you'd wanna have multiple brand experts, multiple people looking at this so that it's not just one person's opinion, it's several people's opinion that have that same expertise. And so it's not enough to just have one person collecting the data. So now this becomes a very daunting prospect. You're gonna have to convene the committee not with one lawyer, but maybe with three lawyers, maybe with three brand experts. And this is how you're gonna collect the data. Turns out that this was such a daunting prospect for my client, that they decided that they weren't going to do this. They weren't going gonna to do this. They just kept using the human beings meeting in the committee 'cause they didn't wanna have to expand the committee, and make it even more difficult for them to process these decisions. And so you might find that some problems are not really the right ones for using AI and machine learning because it's too difficult for you to collect the data, and that's actually what happened here. So use these lessons that we just taught you here to try and determine whether the problem that you're picking for your project is actually one that makes sense to be used for AI in machine learning or whether you might wanna pick a different problem.