Personalized AI Tutors? Sal Khan on Transforming Education

GOLODRYGA, HOST: Well, as we’ve been discussing, from deepfake images to voice scams, the danger of artificial intelligence has prompted global calls for regulation. But our next guest also sees an upside, and says A.I. is potentially the biggest positive transformation that education has ever seen. Sal Khan is the founder and CEO of the nonprofit Khan Academy. An online tutoring service that recently piloted a tutor and teaching assistant powered by A.I. He tells Walter Isaacson how he thinks A.I. can supercharged world-class education.

(BEGIN VIDEO CLIP)

WALTER ISAACSON, HOST: Sal Khan, welcome back to the show.

SAL KHAN, FOUNDER AND CEO, KHAN ACADEMY: Thanks for having me.

ISAACSON: We hear a whole lot about the problems and the peril of artificial intelligence, but one of the amazing things that it seems to be doing, and you’re leading the way on this, is creating a personal tutor so that every kid, every kid on the planet, could have a personal tutor that can tutor them in math, in writing, in history and everything else. Explain to me how you’re trying to do that.

KHAN: Yes. Educators have known for millennia that one-on-one tutoring, working with the students at their own time and pace is the best way to learn. That’s what Alexander the Great had with Aristotle. You fast forward to about two or 300 years ago, we had a very utopian idea, mass public education, but we had to compromise. We couldn’t give every student a personal tutor. We didn’t have the resources. So, we batch students together in groups of 30. We’d have some lecture at the front of the classroom, and that’s what we’ve been doing and it’s done a lot of really good things.

But over the last many decades, there’s been tons of efficacy research that it’s great to have 30 kids in a classroom, but it would be even better if you could have one-on-one tutoring. That if you do that, you could take the average student and make them an exceptional student. You could take a below average student and make them an above average student. And a lot of folks in technology over the last several decades have thought about, how could we use technology to emulate what a one-on-one tutor would do? Arguably, that’s what all of us at Khan Academy have been doing for the last 15 years or so, we’re not-for-profit mission, free world class education for anyone anywhere. But when OpenAI reached out to us last summer, and we were under an NDA until only a few weeks ago, and they showed us the technology and they said, look, we want to do some positive used cases with it. We immediately said look, we think this is ready to actually hit that holy grail of education, which is, can we create an artificial intelligence tutor for every child? And we launched, as part of the GPT-4 launch in March, and what we started piloting is something we call Khanmigo, it’s our artificially intelligent tutor that is power by GPT-4.

And what it does, there’s a lot of new news about using ChatGPT to cheat, this does not allow you to cheat. If you ask it a question, it’ll say, hey, I’m here to be your tutor, how would you approach it? It acts like Aristotle or Socrates would with their students. And so — and it works across every subject that Khan Academy does. It has all the context that the student would normally have on Khan Academy and it also acts as a teaching assistant for teachers.

ISAACSON: Well, give me an example of it would do, let’s take history. Supposed it’s an American history course you have I have done some together for Khan Academy. How would it help us figure out how the constitution was written?

KHAN: We did some user testing with the students at — actually, with many students. We have a lab school, Khan Lab High School. We also have another online school, Khan World School, and we did some user testing with these students, and one of the students was looking at a part of AP U.S. government on judicial review — or judicial confirmation, Senate confirmation.

And so, she just asked the A.I. — she watched a video of it on Khan Academy and then, she asked the A.I., you know, why is this relevant to right now? This was a question that she generally had as a high school student. And it immediately brought it up, some of the recent confirmation hearings, and it brought up the whole Merrick Garland situation and all of that. And the student immediately said, wow, this just brought it to life in ways that I could never have imagined. It also allows students to do things that would have looked like science fiction, even a year ago, where they can actually talk to simulations of historical characters. So, you can actually debate federalism with Madison or Ambleton (ph).

ISAACSON: You know, school boards around this country are being rattled by all sorts of controversies about what’s getting taught, about inclusivity or diversity or history or reparations or sexuality or gender issues. How do you think a tutor, an A.I. tutor like this, would deal with such controversial issues?

KHAN: You know, I might be naive, but I actually feel that 90, 95 percent of Americans are actually reasonably consistent on a lot of these issues, and a lot of the polarization happens based on hearsay. Based on, I heard that that’s happening in a classroom or I heard that that’s happening in the — and it gets folks triggered, it gets folks angry.

One of our strategies has always been to just be hyper transparent. If someone tells you that there’s a Khan Academy video that’s biased in some way or that Khanmigo, our artificial intelligence, is biased, show us, show someone else. It’s not like it’s some secret. You can — it’s there for anyone to try and be transparent on and then we will adjust if we feel that there is a bias or if we feel that it’s trying to give a point of view that might not be fair.

I know the folks at OpenAI and Google and other places, they’re trying really hard to make the underlying engines as unbiased as possible, and we’re trying to take our DNA — you know, we’ve had — actually, some of the content that, Walter, you and I have done together, we’ve had conservative — there’s appellate court judge who was skeptical of Khan Academy coming out of California. And he looked at our content on the constitution and he said, yes, that’s the way it should be taught. This is the constitution. And I had another civil rights leader who was like, well, OK, have you all papered over certain aspects of history? He also looked at some of our content and said, no, this is a full treatment of American history. And I think when both sides can see it in its totality and they say, yes, this is fair, this is academic, it’s not biased. I think most folks actually get behind it.

ISAACSON: Well, let’s talk about math for a second. You know, when somebody gets a math problem wrong, there are 100 different ways they could have it wrong. I mean, let’s just say some piece of algebra, where you don’t understand the distributive process or something, how does a one-on-one tutor help you with math in that way?

KHAN: Yes. And this is what’s really been interesting, because as many folks know, these A.I.s that have been coming out, the GPT and others, these are large language models. And so, a lot of people have been skeptical about how good are they going to be at mathematics. And even when we saw some of the first examples with GPT-4, it was doing really well in the humanities, it was doing really well in science, conceptual knowledge, but it wasn’t doing so well in math.

So, we’ve been doing working very closely — well, we’ve been spending a lot of time internally, but also working with researchers at places like OpenAI to try to get the math right. And so, when you go to Khanmigo people are actually surprised how good it is at math. I won’t say it’s perfect. It’s still going to make mistakes, but if a student just asked how to do the problem, it won’t tell you how to do a problem, it’ll ask socratically (ph), what do you think is the next step?

And if the student, let’s say, to your example, does distribute a property incorrectly, it’s actually — we have this concept called A.I. thoughts where it, on its own, thinks about how it would have approached the problem or how the student could have approached the problem. It doesn’t share that with the student, but then it compares the student’s response to that. And really, if you think about it, this is what a good tutor would do too. It would think about it.

And if the student did something different than what the A.I. thinks a reasonable path, the A.I. will often say, well, I got something a little bit different than you, can you explain your reasoning? Which is a very good pedagogical thing to do. And then, when the students explain the reasoning, the A.I. can understand it that much better. And in a lot of ways, this is what a great tutor would do. Hey, I got something different than you. Instead of saying you’re wrong or instead of saying, this is how you do it, say, explain your reasoning.

And when the student explains the reasoning, 80 percent of the time, the student might say, oh, I see where I messed up. But 20 percent of the time the A.I. might say, oh, actually, that was a better approach than the way I approached it.

ISAACSON: Well, for it to be transformative it’s got to be equitable, and it can’t help increase the divide between rich and poor. So, how do you think about that and to what extent do you hope and envision that it might be free for every kid on this planet?

KHAN: This is the core issue. As we know, one of the major sources of inequity in the education system is you might have two students who go to the same classroom, but one student whose parents are college educated, who understand the system, who have access to resources, they might actually get real tutoring when they go home or they might — their parents might tutor them while the other student might not have access to those types of resources.

So, the whole idea of having a scalable artificially intelligent tutor is to try to level that playing field as much as possible. We think already, even with the computation costs, that it’s far more accessible than traditional tutoring. And then, if the cost curve keeps going the way that it looks like it will, we think in upcoming years it will truly be something that we could give to every student as a tutor and every teacher as a teaching assistant.

ISAACSON: You talk about the personalization. That fascinates me because, obviously, when Aristotle was tutoring Alexander the Great, it was all very personalized. He knew what Alexander the Great was having trouble with, what he had difficulties with a year ago. To what extent will this thing remember you throughout your entire school career and be personalized directly for you?

KHAN: That’s what we are literally working on as we speak. Right now, if you were to us Khanmigo, it represents it represents — it remembers the conversations you’re are doing, it also remembers some of the work that you have been doing on Khan Academy, the more traditional work. But we’re hoping that by back to school, it will actually remember its conversations that it has had with you, it will remember if you’ve told it, hey, I prefer this type of tone, or it knows your reading level, it’s going to be able to really fine tune to that. So, this isn’t some science fiction, you know, even three, five, 10 years out, this is more like three months out, that’s going to be there. And then, we’re just going to continue to just make it more and more personalized so that — and we’re going to be running advocacy studies.

We’ve obviously done a ton of efficacy studies on the core of Khan Academy, but now, we’re going to, in this coming school year, see how adding the layer of artificial intelligence to your traditional Khan Academy can really accelerate student, not just in the learning, but likely their engagement as well.

ISAACSON: I mean, that sounds really awesome. But there’s one possible dark side to it remembering everything about you and being totally personalized, which is your privacy. I mean, do you have some guardrails so that I can’t subpoena or nobody will be able to get the private data it has?

KHAN: Yes. I mean, that is core of who we are. You know, back 15 years ago, when I set up Khan Academy as a not-for-profit, and I didn’t even envision that generative A.I. would advance this quickly, one of the reasons why we were not-for-profit is we recognize that student data, even pre-A.I., is a very sensitive thing and we wanted, amongst many other things, our true north to never use that data for anything that could be counterproductive.

It should only be used to improve the learning experience for the student, personalize it more, or actually improve the efficacy of the platform. So, these are things we’re taking very seriously. Even the current A.I., we are not using that information to train the artificial intelligence and some of the questions that you bring up. You know, I think for students, there’s a different context. We are making it so that — and this is one of the safety mechanisms, that everything a student does is monitorable by the teacher and by the parent. We also have a second artificial intelligence that’s monitoring the conversation with the student, and the first artificial intelligence to flag any conversations and then notify parents or teachers. So, we do have some of those safety mechanisms. But to your point, we definitely, over the coming years, especially as the A.I. starts to have this longitudinal narrative of the student, make sure that it’s only used for positive used cases.

ISAACSON: Recently, we had one of the godfathers of A.I., Geoffrey Hinton, on this show. And he told Hari Sreenivasan that there could be sort of an existential threat of A.I. What is your perception of that?

KHAN: I think it’s not for any of us right now to predict exactly what’s going to happen. But the thought experiment that I run in my mind is there’s the more conservative stance on A.I. We say, hey, we don’t know where this is going. Let’s slow it down. Let’s regulate it before we actually see it cause problems. And the problem with that is the only people who are going to follow that are the good folks, are that rule followers.

The people — the criminal organizations, the authoritarian states, they’re not going to slow down one bit. And in my mind, the most dystopian scenario is one where authoritarian governments and criminal organizations have better artificial intelligences than that rule followers. So, I actually don’t think that that’s a viable path to just act with fear and try to slow things down.

I think the other path, you will have a lot of folks say, oh, this is all going to work out, kind of like the industrial revolution. It’s somehow going to create more jobs than destroy jobs. Maybe. But I don’t think it’s enough to just hope, use hope as a strategy, so to speak. I think it is important for actors like Khan Academy and many, many others in every domain to be very proactive and say, all right, what are the risks here?

How do we mitigate them? And then, what are the benefits? And how do we maximize them so that A.I. net, net becomes a massive positive for humanity as opposed to a negative?

ISAACSON: How do you envision 10 years from now education?

KHAN: I actually think you’re going to be able to talk to a tutor, an artificially intelligent tutor, much like we’re having a conversation right now. And it might even happen in five years. And it’s going to be able to draw things out. So, it’s almost like a real-time personalized Khan Academy videos. It’s going to be able to happen in any language. You might sometimes engage with it on your phone on, on your laptop or through virtual reality. So, you feel like you are in the same room with it. So, I think it’s going to be pretty immersive.

I think you are also going to see changes to other parts of the system. Traditional assessment, the only thing you can grade in a very scalable way were your traditional Scantron multiple choice, and because of that, that’s what the education system got focused on, things that you could actually assess in a reasonably low-cost way.

Now, artificially intelligence can assess your writing, it can assess your thoughts. You have a simulation with it, you can have a dialogue with it. You can essentially have an oral exam with it, which is, you know, the gold standard for a Ph.D. thesis defense, you can now do that on demand. So, I think in five or 10 years, assessment is going to be — is going to look a lot richer. I think the teacher’s role in this artificially intelligent world, so to speak, a lot of their administrative tasks are going to be taken away, hopefully by Khanmigo, and they’re going to be able to focus on the one-to-one personal attention. And — but they’re always going to have that artificial intelligence there to help advise them.

We are actually — we have another nonprofit called Schoolhouse.world focused on peer-to-peer tutoring. We’re already leveraging artificial intelligence to give the tutor feedback on how they could tutor better based on the transcript. And then, we think we’re about a year away of — in real-time, being able to give the tutor feedback on, hey, you haven’t called on this student lately or I think this is what they are actually asking. Here’s an example problem that you could work through.

So, it’s really going to be something where it’s not humans versus artificial intelligence, it’s really going to be artificial intelligence to allow the humans to be more human.

ISAACSON: Now, I want to ask you, should colleges, when they decide to admit students, should they be — should you be allowed to submit all of that so a college can say, OK, this is how this person learns? And what about job applications? Is that something that’s too much of an invasion of privacy, or is that something that would be really useful so college admissions would be more fair?

KHAN: I think anything is reasonable as long as the people who are affected by it are bought in to it. So, I could imagine a world where a student interacts with Khanmigo over many, many years, maybe their entire K-12 experience, and then when they apply to college, they could ask Khanmigo to write a recommendation for them. And it would say but once again, this is the student asking for it. I wouldn’t want to do that behind the scenes without the student’s actual permission.

I wouldn’t be surprised if university admissions, going back to your future of education question, University of Admissions, they have to sit through 30,000 applications. They have all of these readers. It’s got to be inconsistent because, you know, depending on whether some are in a good mood, et cetera, they are, in 10 years, going to be using this type of technology.

But as long as people know how they are using it, they’re testing it as much as possible for a bias, nothing is going to be perfect, but as long as it is more perfect than what we are doing today, then I think it is a step in the right direction.

ISAACSON: Sal Khan, thank you so much for joining us.

KHAN: Thank you.