I will confess up front that I have never been a fan of Khan Academy.
If I had ever stood in front of a classroom and delivered a canned lecture and then when students asked for help or clarification, simply delivered the exact same lecture again, parents would have called my principal, and they would have been correct to do so.
Can a vast library of short lessons of varying degrees of quality be a useful resource? Certainly. For folks who would rather watch than read, Khan Academy, like Youtube in general, can be a handy way to look something up (particularly at 11:30 at night).
When ads for Khan’s new generative language program Khanmigo started popping up, I was dubious. It’s no surprise that they want to jump on the “Artificial Intelligence” bandwagon, and join the uncountable number of companies promising to change the future of education. But while that bandwagon can carry you to some useful destinations, it has some serious limitations that you should understand before you buy a ticket.
Artificial intelligence is not intelligent
Generative language algorithms, aka chatbots, do not understand what they’re saying in any conventional sense of the word, nor do they “understand” the prompt that you give them. They are, as Science Fiction and tech writer Cory Doctorow puts it, “plausible sentence generators.”
The most “famous” of the chatbots is ChatGPT, but they all work basically the same way – in fact, at this point, many of them are using ChatGPT as their guts and spine. Khanmigo is built on GPT-4, the same generative language program as ChatGPT, the result of OpenAI – GPT-4 and ChatGPT’s parent company, which reached out to Sal Khan last year.
What ChatFPT does is actually pretty simple: they predict the next probable word. Just like your phone, suggesting the next word in that text message you’re typing out.
What makes ChatGPT good is the sheer mountain of “data” it has stored, and by “data” we mean heaping mountains of word-strings, hoovered up from the internet. It can then do what computers do best—extract a huge number of patterns quickly (if you want a really deep dive into the guts of chatbots, this post will take care of you).
A chatbot cannot tell that a sentence means something; it can just determine that it shares a bunch of characteristics with millions of sentences that, it has been “told” by its programmers, are meaningful. It can tell if the string of words are words that have a high probability of hanging out together. But it does not know whether the sentence is true or false.
Given enough data, the chatbot can use cue words from the prompt to narrow down the data it uses and thereby the rules that it follows. That’s how it can imitate famous authors, or the Bible. Take a string that starts “Watch the big red… .” If you have cued it to look at movies, a good next probable word would be “one.” If you have cued it to look at children’s books, then “dog” looks good.
Humans do this kind of linguistic prediction all the time. Let me give you an ___.
Yeah, you knew the word would be “example” before I ever got there. But you follow not only the rhythm and flow of the sentence, but you understand what it means. A chatbot does not.
Still, chatbots are good at some things
Form letters. College application essays. Legal, or legal-ish, letters. Letters of reference. Recipes. Those stupid listicles that clog the internet. Or any internet article that you want to grab a search engine’s attention with. Basically, any sort of writing that is essentially formulaic. Doctorow sums that part up aptly:
Bullshit begets bullshit, because no one wants to be bullshitted. In the bullshit wars, chatbots are weapons of mass destruction. None of this prose is good, none of it is really socially useful, but there’s demand for it.
Unfortunately for teachers, dull formulaic writing also describes plenty of assignments. The beloved five paragraph essay (intro, support, support, support, conclusion). Or a paper about some use of a literary device in a well-trod work of literature. I’ve long complained about the “shake and bake” high school research paper, in which students are directed to find a handful of sources about a subject, and then write a paper that says what those sources say while managing to not exactly plagiarize the sources.
Chatbots were made to write those shake and bake papers, because if someone has already written about the importance of the rise of nationalism in 19th Century Europe, a chatbot will just rummage around in its data mountain, pull up all the writing on that topic, run it through a blender, and pour it out again, in smooth, plausible sentences.
Much has been written about how AI like ChatGPT poses an existential threat to English classes, and that’s only partly true, because, as writing scholar John Warner points out, “ChatGPT can’t kill anything worth preserving.” If a piece of software can do your assignment plausibly well, perhaps you need a different assignment.
Chatbots are not good at other things
It cannot be said enough: chatbots do not understand what they are writing. So they can easily, and plausibly, crank out nonsense. Asked to write me an essay about Hamlet and to include quotes, ChatGPT wrote me an essay interspersed with quotes that had nothing to do with its point. Asked to write a biography of a non-existent Elizabethan writer, with sources, it produced the biography and the sources. I periodically ask it to write a biography of me; it is always wildly but confidently inaccurate, and credits me for books that not only were not written by me, but that do not exist.
Various services are creating workarounds to deal with these hallucinations, but a fundamental problem of chatbots remains that you can’t always trust them.
I have played with language generating programs for years, and the new generation excel in one new way. Their writing still lacks depth and insight and occasionally adds to that flashes of striking nonsense, but it does it all with a smooth polish. The flow and fluency are just beautiful. It is like an essay by that kid in class who slept through the lecture and skipped the reading and is just trying to bullshit his way through—but he’s one of the most technically proficient writers you’ve ever taught.
Meanwhile, chatbots will write your book for you, websites are filling up with human-written articles passed through an AI rewriter, raising the question—what if AI’s are increasingly “trained” with writing created by other AIs?
What use is a chatbot to students?
Khanmigo promises parents a tutor. Chegg offers a study buddy. Various companies offer guidance, assistance, adaptive support, extra help. Chatbots offer an even faster way to scout the internet for answers to homework, a search and summarize that Microsoft’s search engine Bing has now in a built in chatbot feature (that is almost as insistent as Clippy).
Khan knows there’s an issue, though, as in this interview Sal Khan tries to thread the needle. “Unlike ChatGPT, Khanmigo doesn’t do school work for students but instead acts as a tutor and guide to help them learn.” The terms of service for Khanmigo include the note that users are forbidden:
Use of Pilot Features to commit any act of educational dishonesty, or to mislead any person that Output generated use of AI features included in Pilot Features is human-generated.
That’s nice, but as OpenAI says, there’s no way to detect students who are cheating with a chatbot.
I have found that when it comes to essays, chatbots end up in the same general rut. In other words, a teacher might not spot one chatbot essay, but she might very well notice ten. As Khanmigo warns, again in its terms of service:
You acknowledge that due to the nature of machine learning and the technology powering certain Pilot Features, Output may not be unique and the same or similar output may be provided to other users.
Chatbots can be used for editing; let them read your essay and tell you what to fix (Word has had versions of this feature for a while), though they are unreliable in catching content errors. And chatbots can be relentless drillmasters; if you want to practice certain sorts of problems over and over until you get it, a chatbot can stay up all night and keep feeding you practice drills.
And, of course, a chatbot can just do your assignment for you.
What use is a chatbot to teachers?
Teachers are still working on this one. Some are wasting time trying to figure out how to bar chatbots from their classrooms. Others are experimenting with chatbot capabilities, or reading their way through the miles of ChatGPT guides.
Chatbots can create tons of worksheets and exercises customized to particular needs of classes (“I need 100 subject-verb agreement practice sentences with half focusing on irregular verbs”). They can also create lesson plans, though these are essentially mash-ups of available lesson plans on the internet, or they can “write” tests.
Just as there are ethical concerns for students, those exist for teachers as well. Too tired to grade all those essays? Let ChatFPT do it. Khanmigo’s terms of service address this as well; included in the “forbidden uses” list we find:
Use of Pilot Features for assessment purposes or to make decisions about a child or any User.
So in a world with ChatGPT, why does anyone need Khanmigo?
This is the big question. If so many of these chatbots are built on OpenAI’s GPT-4 (or 5 or other upcoming iterations as the software eats more and more data), why do folks need any of them?
Part of the trick with chatbots is the kinds of limitations and safeguards put in place, because chatbots get their language examples from the internet, and the internet, you may have heard, is not always a nice place. In 2016 Microsoft famously shut down their chatbot Tay after it started issuing lewd and racist posts.
ChatGPT has guardrails, and those who license it or one of the competing chatbots can create their own sets of guardrails in an attempt to differentiate themselves from a marketplace rapidly filling with companies all selling versions of the same product.
So we have FreedomGPT (built on Alpaca from Stanford University), which sells itself on the promise that there are no limitations, so that the chatbot may be just as offensive as the user wishes. Magisterium AI was created to “revolutionize academic research in Catholic education” and only uses official documents of the church. “It’s never going to give you a wrong answer,” promises Fr. Philip Larrey of the Pontifical Lateran University in Rome. Good luck with that.
Khanmigo promises to have adapted GPT-4 to the world of school, to tutor students without just giving them the answers. The pitch is “tailored support” and “engaging lessons,” but all done “ethically.” In other words, Khanmigo is promising that it will take your student right up to the line of cheating, but not actually cross it, even though the chatbot doesn’t actually understand a single word it’s texting out.
Putting the toothpaste back in the tube
Chatbots were inevitable. The next step after a worldwide electronic collection of everything, good or bad, that could come from a human brain was always going to be some electronic means of sifting quickly through all of that.
But if the technology is unavoidable, so are the ethical issues. We haven’t even scratched the surface of issues for creators in the face of a super-powered plagiarism machine. And while this technology can short-circuit many human endeavors, it can absolutely destroy the whole work of education, collapsing quickly into a world in which the computer makes the assignment, the computer completes the assignment, the computer grades the assignment, and nobody learns anything.
Yet Khanmigo, which has a step up on the competition thanks to brand recognition, is going to be hugely attractive to homeschoolers and the microschool movement, a movement in which six students with a computer in someone’s kitchen can be a school. That’s good news for the school voucher movement, which needs an answer to the question, “What do low-wealth families do if a voucher still isn’t enough to get them into a decent private school?” Programs like Khanmigo are perfect for bypassing the problem of providing education for everyone at a low, low cost — just sign them up for an education chatbot.
This technology is not going to solve the many ethical problems that come bundled with it, and right now we haven’t even figured out what all the problems are going to be. Khanmigo is going to be used in ways its creators didn’t imagine, and it’s going to create problems that nobody has fully anticipated. But that toothpaste is not going back into the tube.
But human beings are going to have to talk seriously about more than market opportunities. Computer software does not know or understand what it’s doing or why it matters. That part of the work remains with flesh and blood types, and the computer tutor is not going to be able to help.
