Published 2026-07-11
Tags: #education
Unregulated AI usage risks setting up students for failure. But is there anything practical that busy instructors can do beyond writing AI policy documents? In this post, I’ll sketch out my personal course structure, then lay out the source materials from the real courses and the literature.
I’ll start with the conclusion – what I will personally adopt when I teach, given my reading so far.
My main focus is to restore (or, augment) pre-AI out-of-the-classroom learning, a lot of which is recently being bypassed by unguided AI usage on homework. We do this by providing convenient, learning-inducing substitute AI, and by making the course grade be tied to proctored, in-person assessments.
Grading. The grading will be heavily placed on in-person quizzes and exams. Homework will have less weight, and each problem will just be graded on completion. E.g. 20%. The motivation to do homework is that the in-person quizzes will repeat some HW questions. If students bypass the thinking on homework, they’ll quickly learn they’re only harming their in-person quiz performance.
HW AI chat. For students who want to use AI, they are only allowed to use a problem-specific gemini gem, and are asked to turn in the chat transcript. I will have created a class-agnostic workflow that takes in the original problem statements and the reference solution so I can scale this process. This is to make use of the “sanctioned substitute” insight from psychology.
HW policy. AI usage is optional. If AI is used, students must (1) first provide a 30-minute first attempt that’s written on paper before (2) using the provided HW-specific chat, then (3) turning in the chat transcript.
Assessment content. I’ll be sure to add these flavors of tasks: (1) code review (2) plan review (3) provide test cases for this code and run them (4) conduct a runtime performance analysis of this code. This is to address the workflows that will become more common with AI-generated code.
These are other policies that I want to adopt from this post.
Active engagement. Lectures will include active worksheets, not just passive delivery.
Mastery. Students can retake quizzes up to two times, but for each to be a pass, they have to earn a very high grade (E.g. >90%). See mastery learning
Spaced repetition? I’m not sure if I can afford to add additional assessments, so I’ll only semi-encourage this through delayed assessments like midterms and final exams.
AI detection. I don’t want to get into the cat-and-mouse game of detecting if students are misusing AI in the homework. If they do it and bypass the learning needed to succeed in the in-person exams, they will fail the class. That’s it.
Beyond this post. The interventions listed in the post surprisingly feel… very hands off. We hope that the student will use the convenient AI substitute, and realize that if they don’t do the hard thinking, they will fail the weekly quizzes. I’m curious what successful CS courses will look like a decade from now, and if we figure out interventions that are very different from the ones listed in this post.
Let’s first take a look at recent algorithms courses: UW, CMU, Berkeley, Stanford, Princeton. Here are the main findings.
The contents of in-person exams, lectures and curriculum don’t seem to have changed over the years. It’s worth noting that CMU and Berkeley have oral exams and oral homework, but I have doubts on how scalable this would be for single instructors.
What seems to change the most are the policies that shape how students interact with homework.
Grades: less emphasis on non-proctored. Proctored exams take the most weight. 4 out of 5 courses placed at least 70% weight on exams, with the exception of UW only placing 50%. Princeton and CMU don’t even grade homework, or only grade based on completion.
Homework: policy only. The homework content does not seem to be changed. The main change seems to be in the policy, which mentions that AI usage is optional, and if it is used, there is a guideline on how to use it. Here’s one from UW: “You are allowed to use the course approved GPT only after you have worked on the problem for at least 30 minutes.”
Custom AI. Only UW provided its own course-specific AI. Here is a chat transcript that I used to test it. The main benefits that I saw seem to be (1) non-disclosure of solution: “I don’t want to give away the full solution” and (2) next-step hints: “try comparing … ask yourself … A small conceptual hint: …” (3) encouragement to think before re-engaging: “Spend 15–30 minutes trying to find the counterexample yourself before asking the next question.”
In summary, homework has less grade weighting, but comes with AI guidelines to ensure that the pre-AI thinking associated with homework is not bypassed. More details in the note: How real algorithm courses tackle AI.
We just saw a couple of concrete examples, and now we will broaden our search by looking at the CS education literature. We will mainly rely on Prather et al. (2025), which provided a list of course changes compiled from several papers and instructor interviews.
There is an increased weight on proctored exams, and less weight on unproctored assessments like homework.
More on code review, less on syntax production.
Educators believe code reading has become more important than writing code from scratch, and that higher level skills like code testing, problem decomposition, problem understanding, and debugging have become more essential (ES-4, ES-5). … Developers pointed out a similar shift (DS-10), but added the need to critically evaluate GenAI use and its output, writing prompts, and meticulousness as new relevant competency components
Assume that AI will be used, and guide the usage.
Submission. Make clear what the policies are. E.g. Must share chat transcript if AI is used.
Guard-railed AI. Harvard (Liu et al., 2024) built course-specific AI that used socratic method and has access to the course materials. Here is a chat log where I played around with it. Note that UW’s custom GPT also is similar in spirit.
No AI for first step. Güner & Er (2025) suggested that students who code first, then later use AI just to refine, perform better than students who start their thinking with AI. A workflow that we want to encourage, then is as follows: (1) human does the first attempt without AI before (2) starting the loop of [AI provides minimal hints, human critically reviews before asking again].
Note that the effectiveness of some of these interventions is not rigorously tested – a lot of these were anecdotal, and not compared against strong baselines. See Bauer et al. (2025) for a more detailed look at the weakness of these studies. So! Treat this list more as a brainstorming tool.
Let’s broaden our perspective even more and look at findings from psychology! If we reframe AI cheating as a behavior that students already intellectually know as bad, but find it hard to avoid in the moment, what are some effective interventions from the realm of psychology that we can adopt?
The most effective intervention is to provide a sanctioned substitute to generic AI offerings, and provide an if-then plan (Sheeran et al., 2024; Gollwitzer & Sheeran, 2006). In the context of an algorithms course, we can provide a custom chatbot with pedagogically-sound prompts and give clear HW guidelines on how to use it. For instance, underneath every problem, we can give a clear instruction like this: “Start a stopwatch for 30 minutes to write your first answer on paper, then use this problem-specific course GPT (link) to check or get hints.”
Appealing to honor codes, persuasion, or making students feel the delayed cost of cheating through frequent quizzes, doesn’t seem to have as strong evidence for reducing cheating on homework. This is pretty concerning, since this seems to be where most courses concentrate their interventions.
For more details, see the note Psychology: how to discourage ai cheating.
Given that providing an AI substitute is likely to be one of the more effective interventions, it’s worth diving deeper into how to build one.
We previously saw how Harvard (paper, chat log) and UW CS courses have their own chat bots. However, the effectiveness of these chat bots hasn’t been studied to the level of rigor described in Bauer et al. (2025).
We will instead look at two more rigorous studies – from a Harvard physics course (Kestin et al., 2025) and Google DeepMind (LearnLM Team, 2025). Both did Randomized Controlled Trials (RCTs) to show that interacting with the custom chatbot helped students obtain higher test scores with less study time, than students in a traditional active-learning class or with human tutors alone.
Here are the main components of building this solution.
One might think that building a high-quality class-specific AI would impose too much additional work on single instructors. This is an open problem that I would like to tackle, but here are some preliminary thoughts that make me think it’s tractable.
Platform cost is free. UW used custom GPT, which doesn’t incur any cost to the instructor (details: CustomGPT QnA). If your institution offers Google accounts, you can use Gemini’s gems instead. Scaffolding is implicitly baked into the static homework statement, which links each section part to a different chat link.
Pedagogical principles are also easy to encode – We can just reuse the same prompts from LearnLM Team (2025) and Kestin et al. (2025).
Human-curated step-wise explanations are probably a main bottleneck. We might reduce the cost by semi-automating it with AI. E.g. “Given this human-provided final solution, and the problem statement, break this down into steps”, then let human adjust the steps.
Evaluation as the hidden cost. Evaluation of quality is harder, but I suspect we might automate some steps that can produce a decent evaluation pipeline, and a self-improving pipeline that provides a good result. E.g. for this homework problem, generate 10 common student responses, evaluate yourself, self-improve your prompt, …. This will impose some monetary cost on the instructor, TBD on how much.
So, the main cost will most likely lie in the evaluation / self-improvement workflow and the step-wise explanation creation. It will be an interesting project to see how much of these can be automated, and various practical integration patterns to accommodate instructors of varying bandwidth. TBD for a future post!
Deep dive notes