1 00:00:00,560 --> 00:00:05,760 The rapid evolution of generative AI  tools has introduced an expanding set 2 00:00:05,760 --> 00:00:10,960 of educational applications. In this  episode, we discuss how these changes 3 00:00:10,960 --> 00:00:22,080 are affecting faculty and classrooms. Thanks for joining us for Tea for Teaching, 4 00:00:22,080 --> 00:00:27,040 an informal discussion of innovative and  effective practices in teaching and learning. 5 00:00:27,040 --> 00:00:30,880 This podcast series is hosted  by John Kane, an economist... 6 00:00:30,880 --> 00:00:33,440 ...and Rebecca Mushtare, a graphic designer... 7 00:00:33,440 --> 00:00:37,200 ...and features guests doing important  research and advocacy work to make 8 00:00:37,200 --> 00:00:49,760 higher education more inclusive  and supportive of all learners. 9 00:00:49,760 --> 00:00:55,440 Our guests today are Dan Levy and Angela Perez  Albertos. Dan is an economist and a senior 10 00:00:55,440 --> 00:01:00,960 lecturer in Public Policy at Harvard University  where he teaches courses in quantitative methods, 11 00:01:00,960 --> 00:01:06,960 policy analysis, and program evaluation.  Angela is a graduate of the MPA program in 12 00:01:06,960 --> 00:01:11,520 International Development at the Harvard  Kennedy School, and is the U.S. Head of 13 00:01:11,520 --> 00:01:16,960 Strategy at Innovamat. Dan and Angela are  the authors of the first, and now the second, 14 00:01:16,960 --> 00:01:23,360 editions of Teaching Effectively with  ChatGPT. Welcome back, Dan and Angela. 15 00:01:23,360 --> 00:01:25,120 Thank you so much for having us. 16 00:01:25,120 --> 00:01:26,800 Thank you very much. 17 00:01:26,800 --> 00:01:30,960 Today's teas are:... Dan. Are  you drinking tea with us today? 18 00:01:30,960 --> 00:01:36,240 Yes, Moroccan tea. My family is originally  from Morocco, so I tend to drink this one. 19 00:01:36,240 --> 00:01:40,560 Always a good flavor, for  sure. How about you, Angela? 20 00:01:40,560 --> 00:01:44,800 Yeah, this morning I'm drinking chai tea  latte. This is my usual morning drink. 21 00:01:44,800 --> 00:01:46,160 How about you, John? 22 00:01:46,160 --> 00:01:51,200 I have a tea which actually has a French  name, which I'm not going to try to pronounce, 23 00:01:51,200 --> 00:01:57,040 but the English translation is chimps’  tea. It's a black tea with fig and honey 24 00:01:57,040 --> 00:02:01,200 and a few other things in it, and it's  really delicious. It was sent to us a 25 00:02:01,200 --> 00:02:05,440 while back by one of our listeners  in France, which was really nice. 26 00:02:05,440 --> 00:02:07,920 It's always nice to be able to try new things. I 27 00:02:07,920 --> 00:02:12,720 have an Assam tea this morning  from Pekoe Tea in Edinburgh. 28 00:02:12,720 --> 00:02:18,720 Impressive. She got it there herself, mine  came through the mail. We've invited you 29 00:02:18,720 --> 00:02:24,240 here today to discuss the second edition of  Teaching Effectively with ChatGPT. Since some 30 00:02:24,240 --> 00:02:28,800 of our listeners may not have heard our earlier  discussion about the first edition of this book, 31 00:02:28,800 --> 00:02:33,600 can you provide an overview of the purpose  of the book and who is a target audience. 32 00:02:33,600 --> 00:02:37,440 Thank you so much, John. The main  purpose of the book is to provide 33 00:02:37,440 --> 00:02:43,680 a guide for educators to understand how  ChatGPT and other AI tools can be useful 34 00:02:43,680 --> 00:02:49,120 for more effective teaching and more  effective learning. Now what our main 35 00:02:49,120 --> 00:02:54,960 approach is to provide plenty of examples  that are practical, to really demystify 36 00:02:54,960 --> 00:03:00,000 what AI looks like and how it can look like  in and outside the classroom for learning. 37 00:03:00,000 --> 00:03:04,000 In Chapter Two, you describe three guiding  pedagogical principles that you use throughout 38 00:03:04,000 --> 00:03:08,640 the book, and these principles we certainly  talked about on the podcast before and are great 39 00:03:08,640 --> 00:03:13,200 advice to anyone involved in course design,  whether or not they're using generative AI. 40 00:03:13,200 --> 00:03:16,960 Can you tell us a little bit about these  principles and why they're so important? 41 00:03:16,960 --> 00:03:22,480 Sure. So the three key pedagogic principles  were meant to give the reader of the book an 42 00:03:22,480 --> 00:03:28,720 overview of what are the underpinnings  of the pedagogic advice of the book. 43 00:03:28,720 --> 00:03:34,800 For those of you who listened to this podcast,  you might not be surprised in hearing the first 44 00:03:34,800 --> 00:03:40,560 one is to be student centered. That sounds  very obvious, but yet we spend a lot more time 45 00:03:40,560 --> 00:03:44,880 thinking about what we do in the classroom and  less time about what our students are doing in 46 00:03:44,880 --> 00:03:50,400 the classroom. We spend more time thinking about  what we will cover in a course, rather than what 47 00:03:50,400 --> 00:03:57,040 our students will uncover. And so being student  centered just means paying very much attention, 48 00:03:57,040 --> 00:04:03,760 and the focus of the teaching enterprise is  stimulating the learning of our students. 49 00:04:03,760 --> 00:04:08,960 So that's kind of that principle. The second  principle is to plan for active learning. I think 50 00:04:08,960 --> 00:04:14,800 the two of you in this show have said, many times,  there's ample research suggesting that people 51 00:04:14,800 --> 00:04:21,360 don't learn by just listening to someone telling  them stories they learn when they actively engage. 52 00:04:21,360 --> 00:04:26,080 And the third one is to begin with the end in  mind, and this is a little bit of a different 53 00:04:26,080 --> 00:04:33,680 way of saying we use backward design. So the idea  we start with, what do we want the students to be 54 00:04:33,680 --> 00:04:39,360 able to do or master at the end of every  learning experience, and then we design 55 00:04:39,360 --> 00:04:45,920 backwards from there. And so the book, very much,  has these three principles embedded. A lot of the 56 00:04:45,920 --> 00:04:51,280 activities that we propose for educators to  do have to do with how they can create more 57 00:04:51,280 --> 00:04:58,000 active learning in the classroom, and so the three  principles embody a lot of what the book is about. 58 00:04:58,000 --> 00:05:02,880 Could you each give us an example or two  of ways in which Chat GPT can be used to 59 00:05:02,880 --> 00:05:06,080 design new classes or redesign existing courses? 60 00:05:06,080 --> 00:05:10,880 Absolutely. So I can get started with a  very cool use that, Dan and I actually 61 00:05:10,880 --> 00:05:18,240 did together when we were redesigning one of the  classes on the Bayes’ rule in a statistic class, 62 00:05:18,240 --> 00:05:25,120 and there was a specific question or activity  that was based on COVID and a positive and 63 00:05:25,120 --> 00:05:30,720 negative test and the possibility of  a false positive and false negative, 64 00:05:30,720 --> 00:05:35,520 and we felt that was already an outdated activity  by the time that we were working together, 65 00:05:35,520 --> 00:05:42,480 and so we wanted to update it to new examples  that felt more appropriate for the times. And 66 00:05:42,480 --> 00:05:47,680 we could not pin down what could an idea of  a different theme or contextualized activity 67 00:05:47,680 --> 00:05:53,680 could be. And we asked ChatGPT to just share what  could be like 10 potential topics or things that 68 00:05:53,680 --> 00:05:58,960 we could explore that would help the students  understand how Bayes’ rules can be valuable in 69 00:05:58,960 --> 00:06:05,120 a real-world context. And it gave us an infinite  amount of ideas that we would have never guessed 70 00:06:05,120 --> 00:06:10,800 just by brainstorming him and I together, and  just to give like two specific examples that 71 00:06:10,800 --> 00:06:17,680 we liked. One was about art forgery and  being able to identify fake art pieces, 72 00:06:17,680 --> 00:06:24,400 and the other one that we ended up using was  about being able to detect AI plagiarism in 73 00:06:24,400 --> 00:06:30,720 student work. And so it's incredibly helpful  for anything that has to do with brainstorming 74 00:06:30,720 --> 00:06:36,000 contextualized tests and activities to  context that are valuable for students. 75 00:06:36,000 --> 00:06:43,520 Yeah, so similar to Angela, I can think of  several examples in terms of designing course, 76 00:06:43,520 --> 00:06:50,080 but just to give a recent one, this is more about  designing an assessment, similar to what Angela 77 00:06:50,080 --> 00:06:55,280 just mentioned. I created a project for a course…  and I'm sure we're going to talk in a little bit 78 00:06:55,280 --> 00:07:00,320 about projects... the syllabus has a lot of  things, and there was a question that we had 79 00:07:00,320 --> 00:07:09,040 created last year that we wanted to change.  And so I uploaded the question to ChatGPT, 80 00:07:09,040 --> 00:07:16,080 and I said, “Can you tell me what you perceive  to be the learning goals of this question that I 81 00:07:16,080 --> 00:07:23,840 just gave you?” And I think it did an amazing job,  better than I could have stated them myself. This 82 00:07:23,840 --> 00:07:29,520 was very helpful. I think part of it is because  it had access to the goals of the course, but I 83 00:07:29,520 --> 00:07:37,600 think it was very helpful in helping me be better  at interacting with ChatGPT in the refinement of 84 00:07:37,600 --> 00:07:43,200 this learning activity. So that's just like one  example. I realized both examples that Angela 85 00:07:43,200 --> 00:07:49,680 and I gave are in the realm of assessment, but it  turns out to be a very important realm in terms 86 00:07:49,680 --> 00:07:57,200 of designing sort of a new session or new course,  the book goes over to what I think, still today, 87 00:07:57,200 --> 00:08:02,800 is my favorite and maybe Angela's favorite  activity. This was kind of maybe the spark 88 00:08:02,800 --> 00:08:10,160 of the book. We were trying to think about an  activity surrounding the Cuban Missile Crisis. 89 00:08:10,160 --> 00:08:18,400 And over two weeks, basically had a conversation  with ChatGPT along the lines of, “help me design 90 00:08:18,400 --> 00:08:24,720 a better class session on this topic that I don't  know too much about.” After I saw that, I thought 91 00:08:24,720 --> 00:08:30,560 this has a potential to be an incredibly useful  tool for teachers and learners, as Angela said. 92 00:08:31,440 --> 00:08:36,320 Your first example seems like a really good  reminder that we can use tools like this 93 00:08:36,320 --> 00:08:40,320 to help implement things like TILT (or  transparency in learning and teaching), 94 00:08:40,320 --> 00:08:47,360 having it help us be more clear about what the  course objectives or assignment objectives are, 95 00:08:47,360 --> 00:08:51,040 so that we can then convey  that information to students. 96 00:08:51,040 --> 00:08:55,440 Right. And you might have the reaction, but how  can you not know? It's not that you don't know, 97 00:08:55,440 --> 00:09:00,800 it's that maybe some of them are subconscious  to you. I mean, one of the most useful things 98 00:09:00,800 --> 00:09:07,840 that a teacher can have is someone, or in this  case, it's not a person, but put a mirror to 99 00:09:07,840 --> 00:09:12,640 what you're trying to do in the classroom. So  that's why having a colleague come and see you 100 00:09:12,640 --> 00:09:18,560 teach is so valuable, because you might do things  and you might not realize why you're doing them, 101 00:09:18,560 --> 00:09:25,760 and maybe having someone ask you or sort of give  you ideas for why you're doing them, it's helpful. 102 00:09:25,760 --> 00:09:31,360 The book is based on the premise that  AI can be an incredible assistant, 103 00:09:31,360 --> 00:09:36,160 but also an incredible thought partner.  And I think both of those examples have 104 00:09:36,160 --> 00:09:40,560 those two roles being played quite intensively. 105 00:09:40,560 --> 00:09:45,760 One of the other things that you certainly have  examples of in the book is the way that ChatGPT 106 00:09:45,760 --> 00:09:50,640 can be used during class sessions.  Can you give some examples of this? 107 00:09:50,640 --> 00:09:57,040 Yes, we can give several examples. So, one of  them, it's an example that is described in the 108 00:09:57,040 --> 00:10:03,600 book. But our colleague, Mitch Weiss, who teaches  at the business school, has a wonderful activity 109 00:10:03,600 --> 00:10:11,760 that is about discovering the capabilities of AI,  and students use that in the classroom in a case 110 00:10:11,760 --> 00:10:17,600 called Storrowed. And so probably not the best  use of time here to describe the full activity, 111 00:10:17,600 --> 00:10:23,520 but suffice it to say that students are put into  two groups, and they're kind of competing to be 112 00:10:23,520 --> 00:10:30,640 able to see who comes up with the best ideas,  but they're able to use AI for proposing some 113 00:10:30,640 --> 00:10:35,520 of their ideas. So that seemed like a very  interesting one. The book also highlights 114 00:10:35,520 --> 00:10:42,080 two examples of using AI to summarize student  input in the classroom, and I think that's 115 00:10:42,080 --> 00:10:48,080 an incredibly powerful tool. I think a lot of  educators have heard about the one minute paper, 116 00:10:48,080 --> 00:10:54,480 the way that it typically worked, at least until  not too long ago, was you ask people to give you 117 00:10:54,480 --> 00:10:58,880 what was the main idea that they learned, or  maybe the muddiest point, or whatever it is, 118 00:10:58,880 --> 00:11:03,760 at the end of class. This is incredibly  valuable, but normally you would just get 119 00:11:03,760 --> 00:11:09,840 out of the classroom and then have to read and  sort of think about how it works. And now you can 120 00:11:09,840 --> 00:11:16,800 do it in the classroom, live and understand what  are the main themes that people are picking up. 121 00:11:16,800 --> 00:11:22,960 You can also do it with questions. And then  a very exciting use that I experimented with 122 00:11:22,960 --> 00:11:31,280 a month ago is with group work. I don't know how  much you have been in situations where you assign 123 00:11:31,280 --> 00:11:37,360 students to work in groups, and then 10 minutes  later, you're trying to debrief. But debriefing 124 00:11:37,360 --> 00:11:43,120 is a little bit hard because you don't really know  what happened in the groups. You might circulate, 125 00:11:43,120 --> 00:11:48,320 but if you circulate, typically, the moment  you approach a student group, the conversation 126 00:11:48,320 --> 00:11:54,400 changes, so you're not really listening to what  they're describing. And so what I did is to build 127 00:11:54,400 --> 00:12:00,240 a simple custom GPT that did the following. So  the students were put into groups, and they were 128 00:12:00,240 --> 00:12:07,600 asked to input their answers to the activity  in a Google slide. So this was typical use, 129 00:12:07,600 --> 00:12:12,960 especially during COVID. You did breakout rooms,  everyone fills in a slide. As soon as they're done 130 00:12:12,960 --> 00:12:21,200 doing that, what custom GPT does is it produces  four slides for the instructor. So this is in a 131 00:12:21,200 --> 00:12:27,840 matter of a minute. And so the first slide has  the main themes. So of all the groups you had in 132 00:12:27,840 --> 00:12:33,840 the classroom here, the main themes that were  emerging. The second slide has particularly 133 00:12:33,840 --> 00:12:40,560 novel or interesting ideas that emerge. But not  only they tell you the idea, but it also tells 134 00:12:40,560 --> 00:12:45,680 you it was group three that came up with this  idea. So you can use that in the debrief, “Hey, 135 00:12:45,680 --> 00:12:51,920 group three, I noticed you came up with this idea.  Can you expand on this?” The third slide tells you 136 00:12:51,920 --> 00:12:57,520 about points of contrast or tension between what  the groups did, and then the fourth slide gives 137 00:12:57,520 --> 00:13:03,920 you some ideas for how you could debrief. And so  the fact that all of this is available within one 138 00:13:03,920 --> 00:13:10,480 minute of students finishing the group work, I  found it to be incredibly empowering to lead a 139 00:13:10,480 --> 00:13:16,960 much better debrief discussion. So those are the  kinds of uses that I think advance learning in 140 00:13:16,960 --> 00:13:22,720 the classroom in a way that it would be very  hard to do without a technology like this. 141 00:13:22,720 --> 00:13:28,560 You have a chapter discussing how ChatGPT can  be used to either design or grade assessments, 142 00:13:28,560 --> 00:13:32,400 and there's been a lot of discussion on  this in various social media forums and 143 00:13:32,400 --> 00:13:37,280 also in some editorials and articles and  so forth, in terms of whether generative 144 00:13:37,280 --> 00:13:42,800 AI should be used to create assessments and  to grade student work, with some students 145 00:13:42,800 --> 00:13:47,520 being concerned that they're not getting the  direct feedback from their professors. Could 146 00:13:47,520 --> 00:13:53,120 you talk a little bit about what ways might  be appropriate and how this may be helpful. 147 00:13:53,120 --> 00:13:59,040 So, I would like to separate the question into  two parts. One is about designing assessment, 148 00:13:59,040 --> 00:14:04,720 and the other one is about giving feedback in  terms of assessment. I am convinced that for 149 00:14:04,720 --> 00:14:11,920 designing assessment, AI can be incredibly helpful  without sacrificing the voice of the instructor or 150 00:14:11,920 --> 00:14:17,920 the learning objectives. And the two examples that  we gave at the beginning, I think, are suggestive 151 00:14:17,920 --> 00:14:23,360 of how can they be used. It can convert you in  a much more creative teacher, because you have a 152 00:14:23,360 --> 00:14:29,040 thought partner with which to bounce ideas. It can  give you feedback on the assessment that you're 153 00:14:29,040 --> 00:14:36,560 doing. My colleague from the business school,  Mitch Weiss, again, he has this custom GPT that is 154 00:14:36,560 --> 00:14:45,600 like eliminate ambiguity in my exam questions. And  so he gives the exam question to the AI tool and 155 00:14:45,600 --> 00:14:51,680 essentially “write it in this way so that there's  less ambiguity for students.” So you could imagine 156 00:14:51,680 --> 00:14:58,880 ways in which the design of assessments can be  improved with the help of AI. I'm not saying we 157 00:14:58,880 --> 00:15:03,280 should farm out the design. It's like, “okay, I'm  teaching this class, and now you go, AI, create 158 00:15:03,280 --> 00:15:09,680 the assessment, and that's it.” But I do think,  in my mind, whatever drawbacks there are, weighing 159 00:15:09,680 --> 00:15:17,360 everything, I think the impact is positive. For  providing feedback on assessment, I think you have 160 00:15:17,360 --> 00:15:24,560 decided this is a big topic, a lot of controversy  about student expectations on assessment and so 161 00:15:24,560 --> 00:15:31,840 on. I am not going to tell you that I'm convinced  about what the right answer is, but I would say, 162 00:15:31,840 --> 00:15:38,160 as we describe in the book, there’'re both  benefits and risks associated with doing this. 163 00:15:38,160 --> 00:15:45,920 But what I would like to suggest is that what we  know from the science of learning is that students 164 00:15:45,920 --> 00:15:53,760 benefit from both practice and timely feedback  on that practice. So that we know. And so the 165 00:15:53,760 --> 00:16:01,920 question to me is, are our students getting enough  opportunities to practice and timely feedback? And 166 00:16:01,920 --> 00:16:08,400 my sense is the majority of us educators feel that  the answer to that question is, no matter how hard 167 00:16:08,400 --> 00:16:15,360 we work, it's very hard to provide timely feedback  to students of the sort and in the quantity that 168 00:16:15,360 --> 00:16:21,920 they need to really, really make a difference in  terms of their skill development. And so what we 169 00:16:21,920 --> 00:16:28,240 end up is with assessment strategies that are the  most feasible given all the constraints that we 170 00:16:28,240 --> 00:16:33,520 face. But I think we're leaving some value on  the table in terms of our students being able 171 00:16:33,520 --> 00:16:39,680 to practice. So I'm not suggesting that AI should  do all the assessment work for us, but if it could 172 00:16:39,680 --> 00:16:46,320 work as a complement to the work that we already  do, I think this could potentially be a useful 173 00:16:46,320 --> 00:16:53,440 thing. Just to be super concrete, maybe if you had  five ways of assessing students in the semester, 174 00:16:53,440 --> 00:17:00,240 but you're only able to grade two because of time  constraints or whatever it is, maybe you can use, 175 00:17:00,240 --> 00:17:06,480 in a very transparent way, here's AI to help  you provide feedback on the assessment. Now, 176 00:17:06,480 --> 00:17:11,840 of course, this has to be done with a lot of  care and thought, but we can't do it by hiding 177 00:17:11,840 --> 00:17:16,560 from the students that we're doing it. And also, I  think we need to explain to the students why we're 178 00:17:16,560 --> 00:17:22,240 doing it, and if we're just saying to ChatGPT,  “just give them feedback without our input,” 179 00:17:22,240 --> 00:17:28,400 then our students, very justifiably will say,  “Well, what are you doing? So I can do that.” 180 00:17:28,400 --> 00:17:36,160 So I think there is thought that needs to go into  how AI is used to be able to do it. Just to close, 181 00:17:36,160 --> 00:17:42,480 we have here the Kennedy School, a course taught  by Shard Goel, Teddy Svoronos, and myself, 182 00:17:42,480 --> 00:17:49,680 and we have our Teaching Fellow, Calvin Isley. He  basically helped us design a system by which the 183 00:17:49,680 --> 00:17:58,720 assignments would have a preliminary assessment  done by AI, and then our teaching team would 184 00:17:58,720 --> 00:18:05,520 take into account that and use it as input for  grading the assignments. Students were able to 185 00:18:05,520 --> 00:18:13,840 opt in to this pilot. And what I think happened  is the students who opted in got more detailed 186 00:18:13,840 --> 00:18:21,040 feedback than they would have gotten in a system  where we're just counting on human labor. So, 187 00:18:21,040 --> 00:18:28,400 in sum, I'm not saying AI should do everything in  providing feedback, but I do think there's a role. 188 00:18:28,400 --> 00:18:32,400 And humans get tired. And if we're  providing feedback on many students, 189 00:18:32,400 --> 00:18:37,360 the quality of that feedback may very much vary  over time in a way that it wouldn't if you're 190 00:18:37,360 --> 00:18:41,360 working with an AI tool for assistance  and at least providing a starting point. 191 00:18:41,360 --> 00:18:44,640 For sure. I mean, one of the things we  say in the book is that AI has like, 192 00:18:44,640 --> 00:18:50,560 two very important virtues: it’s available  24/7, and it has infinite patience, 193 00:18:50,560 --> 00:18:54,480 and I don't know many human  beings for whom we can say that. 194 00:18:54,480 --> 00:18:57,920 One of the things that you just highlighted  was student practice. And you have a number 195 00:18:57,920 --> 00:19:02,080 of chapters in your book about how  ChatGPT can help students learn. 196 00:19:02,080 --> 00:19:07,040 Can you give us some examples in which  ChatGPT can support student learning? 197 00:19:07,040 --> 00:19:12,000 Absolutely, these were one of the chapters that  I was more excited about writing, because when 198 00:19:12,000 --> 00:19:17,040 we find many of the books that are there is that  they focus a lot on the teaching side of things. 199 00:19:17,040 --> 00:19:22,640 But there's so much that as educators, we can do  to just teach students to use ChatGPT in a way 200 00:19:22,640 --> 00:19:27,840 that is beneficial for learning. And just to give  a couple of examples, the first one I'll give is 201 00:19:27,840 --> 00:19:33,440 in continuation with this idea of practice, we  have an example in the book of one student that 202 00:19:33,440 --> 00:19:40,000 was studying medicine, and she was learning and  preparing for an exam, and as she was finalizing 203 00:19:40,000 --> 00:19:44,880 her preparation, she wanted more activities to  practice, because she had already completed all 204 00:19:44,880 --> 00:19:51,840 of the activities that her teacher provided. And  so she just asked ChatGPT, “Can you create more 205 00:19:51,840 --> 00:19:57,040 questions? These are some of the questions that  I received. Can you generate more questions and 206 00:19:57,040 --> 00:20:01,120 then don't give me the answer? I will give you  the answer, and then you will let me know if 207 00:20:01,120 --> 00:20:07,600 it's correct or not.” So it's just an extension of  this practice that, again, it's one of the things 208 00:20:07,600 --> 00:20:12,240 that for educators, it's hard to provide enough  practice opportunities for students because it 209 00:20:12,240 --> 00:20:17,920 means generating more questions and more questions  that cannot be used for formal assessments. So 210 00:20:17,920 --> 00:20:24,240 that's one possible example. The other possible  example, it's the personalization of learning, 211 00:20:24,240 --> 00:20:30,800 and this, again, it's one of the big moments that  for Dan and I, it just sparked our interest even 212 00:20:30,800 --> 00:20:39,440 more for what AI could do for learning. And the  specific moment was related to time when in one 213 00:20:39,440 --> 00:20:46,000 of Dan's classes, he encouraged his students to  learn about risk management strategies before the 214 00:20:46,000 --> 00:20:51,360 class itself, and then he asked. The students to  share what the conversation with ChatGPT had been 215 00:20:51,360 --> 00:20:57,680 like. In addition to that, he nudges students to  ask ChatGPT about risk management in a certain 216 00:20:57,680 --> 00:21:03,920 way. So for example, sharing what context  they had about risk management strategies, 217 00:21:03,920 --> 00:21:10,320 etc. And we basically saw very different  conversations depending on the student. 218 00:21:10,320 --> 00:21:15,760 You had, students that were much more familiar  with risk management strategies and the kind 219 00:21:15,760 --> 00:21:19,760 of explanations that they received were  much more technical, much more detailed, 220 00:21:19,760 --> 00:21:24,880 less intuitive for somebody that would be just  exposed to this concept. Well, for some students, 221 00:21:24,880 --> 00:21:30,160 it was the complete opposite. It was a much  more intuitive, just like high level conceptual 222 00:21:30,160 --> 00:21:35,120 explanation with lots of examples that could be  relatable. And this is something that would be 223 00:21:35,120 --> 00:21:40,720 very hard for an educator to replicate, a specific  one-on-one explanation that would fit the needs of 224 00:21:40,720 --> 00:21:47,040 every single student. So many multiple uses for  students that can really help improve learning. 225 00:21:47,040 --> 00:21:53,920 In the spring of 2025 we had a reading group on  campus that read your book. And just as a little 226 00:21:53,920 --> 00:21:58,320 bit of an aside, one of the things that happened  is, right at the very beginning, people suggested 227 00:21:58,320 --> 00:22:03,280 that people would experiment with something, try  it, and then bring it back and report at the next 228 00:22:03,280 --> 00:22:07,200 reading group meeting. And that worked really  well. It's a great book for a reading group. But 229 00:22:07,200 --> 00:22:12,720 one of the things that surprised people a little  bit who had experimented with ChatGPT was the 230 00:22:12,720 --> 00:22:18,400 ability to customize it to meet your individual  needs. Now some people weren't able to do that 231 00:22:18,400 --> 00:22:23,520 very well if they were using it in a wide variety  of applications, but now, with the new edition, 232 00:22:23,520 --> 00:22:29,680 you're able to talk about projects in ChatGPT.  Could you talk a little bit about how you can 233 00:22:29,680 --> 00:22:36,960 customize ChatGPT in general, and also how you  can use projects for a similar sort of purpose? 234 00:22:36,960 --> 00:22:42,720 Absolutely. So to put it simple, because sometimes  I feel it can be hard to get lost in all of these, 235 00:22:42,720 --> 00:22:51,520 like features and new releases of ChatGPT. The  idea here is that you can customize ChatGPT so 236 00:22:51,520 --> 00:22:58,880 that it's more responsive to your needs or your  specific context. And to put this into an example, 237 00:22:58,880 --> 00:23:05,360 my specific context, for example, may be  I teach an economics course to first grade 238 00:23:05,360 --> 00:23:11,920 university students. But the reality is that  I don't use ChatGPT only for my classes. I 239 00:23:11,920 --> 00:23:19,600 may use ChatGPT also for other uses. And so  what openAI has rolled out in this past year 240 00:23:19,600 --> 00:23:27,280 is the possibility to customize the responses  of ChatGPT at different levels. The first level 241 00:23:27,280 --> 00:23:35,360 is to have instructions that apply to any chat  that you open in ChatGPT, and that's what system 242 00:23:35,360 --> 00:23:41,120 instructions do in chat GPT. For example,  you can keep this to a very general level, 243 00:23:41,120 --> 00:23:48,880 like, “I'm a woman, I'm 30 years old. I do  this as a profession. I'm from this country, 244 00:23:48,880 --> 00:23:57,040 etc, etc” things that may be relevant. For cases  where you may use ChatGPT frequently, but it 245 00:23:57,040 --> 00:24:02,560 doesn't apply to any use of ChatGPT that you do,  you may want to choose projects instead, where 246 00:24:02,560 --> 00:24:08,080 you may have a project for a specific course that  you teach. You may have a project for something 247 00:24:08,080 --> 00:24:12,720 that you do frequently with ChatGPT. Perhaps you  were talking about travel planning before, like, 248 00:24:12,720 --> 00:24:16,800 perhaps you travel plan a lot, and you want to  have a project specifically where you can provide 249 00:24:16,800 --> 00:24:20,720 instructions around the kind of travel that you  like to do and the kind of places that you want 250 00:24:20,720 --> 00:24:25,200 to go to and the kind of things that you want to  do when you go on traveling, and so that's what 251 00:24:25,200 --> 00:24:30,320 projects allows you to do. It allows you to have  instructions that you don't need to be repeating 252 00:24:30,320 --> 00:24:36,560 every time you open up a new chat, but then  don't apply to all of your chats within ChatGPT. 253 00:24:36,560 --> 00:24:42,160 The only thing I would add to what Angela just  said is that not only can you add instructions 254 00:24:42,160 --> 00:24:47,920 which are very helpful in customizing, but  you can add files, and those files can be 255 00:24:47,920 --> 00:24:53,040 incredibly helpful. So, for example, for  each of my courses right now, I have not 256 00:24:53,040 --> 00:25:00,240 only instructions about teaching philosophy,  goals of the course and so on, but also the 257 00:25:00,240 --> 00:25:06,800 syllabus and relevant material that I think has  helped, as Angela said, the advice that you get 258 00:25:06,800 --> 00:25:15,120 from ChatGPT be much more customizable. Back  in the old days, and old days here means 2023, 259 00:25:15,120 --> 00:25:22,080 when you were writing one chat and then you would  begin another chat, it was like two completely 260 00:25:22,080 --> 00:25:28,560 independent conversations, and now I think AI  is much better at understanding through not just 261 00:25:28,560 --> 00:25:32,880 the customizations that we just talked about  now, but through the use of memory and so on, 262 00:25:32,880 --> 00:25:41,360 it's much better understanding who you are as a  user. I think one aspect of the progress in LLMs, 263 00:25:41,360 --> 00:25:46,160 ChatGPT in particular, but I think it's true  for all of them that we have picked up in the 264 00:25:46,160 --> 00:25:52,560 second edition of the book, is increased  personalization and customization that is 265 00:25:52,560 --> 00:26:01,600 now available to a user to be able to get more  customized and specific advice from their LLM. 266 00:26:01,600 --> 00:26:05,680 And the last thing I want to say is that  everything that Angela said is very, very, 267 00:26:05,680 --> 00:26:12,000 very, very easy to do. You don't need to  code. You don't need anything. If you know 268 00:26:12,000 --> 00:26:19,040 how to prompt ChatGPT, you can create a project,  and it's as simple as you put in instructions, 269 00:26:19,040 --> 00:26:24,080 you put in files, and then you create the chats  within that project, that's all you need to do. 270 00:26:24,080 --> 00:26:28,480 You've hinted at some ways of  taking advantage of projects. 271 00:26:28,480 --> 00:26:33,680 Can you give one or two specific examples of  where this could benefit student learning? 272 00:26:33,680 --> 00:26:40,720 So I think the easiest way Angela mentioned  is you can create a project for each of your 273 00:26:40,720 --> 00:26:48,560 courses. Once you do that, then a lot of the  help that you asked ChatGPT would be useful 274 00:26:48,560 --> 00:26:55,920 for creating activities for students, helping you  design assessments and so on. So that's one use of 275 00:26:55,920 --> 00:27:02,240 projects. If you want to create something that  students consume directly, then you would move 276 00:27:02,240 --> 00:27:09,920 from projects to custom GPTs. Custom GPTs can  be shared with students and they can interact 277 00:27:09,920 --> 00:27:16,560 with it. In the first iteration of custom GPTs  that were most popular were the tutor bots, 278 00:27:16,560 --> 00:27:26,000 so where you might ask students to interact with  a tutor bot, for example, to learn some skills in 279 00:27:26,000 --> 00:27:31,040 whatever subject that it is that you teach, but  where the instructor was playing kind of a design 280 00:27:31,040 --> 00:27:38,560 role, was adding materials and so on. I think  those tutor bots continue to be helpful. One 281 00:27:38,560 --> 00:27:43,840 of the things I think, I suspect many of us have  discovered is that if the tutor bot is not part of 282 00:27:43,840 --> 00:27:51,200 the natural workflow of students, then it becomes  a little bit harder for them to benefit from it. 283 00:27:51,200 --> 00:27:57,040 So if students are using ChatGPT as their AI tool  or any other AI tool, it's going to be hard to 284 00:27:57,040 --> 00:28:04,720 move them to the custom GPT or to the tutor bot,  unless you sort of have very directed task at them 285 00:28:04,720 --> 00:28:10,480 doing that. But there are many, many other ways  in which you can use custom GPTs with students. 286 00:28:10,480 --> 00:28:16,400 And before we were talking about giving students  an opportunity to practice, but imagine you were 287 00:28:16,400 --> 00:28:22,880 teaching negotiation, for example, you could  imagine, and there are some available out there, 288 00:28:22,880 --> 00:28:30,800 bots that allow students to practice negotiations  or public presentations or so many other skills. 289 00:28:30,800 --> 00:28:38,480 So if you teach any skill that you think benefits  from repeated practice and feedback on that 290 00:28:38,480 --> 00:28:45,760 practice, I think the potential of AI to help  you and your students in that task is immense, 291 00:28:45,760 --> 00:28:51,680 and custom GPTs are a way to make this  happen. And again, the book is called 292 00:28:51,680 --> 00:28:58,320 Teaching Effectively with ChatGPT, so we put  our stake in the ground with selecting one LLM, 293 00:28:58,320 --> 00:29:05,040 partly because it was most popular sort of  tool. But if you're using some other LLM, 294 00:29:05,040 --> 00:29:11,680 like Claude or Gemini, the equivalence of  everything we have talked about exists. So Claude 295 00:29:11,680 --> 00:29:20,160 has projects. You can build custom GPTs in Gemini,  through what they're calling GEMs and so on. So I 296 00:29:20,160 --> 00:29:25,200 think a lot of these companies are all building  and competing with each other in this space. 297 00:29:25,200 --> 00:29:30,320 And in addition to providing those tutor bots,  you can also design them to help you with course 298 00:29:30,320 --> 00:29:35,440 design, things that will generate things in the  TILT framework, for example, or make suggestions 299 00:29:35,440 --> 00:29:40,160 for adding more active learning activities. Pretty  much any of the things that you talk about in your 300 00:29:40,160 --> 00:29:46,000 book, if it's going to be done repeatedly,  could be turned into a custom GPT. And your 301 00:29:46,000 --> 00:29:50,800 book was really good at providing them, that was  a really popular chapter with the reading group. 302 00:29:50,800 --> 00:29:54,960 Thank you. I just want to say one thing.  So since the first edition of the book, 303 00:29:54,960 --> 00:30:02,320 projects became a thing, and I would say, I  think building custom GPTs is relatively easy, 304 00:30:02,320 --> 00:30:09,760 but if you're only going to use that tool for  yourself, then building projects is an even easier 305 00:30:09,760 --> 00:30:14,960 way to accomplish the goal. Obviously, custom  GPTs have the advantage that they're shareable, 306 00:30:14,960 --> 00:30:19,280 so you can share not just with students,  but with colleagues and so on. But if 307 00:30:19,280 --> 00:30:24,320 you're getting started and are feeling somewhat  intimidated with how do I do all this stuff, 308 00:30:24,320 --> 00:30:31,840 projects is a very, very natural step  up from your ad hoc use of ChatGPT. 309 00:30:31,840 --> 00:30:38,720 There's still a lot of faculty who maybe would  prefer to ban student use of AI or focus their 310 00:30:38,720 --> 00:30:47,200 efforts on trying to detect student AI use. Are  there any ways of reliably detecting AI use? 311 00:30:47,200 --> 00:30:54,000 So I'm not a computer scientist, but the ones  I speak with give me pause that we will ever be 312 00:30:54,000 --> 00:31:03,760 able to really do this at a level of reliability  that suggests that we can take action on this. My 313 00:31:03,760 --> 00:31:11,440 general sense is that this is a lost battle in  the battle of an instructor against students. 314 00:31:11,440 --> 00:31:17,280 I think students always win, is my sense.  If this is the battle that we're fielding, 315 00:31:17,280 --> 00:31:23,040 we're not going to win. I think we have to  do differently. But just to give you a sense 316 00:31:23,040 --> 00:31:29,920 of for why I think it's so hard, is that now  AI is so embedded in everything that we do, 317 00:31:29,920 --> 00:31:36,400 that I think it's going to be very hard. Back in  the old days, again, 2023, you could ask students, 318 00:31:36,400 --> 00:31:42,160 okay, go to ChatGPT and then have a conversation  and then give me the transcript. But now it's 319 00:31:42,160 --> 00:31:47,280 embedded in the things that we do. It's auto  complete things in Word and so on. And more 320 00:31:47,280 --> 00:31:55,840 importantly is that you could, as a student or  as a human being, produce an essay by yourself. 321 00:31:55,840 --> 00:32:03,280 You created all these ideas, and then you give it  to ChatGPT and say, “Please improve the clarity 322 00:32:03,280 --> 00:32:11,040 of my writing.” That's all you're asking it.  It will add a few em-dashes and a few words, 323 00:32:11,040 --> 00:32:18,400 and then everyone will say, “this was written by  ChatGPT.” And you might decide, as an instructor, 324 00:32:18,400 --> 00:32:23,920 particularly if you teach writing, that even the  use that I just described should not be allowed. 325 00:32:23,920 --> 00:32:29,040 But I'm going to suggest that at least for some  subjects, that use is fine. And if that use is 326 00:32:29,040 --> 00:32:36,800 fine, an AI detector is going to say that this  was created by AI. I think we're in trouble. 327 00:32:36,800 --> 00:32:38,960 Since the first edition of your book, 328 00:32:38,960 --> 00:32:45,360 openAI has released ChatGPT versions 4 and  5 more recently. You mentioned projects, 329 00:32:45,360 --> 00:32:50,400 but what are some of the other changes  that have been added to ChatGPT since then? 330 00:32:50,400 --> 00:32:56,160 Great question. So projects, as you very well  mentioned, is one of the main features that 331 00:32:56,160 --> 00:33:01,760 was released. There's another very interesting  one, which has to do with the models that are 332 00:33:01,760 --> 00:33:10,160 available within ChatGPT. So before ChatGPT 5,  there were multiple models available, as most 333 00:33:10,160 --> 00:33:17,680 people in the audience would know, where there was  a basic default model, which was valuable for most 334 00:33:17,680 --> 00:33:24,800 of the tasks. And then there was a deep research  model that is valuable for in-depth, multi-step, 335 00:33:24,800 --> 00:33:32,880 more complex tasks that require time to think and  process before being able to generate an output. 336 00:33:32,880 --> 00:33:39,520 So this can be useful, for example, to summarize  academic research, to create classes or provide 337 00:33:39,520 --> 00:33:46,400 feedback on longer documents, anything that is  more complex than the average task. Now, what used 338 00:33:46,400 --> 00:33:52,640 to happen is that experienced users knew about  this model, and they would change the model that 339 00:33:52,640 --> 00:33:59,360 they wanted depending on the task that they were  wanting to do, but non-experienced users would 340 00:33:59,360 --> 00:34:06,320 just default to the basic model and would not be  using the most out of the capabilities of ChatGPT 341 00:34:06,320 --> 00:34:13,040 and so with the release of ChatGPT 5, one of the  key developments was that ChatGPT 5 automatically 342 00:34:13,040 --> 00:34:20,560 selects the model that best fits the task at hand.  And so this is a great improvement in terms of 343 00:34:20,560 --> 00:34:27,440 being able to really put at the service of users  all of the capabilities that ChatGPT has. One of 344 00:34:27,440 --> 00:34:32,480 the important things is that when it was first  released, there was a big controversy, because 345 00:34:32,480 --> 00:34:39,920 some people claimed or complained about the fact  that ChatGPT was not able to always optimize for 346 00:34:39,920 --> 00:34:45,920 the best model, and they wanted to retain that  agency. And since then, openAI has backtracked and 347 00:34:45,920 --> 00:34:51,280 still provides everybody the ability to be able to  choose the specific model that they want to use. 348 00:34:51,280 --> 00:34:59,120 We've already alluded to this, but to me, a big  development in the last year or so is the ability 349 00:34:59,120 --> 00:35:04,320 to customize and personalize your experience, not  just through projects, through memory. and other 350 00:35:04,320 --> 00:35:11,680 things that we go into in the book. In fact, the  chapter that we used to call Creating Custom GPTs, 351 00:35:11,680 --> 00:35:18,320 we've now broadened the title of that chapter  to include many other ways in which you could 352 00:35:18,320 --> 00:35:26,080 customize and personalize your experience and that  of your students through the use of this feature. 353 00:35:26,080 --> 00:35:33,040 So that, to me, feels like obviously ChatGPT  5 is better than ChatGPT 4, but back then, 354 00:35:33,040 --> 00:35:38,000 now there are these reasoning models that are  automatically selected and so on, is useful. 355 00:35:38,000 --> 00:35:45,120 But the thing that you will notice more as a user,  I think, is the potential for customization. And 356 00:35:45,120 --> 00:35:51,360 obviously the reasoning models can be very, very  helpful for more complex tasks, deep research, 357 00:35:51,360 --> 00:35:59,120 which Angela briefly mentioned, is another  important advancement for much more complex tasks. 358 00:35:59,120 --> 00:36:03,600 As we've noted in this conversation  and also in your book, generative AI 359 00:36:03,600 --> 00:36:08,560 platforms change quickly. And when we  talked to you in the first edition, 360 00:36:08,560 --> 00:36:12,000 you mentioned having a companion  website that you would continue to 361 00:36:12,000 --> 00:36:16,080 update. Are these updates something that  are going to continue with this edition? 362 00:36:16,080 --> 00:36:19,440 Absolutely, that's one of the main  advantages of the companion side, 363 00:36:19,440 --> 00:36:24,160 the ability to be able to update it. As  we know this is a fast evolving space, 364 00:36:24,160 --> 00:36:29,520 and that we felt that was a good format  to be able to digest information. But one 365 00:36:29,520 --> 00:36:35,360 of the main inconveniences is that it's not  life, and so users and readers and listeners 366 00:36:35,360 --> 00:36:39,360 can always go back to the companion  side for more up-to-date information. 367 00:36:39,360 --> 00:36:45,280 And the companion side has all the prompts  that were used in the book, including, 368 00:36:45,280 --> 00:36:51,200 for those educators that shared with us their  prompts behind their custom GPTs, they're also 369 00:36:51,200 --> 00:36:58,800 there. The site is freely available, and then it  also has a lot of the examples from educators. 370 00:36:58,800 --> 00:37:01,440 And we'll make sure we have  that link in our show notes. 371 00:37:01,440 --> 00:37:05,840 And people found it really valuable when we're  doing the reading group to have all those examples 372 00:37:05,840 --> 00:37:12,960 there so that they could modify them for their  own needs. We always end by asking: “What's next?” 373 00:37:12,960 --> 00:37:21,520 Well, I want to say something that's next that  is, thanks to Angela. So I think we are maybe one 374 00:37:21,520 --> 00:37:28,080 or two weeks away. Hopefully by the time you  publish this show, it will be out. We have a 375 00:37:28,080 --> 00:37:34,480 edition of the book in Spanish that Angela gets  all the credit for making it happen. So we hope 376 00:37:34,480 --> 00:37:40,400 that means that more readers around the world  will be able to use some of the insights from 377 00:37:40,400 --> 00:37:44,640 the book. If you're a listener and want the  book to be available in some other language, 378 00:37:44,640 --> 00:37:49,680 let us know, and we'll try to see if we can  maybe collaborate with you and make it happen. 379 00:37:49,680 --> 00:37:51,920 And Angela, what's next for you? 380 00:37:51,920 --> 00:37:56,880 Getting on a plane, flying back home for  the weekend, and getting that book ready. 381 00:37:56,880 --> 00:37:59,840 Well, thank you. It's been  great talking to both of you. 382 00:37:59,840 --> 00:38:03,200 Thank you so much for having us. It's  always a pleasure to come and have tea 383 00:38:03,200 --> 00:38:07,440 with both of you and to talk a bit about  AI developments in teaching and learning. 384 00:38:07,440 --> 00:38:12,800 So always a pleasure. And thank you so much.  Hopefully we'll see you again sometime soon. 385 00:38:12,800 --> 00:38:14,000 Safe travels. 386 00:38:14,000 --> 00:38:15,520 Thank you so much. 387 00:38:15,520 --> 00:38:18,640 Thank you. 388 00:38:22,320 --> 00:38:26,720 If you've enjoyed this podcast, please  subscribe and leave a review on Apple 389 00:38:26,720 --> 00:38:31,680 Podcasts or your favorite podcast  service. To continue the conversation, 390 00:38:31,680 --> 00:38:37,920 join us on our Tea for Teaching Facebook page. You can find show notes, transcripts and other 391 00:38:37,920 --> 00:38:44,400 materials on teaforteaching.com.  Music by Michael Gary Brewer. 392 00:38:44,400 --> 00:38:47,920 Editing Assistance provided by Madison Lee.