Pre-service student teachers: Noticing challenges

On reflecting with Noticing through Noa (2)

During a whole academic year (2023-2024) of my pre-service teachers (PSTs) using Noa for SIRP we regularly discussed their experiences in class. At the end of the academic year, I did a series of interviews with selected PSTs (pseudonyms used) who appeared to reflect with, and hold different attitudes to Noa. In this is the second of two blog posts, I draw on this data to take stock of their experiences of using Noa for reflection with a focus on the challenges they experienced.

The PSTs struggled at the very beginning, when Noticing was initially released. We needed their usage to identify and fix early bugs and iterate on our prompting. Re the latter, the AI appeared too literal in its ‘interpretation’ of our prompting, asking too many questions (sometimes repeating the same question) to elicit as much information as possible; the PSTs found this very frustrating. Antigona felt that Noa

didn't read [her] answers. It just jumped the gun, e.g. But what about this? And [she] would answer.

Due to the very quick rise in popularity of the major AI models at the time our app was released, the sessions sometimes timed out due to traffic. All of these issues were resolved by the end of the academic year.

Time was another consideration. Some PSTs felt the sessions with Noa were time-consuming; others understood that arriving at one’s own realisations (not quick fixes!) can take time, but it’s an investment in professional development. Those, who like Mia, specifically planned to reflect with Noa appeared to have more enjoyable and fruitful experiences. Mia said the following:

Sometimes it (took) 15 minutes, sometimes if it was 30 minutes, but I was aware of that and I didn't find it as challenging as some of my colleagues did.

The PSTs’ mixed initial impressions were also due to the novelty of the medium. As Kira put it:

I felt kind of lost because I didn't know what I [was supposed] to be doing or how I should answer. Should I behave like it's a person, like someone I'm chatting with or just, you know, throw random facts [at it].

While we discussed in class the different ways PSTs addressed Noa, ranging from polite and formal to informal and sometimes even rude, what Noa was to them was an issue every PST needed to ultimately figure out on their own. Antigona, for example, felt like she was the “boss” in their interactions. She considered Noa to be her “assistant”. As she describes it,

someone running after me with a notepad and asking me [more]. […] [There] was a disbalance there. Because I knew I could always shut it off if I wanted to. I can't shut off a human mentor. I can't just stand up and walk away.

On the other hand, Kira was perhaps the most polite user, wishing Noa a good night after a completed session. For her, Noa was a ”friend”:

I would not say a teacher because I was not rated by how my conversations went. Maybe as a friend who can provide me with knowledge that I myself do not have or [am not able to access at the moment].

My PSTs talked about relaxing, with time, into using Noa:

Later, I felt much more comfortable, and I would just throw in some joke, see whether it would react in any way.

Antigona also talked about the Sci-Fi feel of the app to start with, especially in the post-COVID-19 context of attending online university classes, which she considered to be a harmful experience for their generation:

It was a bit daunting at first because, this was a first experience with an AI model […] on a more intelligent level. […] So it was a bit odd for me to rely on an AI model to have these conversations about practical teaching and […] seek advice and sources. […] [Also,] there was Zoom and online classes, and now this AI model, that we have to talk to. […] There were a lot of firsts. There was a starting with Zoom and now with the AI.

The novelty of it all might have presented another challenge: that of how AI models can support learning. Seeing as (1) the Macedonian educational system hasn’t exactly embraced computers in general, and AI in particular, and (2) many educational tasks tend to be perceived by the students as discrete tick-the-box, jump-through-the-hoop, remember-and-repeat-in-the-exam endeavours, it’s not surprising that some PSTs were puzzled by an app which supports their critical thinking and arriving at informed judgements.

Kira summarised her journey of shifting expectations:

I did not see it [Noa] as something that I would get to learn from, [but] as something that I do should just put all the necessary information [into] and just get it over with. […] [Store the information] In order for you [Elena, the teacher educator] to read it, but not for me to learn from it. […] I did not enjoy it at first at all. […] I was, like, it's not going to lead me anywhere. […] I think [it was only in] the 2nd semester when I actually learned how to get the best out of it and to actually enjoy the experience. And by the time [the second semester was] about to finish I was thinking of this [Noa] as something very nice. Like, I'm going to miss it.

Nikita similarly talked about his evolving impressions of working with Noa:

It was a positive character development with [Noa] because, at the end, it became a really great platform.

In part to address the PSTs’ struggles to keep up with a fast-changing world, we have since introduced an introductory chat, so the user and Noa may ‘get to know each other’, making this new experience less threatening. Also, throughout conversations Noa is explicit about its rationale and which development processes it is looking to support, all while acting non-judgementally. This process of getting used to the new medium and, for my PSTs, the new way of learning by arriving at one’s own informed understandings in a safe environment, can take time but be very rewarding. In this sense, Noticing is unlike typical apps designed to get users going as fast as possible, and pays back user investment in the long term.