What’s in It for Companies? Talking Assessment and Tech

This week’s blog prompt for EC&I 833¬†from¬†Tyson, Jen, Nathalieand Nicole is a great one on assessment but… not too doable for me since I’m not teaching a class myself. So, instead, I got some inspiration from Launel who mentioned that she was tempted to talk about the politics of assessment but instead talked about assessment strategies she finds to be “underused and undervalued.” I agree both that the politics are incredibly important and that many great assessment strategies are underused and undervalued.

flickr photo shared by William M Ferriter under a Creative Commons ( BY-NC ) license

Note that I said assessment strategies. When I work with an instructor to help them put together a class, I am much more interested in what they want to assess than in what tool they think they want to use. I can always come up with a variety of ways of assessing something but I want to know what they are getting at. What value does it have for students? Does it have any value for the instructor? What’s the context (is there one? has the instructor even thought about a context for the assessment?)? But sometimes it is easy to get caught up in tools and talking about the tool itself, choosing for reasons other than student learning. It’s cool, it’s fun, students love it. All reasons instructors adopt tools. It makes marking easier, it makes communication easier. Great. But Alec raised some concerns with Class Dojo last week in terms of it influencing the teaching and the uses rather than teachers making more conscious decisions.

flickr photo shared by PS3 Attitude under a Creative Commons ( BY-NC ) license

Tech is not neutral. Someone designed it. Someone programmed it. Someone made assumptions about how you would and should use it. We are not talking about the Force which just is and only the use determines if it is good or bad. Sometimes the tech itself is problematic. And sometimes the tech itself is amazing. Sometimes it can be shifted to good through effort. Sometimes it can be used poorly and be detrimental to students. I know I’ve used things poorly (hi, PowerPoint!). Anyone got a great example of redirecting a tool that might be a bit questionable?

What muddies all this up is the fact that most of our assessment tech tools are created/coded/written/sold/designed by a company somewhere. Sometimes it is pretty obvious what the company gets out of it: Money. I’ll be coming back to these types of products because it isn’t always that straightforward.

flickr photo shared by 401(K) 2013 under a Creative Commons ( BY-SA ) license

Sometimes we can use them for free. I’m cynical and this sends up some red flags for me that get me investigating. And sometimes, things really are free and there are no apparent ulterior motives. Other times the company is getting something else out of student use: data.

That’s right. “Data is the new oil.” The video was about AI but the same goes for learning analytics which, to a certain extent, are sort of in the same field. The intent is to learn how students learn, to build a better guide to learning, to help the machine think like a teacher who can guide students along the learning path.

As the video says, nothing is for free and if you get a service for free in exchange for your data, your data is valuable. And even if you paid, you might still be giving your data. Well, so what? So you’re giving your data? Wait… no, you’re giving your students’ data. And you may have access to some of it in the form of reports and logs and analytics. Okay, cool! You (the teacher) can track your student. We have been conditioned to think this is the way it should be. But what about students having access? Audrey Watters has written a whole lot about student data: Student Data, Algorithms, Ideology, and Identity-less-ness, Student Data, Algorithms, Ideology, and Context-less-ness, Student Data and Privacy for just a few related ones). Just what are these companies learning from our students? And are our students learning from their own data?

I very much think that students should know what we are tracking and be able to see it. It is, after all, their data. It is their work and they didn’t sign a release saying that because they are our students, we own their work (unlike some contracts signed by teachers, content experts, developers, etc, but that’s another matter). But since they haven’t signed it, isn’t it theirs?

It’s something to think about every time we use tech for assessment. It can be fantastic and can make life easier, but sometimes there are other costs to think about.

I think I’m over papers…

It’s a funny thing to think about. I switched my major in my first year as an undergraduate because I realized I enjoyed writing papers and was good at it (as opposed to being a Visual Arts major which burned me out in 4 months). So much of my academic career has been about writing papers. I’m good at it and, to some extent, I enjoy it.

I think I’m over it, though, when it comes to teaching. Yes, I still feel that students need to know how to write. I also think that knowing how to research is important. For some students, I think that learning to write research papers will be highly important. Those students, however, are the ones who will move on in academia or are in fields where they will be required to write such items (yes, all English majors should know how to write in a variety of genres, essays being one of them).

My biggest frustration is the idea of writing a paper that only one or two other people will read. That is not the intent of academic papers. Actually, that is the opposite of what academics hope to have happen with their papers. We are teaching students to write for an audience of one and yet that is not the goal when we ourselves publish papers. I want to share. I want to know what others are working on. If I feel this way, then it seems likely that my students might also feel that way.

I find myself wanting to pull together a variety of materials. In researching for a current paper, I have already had to exclude white papers and blog posts because they are not peer reviewed. I just saw a tweet that would be an excellent demonstration and yet that is also not an appropriate source. This seems at total odds with my actual practice. It seems at odds with the practice of other academics. Why am I being forced into more traditional expectations when it no longer matches the reality of academic writing? Yes, we have a presentation (which I will share) which our class will see and we also have to share our annotated bibliography. Our paper, however, is for an audience of one or two.

Considering I come from an academic background, I used to live and breathe papers, and I have been whining lately about having to write such short papers, one would think I would hold fast to the idea of producing papers.

Nope. Not anymore.

It just does not make sense. I can have students write in other ways. I can have them present research in other ways. I can let them use other sources and connect with learning in ways that make sense to them. I do not have to be the gatekeeper if I teach them well. Some may choose to write papers but I do not need to force that on them.

Here begins a new era of my pedagogy. I choose to be social. I choose to be constructivist. I choose to embrace change and let technology actually impact my teaching rather than replicating conventional means (Laurillard, p. 15). I also want to consider my students, many of whom will have little use for writing papers.

Laurillard, Diana. (2009). The pedagogical challenges to collaborative technologies. Computer-Supported Collaborative Learning, 4, 5-20. DOI 10.1007/s11412-008-9056-2

My learning should be about me… but how?

I wanted to take a bit of time to think about the concept of personalized learning. In my EC&I 831 course, I would say that things are pretty personal. Our assignments are very open for us to make of this course what we will. Although there are weekly online class sessions, we have a lot of freedom. Much more so than many courses I’ve taken, especially when I think back to -12 and undergraduate work.

I really enjoy this freedom. As an experienced learner, I feel fairly prepared to delve into this type of environment. But what about the students I encounter who have made it clear that they want to be told what is on the exam and who want me to give them my PowerPoint slides so they can just memorize everything and spit it back to me for an A? I am concerned that those students aren’t prepared for this and yet, I want to teach them to learn and give them a way to make any course relevant to them.

So if there is a movement toward personalized learning, what does that really mean? How personalised or individualised can it get?

I do notice that there are different ways of looking at this. Barbara Bray created a fantastic chart to show what individualised vs personalised vs differentiated really means. As she said, however, that isn’t always clear.

cc licensed ( BY NC SA ) flickr photo shared by Ian Guest

I like the idea of personalised learning best, I have to admit, although I can see the benefits of the other two types as described by Bray. As a serial grad student, I am more than happy to learn about what interests me and take ownership of my learning and my path. I have figured a lot of that out and it is assumed that, to some extent, I have sufficient knowledge to determine what I want to do and how to get there.

How do we teach students that, though? How much freedom can we give them while still ensuring that they have the core knowledge that is needed, that all of them need?

This is one of those things I find coming up when discussing instructional design. Instructors often feel that there is a core of knowledge of their area that must be dumped into the students at some level. It may be introductory, it may be further on. So it cannot all be personalized. We do need some exposure to certain things in order to move forward or have conversations with others in our field, at some point. Right? I’m told this is true and, to some extent, I do agree. The components of a properly constructed sentence are going to be the same regardless of your interests. The anatomical terms for the body don’t change based on the specialization you want to have after learning them.

The sheer amount of time personalisation can require is, I know, one of the drawbacks and stumbling blocks may instructors encounter. How do you make that more basic learning personal for everyone while still ensuring that the material is conveyed? Personally I think this is where the role of assignments comes into play, where the basic information is provided but is then implemented by the learner in ways that make sense.

I am trying to think through how this could be done in an introductory Religious Studies course. I can certainly see many ways to give students freedom to investigate ideas that are of interest but… part of the purpose of the course is to provide exposure to different ideas, different concepts and religions. Not all students will be interested. Some may not know they are interested until exposed. So where do I draw the line? Does freedom come from the assignments? Does personalization come from letting everyone choose different religions and concepts? How do I assess learning in that case?

My experience as a learner has been that I had freedom at he university level, within limits. I could choose paper topics, within certain boundaries, sometimes choosing from provided options, sometimes proposing my own. As my courses got to be higher level, I could even select the topic itself and propose it to an instructor as a reading course. Sometimes I chose my own materials or shaped the course entirely. I have even created my own exam questions. But the more personalized it got, the higher level I was at. And the more time it took to meet one-on-one although the work of creating the material or compiling it was most often my own. By then, however, I had experience doing this. I felt comfortable with it.

If anyone has good resources to share on how this can be done well, I would greatly appreciate it. Being in the first semester of my Master of Education after spending years in a different field, I am always looking for suggestions of where to look, people to read on different topics, good examples of design.