Grades are clearly emotional to students, which might partially explain why they make students so anxious and, simultaneously, so obsessed with them. But this post is also not about this.
This post is about the less-told side of what grading means to the instructor and, in particular, to an instructor that has created the evaluation.
An important start point is that setting up evaluations is hard. My students seem shocked when I tell them that it takes me between one and two full days or work to write a midterm. This is one of these things that do look very different from the other side. The students are probably thinking: well, this guy has 10-15 years of education and knows it all about the topic, so this job is mostly about putting things on paper and formatting. Well, no: as instructors we have to work hard to make sure that any evaluation aligns with the learning objectives (and really tests them), that is not easy to subvert (makes cheating harder than not cheating), that it is clear an unambiguous, that it is not too difficult (because this means too much work later on, and unhappy students), that it is not too easy (since it would remove the student’s incentive to learn), that can be marked fairly (this is trickier than it sounds), and that it does not make my life miserable (i.e., that it does not take forever to mark). In other words, setting up evaluations is a complex multi-objective optimization problem that also requires lots of creativity. And this is all assuming that the midterm is in person and not take home, which adds the risk of being chatGPTized…
Then, once you think you have a good midterm/final exam etc., (or you run out of time), it’s time for the actual test. When this happens in person (i.e., in a large room) it is not unusual to see students leave the exam about 25 minutes in (for a 50 minute exam). Doubts start to creep in… was the midterm too easy? Too difficult? Did I implicitly provide answers to the questions in the class? — time passes and then you have the opposite set of doubts… why are there still so many people who seem to be in the middle of it? Was it too difficult? will I be ruining student’s life’s because of my zeal to make my questions climb up Bloom’s taxonomy little pyramid? In my experience this is usually OK, just the expression of the amazing variability of human timing, behaviour and skill (note: I recognize the issues with timed examinations and how excluding they can be, but this is a discussion for another day).
Now you got your filled exams, and it is time to grade. This is the most nerve-wracking part of the process. It goes like this:
The first one is terrible. It is like they did not understand the question, or they attended a set of lectures of a different course in a different department. It’s very difficult to keep the inner doubts at bay and remain calm and objective; no matter how many times you have read and re-read the questions, you read them again… is there a possibility that this question could reasonably be interpreted in that other strange way? Am I just a gatekeeper of a language and a way to look at the material that is personal, biased and unfair?
The second one is terrible again, but in completely different ways. Were my lectures awful? Was it impossible to get to a reasonable solution with the material, help and time that I have provided during the class? Your mind starts coming up with possible ways to grade more generously… is is fair to give some points just for writing anything in the box? What happens if everyone fails the midterm?
The third one is quite good. Sigh of relief! At least someone gets something!
The fourth one is perfect… How nice! But… did I really teach them anything? Is it possible that they did already know the stuff?
The fifth one is perfect again… was the exam too easy? Did I make it too easy to cheat? how is this possible?
And, of course, after a while you tend to get the full range of answers, with some mistakes more common than others. If you are not grading blind to the name of the student (I prefer blind, although it can be logistically harder than it seems), some answers do seem like a disappointment, almost a betrayal. And, occasionally, you can feel a bit of pride on what you have done.
I expect that the experience of the instructor may vary significantly, especially depending on personality. However, the roller-coaster that I’ve described above is still, for me, and after many years of setting up evaluations and grading, pretty much unchanged.
In summary, we try to be neutral and fair, and leave emotion aside, but grading is hard work, not only because it is tedious work that simultaneously requires a large mental load to be consistent, but because it is emotional work.
The only things that I found that soften the experience a bit for me:
How does grading feel to you as a student or an instructor/grader? What are your tricks?
]]>Here is a teaser of the talk (30 seconds):
Here is a fuller, 15 min presentation ():
Here are two links to the paper:
https://doi.org/10.1145/3528223.3530111
Local copy
Here is a website with resources for using it:
https://nacenta.com/infotypography
For those of you who are in Vancouver attending SIGGRAPH, the presentation is:
When: Thursday August 11, 2022
Where: East Building, Ballroom C
Session: Perception
The work takes place at the intersection of Human-Computer Interaction and Constraint Programming.
We analyzed how non-experts tried to solve constrained programming problems.

This is work with Ruth Hoffmann, Xu Zhu, and Özgür Akgün, from the School of Computer Science at the University of St Andrews (my previous main institution).
This follows our previous work on how people visually represent discrete contraint problems (at the IEEE Transations on Visualization and Computer Graphics).
Dr Hoffmann has also presented at ModRef a synthesis on both pieces of work.
Finally, the SACHI group posted a blog post with a bit more information.
]]>For a very long time I have stored my data in an external drive. Why? Mostly because I do not want the main output from my employment to be locked in a single computer. Often it is very convenient to be able to access the data from another computer. Also, when there is a fire alarm, my drive is the only thing I take. I know that the cloud is a good alternative, but I like to have the data locally and there is something unsettling about trusting Google or Dropbox with your entire digital life and work.
Of course, external drives fail, so I have purchased a subscription to a service which backs up my data on-line, away from my physical location, for over 12 years now.
One day I arrive home, set up my USB hard drive on my computer, and the drive does not start. Or rather, it tries to partially start, but fails in a sequence of odd ways.
No biggie… I should be able to recover all files. It is going to take quite a while (it’s close to 1TB), but I should be covered!
Well, after wiping off the sweat from my brow, and taking a glass of water, I set out to the unpleasant and boring (but necessary) task of recovering my data. The most urgent staff first in my computer’s fixed hard drive first, and then the rest on a new external drive that I have ordered. Everything should be in my LiveDrive, right?
Well, the first unpleasant surprise is that LiveDrive downloads my files really really really slow. A full day of data recovery does not even start to make a dent in the 0.8 GB of data (approx) that I had backed up. I’m in a good connection, but maybe it is that the data is stored in Europe (where I used to live), and has to go under the water to get to me. I try a few things to solve this, including a VPN, but it turns out that this is not the problem. The service is extremely slow seemingly because it goes file by file, and it checks the status of each file individually, blocking the download until one file has been done. The download is decent when the file is large, but when you have a folder full of small files (e.g., Thunderbird’s e-mail folders, it slows down to a crawl. Unfortunately, the livedrive interface only lets you select folders, or files, recursively, but in one folder at a time. Additionally, every time you select what to download and it finishes, you have to select again from the backup set starting at the root folder. This goes on for a few days, with much effort and desperation, since my whole research is in there. It is hard and really tedious work of clicking buttons and check marks, taking notes of what is backed up and what is not, what folders should be priority, and trying to avoid those folders with lots of small files. I’m able to recover about 30% of my files this way, but with much effort.
The second unpleasant surprise is that some (important) folders simply do not download. This problem seems to be independent of the issues mentioned above. After some back and forth with LiveDrive’s help desk I am explained that some of the data stored is corrupted and therefore not downloadable. Their technical team are working on it, and they will update me of the progress. After some back and forth with them reminding them regularly for about 3 months I simply desist. After a year and a half I’m still to hear from them.
Contact with LiveDrive’s help desk results in reasonable response times, but not typically very useful. Comments about the problems of the interface are put in feature requests for the (presumably distant) future, while comments about the slow down seem to simply be above the pay grade of the helpdesk staff. They are generally pleasant. After they recognize that some of the data is corrupted, they offer to refund me what I have paid them so far “as a courtesy”.
More than a year and a half after the incident, I still have not completely recovered. My skin still crawls when I think back about it, and sometimes thinking back is unavoidable. A silver lining: I have learned to let go of some data, and perhaps not being so reliant on storing things in drives (although I still like to have most of my stuff in my own drive).
I also think that LiveDrive giving back my payments is like an insurance company giving you back your premiums if your house goes on fire. I have no animosity against the people in the helpdesk who helped me. They are probably underpaid, and it feels like the rest of the company is just in “automatic pilot mode”, without really any interest in improving the service, but more in milking their existing infrastructure and software (which is clearly not fit for purpose).
If your data is not backed up, please please please please, back it up somehow now. At least the most important stuff should not be difficult to put on dropbox, google drive, etc.
If your data is backed up in a service, set aside some time to give it a serious test. Consider the following questions: how long would you consider acceptable to recover your data? How much work would you want to do for what you are investing now?
]]>Adam built a new tool to support people when transforming complex networks of thoughts and ideas into text prose. The tool itself (called Write Reason) and a short description of what we found when we studied essay writers using the tool is accessible through the WriteReason site.
One of the most interesting things about our findings is that there is a lot of interesting stuff going on in between the creation of different representations of the ideas. In other words, transformations between representations are much more complex, interesting and important than we thought before.
This post is also in the VIXI website.
]]>Just send me a short e-mail to [email protected] and we will explore!
]]>Audience: you are a PhD or MSc or project student, and you meet your supervisor regularly. In the previous meeting, your supervisor asked you to do research in some topic area, perhaps by a keyword (e.g., please do some search in the area of “provenance in User Interfaces”), or perhaps through another paper (e.g., follow the references in this other paper).
Naturally, you have done the work, and you have read (there will be another post on what this means in the future) a bunch of papers. Now you come to your supervisor, eager to demonstrate that you have done what you were expected to do, and indeed, you have found and read some interesting papers.
Perhaps more important than the “do’s”, consider the things that you should avoid. Unless your supervisor has explicitly asked for any of these, you should NOT:
These are all items that you might not know are important, but are often key for good communication about research.
Background and context: Not all papers are created equal or are going to have the same importance. If you only list the title (and yes, the title is important), you are missing many of the ways in which your supervisor might help you filter out the really relevant papers from those which are not. More specifically, this is information that is likely to be useful:
Note that giving some kind of impression of the above should take very little time. You do not have to list or read aloud the whole list of authors, but having it out there so that your supervisor can take a glance at it can be very useful.
Content: An article or piece of scientific literature typically contains much information, more than you can efficiently convey within a meeting. If you have to summarize any part of the work, because you think it is relevant, this does not mean necessarily that this is going to be a piece of text. There are often different kinds of elements that might be way more effective at summarizing a paper. For example:
Note that any of the elements above might be incredibly difficult to understand in isolation, but this is why you are having a meeting: a quick few sentences can help make any of these items really effective.
Relationship: Probably the most important reason why your supervisor wants you to read a piece of scientific literature is because it relates to your work. Very often this relationship remains implicit. You should be able to describe as succinctly and accurately as possible how what you have read relates to one or more of the following (depending on the stage of your research):
By making explicit the relationship between your work and what you have read, you are also communicating a number of things that are important for the supervisory relationship and signalling your level of maturity in research. For example, your supervisor will learn to what extent you are able to relate your work to the close and fairly distant (perhaps only abstractly related) pieces of literature, and whether you really understand the contributions of existing work. This is a kind of syntopical reading or deep reading that is very important for academic work.
Please let me know if you have found this article useful, or if you have concrete examples, or possible improvements that you think I can use to improve it.
]]>Where: University of Victoria, Victoria, British Columbia, Canada
What: A Post-doctoral Fellowship, 1 to 2 years in length
When: Applications being evaluated now, until position filled. Start is negotiable.
The candidate will work with current research students, with the main supervisor (Dr Miguel Nacenta) on common projects, and will also have time to further their own research. Current research interests of Dr Nacenta include:
You can access recent published work by Dr Nacenta through his Google Scholar Profile
If you are interested in becoming our partner in learning and research, send an e-mail to nacenta(at)uvic(dot)ca with the following information:
One to two year employment as a full member of our lab and the Department of Computer Science of the University of Victoria; Salary starting at CAD$50K, but negotiable based on merit and/or experience of the candidate; Opportunities to contribute to teaching (only if so desired); A supportive community of scholars and students.
The Victoria Interactive eXperiences with Information (VIXI) group is a newly formed research group in the areas of Human-Computer Interaction, Visualization and Cognition. With three faculty (Sowmya Somanath, Charles Perin, Miguel Nacenta), and over 16 students at different levels, we aspire to be a diverse learning community that supports the growth of each member while also contributing to research at an international level of excellence and impact.

Victoria is the Capital of British Columbia, the westernmost province of Canada. Surrounded by water, at the southern tip of Vancouver Island, is part of a vibrant urban area of close to 400,000 inhabitants, yet surrounded by amazing nature and parks. The city has an extensive network of cycling paths (for commuting or leisure), great access to outdoor activities (kayaking, scuba diving, mountain biking, hiking, skiing – further north in Vancouver Island or across the Salish strait), and good connections through its International Airport and ferries to the mainland and the United States. The city is extremely child-friendly and has probably the mildest climate in Canada, with an average temperature in January of 5 degrees Celsius and mild summers. Average precipitation is about half that of nearby Vancouver (107km away as the crow flies) due to the rain shadow effect of the Olympic Mountains to its South.
We acknowledge and respect the lək̓ʷəŋən peoples on whose traditional territory the University of Victoria stands and the Songhees, Esquimalt and W̱SÁNEĆ peoples whose historical relationships with the land continue to this day.


For administrative details, conditions and an initial description of the research see: the official Scholarship Announcement.
Supervision is collaborative between Dr Juan Ye and Uta Hinrichs (St Andrews) and myself (Miguel Nacenta–University of Victoria). Applications are accepted until June 30th, 2020. Feel free to drop me a line at nacenta at uvic dot ca if you want to discuss your application informally with me.
]]>
Remote PGR supervision has its challenges, yet I believe it is important to acknowledge that the new constraints imposed by the virus, quarantines, social distancing etc. add an additional layer of complexity to the whole situation. I also believe that postgraduate students are likely to suffer the extraordinary situation particularly hard because with the additional burden to move to online teaching, supervisor’s energy and time will be in shorter supply. Here are just a few of my thoughts regarding remote supervision (first), and regarding supervising during quarantine (second).
I think it is important to acknowledge that remote supervision is harder than in-person supervision. For a number of reasons:
Everything in the previous section applies to the current quarantine situation, sometimes amplified. And some aspects are new:
I have never been a perfect supervisor, but I try hard to do right by my students. Despite knowing that I can get much better at this, I believe that there are some obvious and not so obvious things that can work:
Some of you might have different tips and ideas. I’m sure I have missed a lot here, and I can learn a lot more. Even better, some of you might disagree with me. In any of these cases, I’ll be very happy if you let me know through a comment or by e-mail.
]]>