Artificial Intelligence and Assessment: Guidance for Staff

The rapid developments in the capabilities of Artificial Intelligence (AI) and Large Language Models (LLMs) have been in the news regularly since the release of OpenAI’s ChatGPT model in November 2022.

ChatGPT allows users to interact with the tool in an easy-to-use and conversational manner, and has impressed with the sophistication of its responses, leading to widely reported challenges in the Higher Education sector about the use of the tool in relation to academic misconduct.

Beyond ChatGPT, there are a number of AI paraphrasing, re-writing, and enhancement tools, including Quillbot and Grammarly, that call into question the reliability of some assessments in measuring of students’ ability to meet learning objectives.

However, the widespread use of AI in the lives and workplaces of the future is likely to become as common as built-in spellcheckers are today, for example, Microsoft will soon embed their own AI system as integral part of Microsoft 365. In preparing graduates for future careers, the ethical and critical use of AI tools will therefore be an essential skill, and we must acknowledge this.

On this background, UWL’s Academic Integrity Working Group (AIWG) have  produced the following guide to support staff in embedding Academic Integrity in their practice.

What do we mean?

Academic Integrity

This is about behaving ethically, fairly, honestly, responsibly, and transparently when studying or undertaking and reporting research. This includes but is not limited to:

  • Acknowledging sources (not plagiarising).
  • Originality and authorship (only submit work and results that are your own).
  • Ownership (not uploading your work to non-UWL tools or giving it to other students).

Academic Misconduct

This may refer to submitting work that does not have Academic Integrity: the most common form of academic misconduct is plagiarism. Academic misconduct also includes the use of ghost writers (including AI tools), research misconduct, and exam misconduct, as well as all other kinds of cheating to gain an unfair academic advantage.

Artificial Intelligence

Artificial Intelligence, in this context, refers primarily to natural language processing and Large Language Models (LLMs). These are systems which typically analyse billions of inputs to develop connections between them in ways inspired by the human brain. These models are then often then further refined through human feedback and training.

Authentic Assessment

Methods of assessment that reflect how students will apply the skills and knowledge they are developing in the workplace or other professional settings.

What do we do?

Promote Academic Integrity

As academics, we can promote Academic Integrity by:

  • Talking openly about Academic Integrity.
  • Upholding and acting according to those values.
  • Working in partnership with students to develop and maintain strong ethical practices.
  • Encouraging pride in submitting work that is original.
  • Being transparent about risks to Academic Integrity and their consequences.

In addition, academics should actively support students and each other in developing good critical and ethical skills, including appropriate referencing and citation. To this end, additional support is available from subject librarians, the study support team, and fellow academic experts within the schools and colleges.

Another way of supporting students maintaining good practice, is through the use of formative assessments, which can help students develop the skills and practices needed for summative assessment.

Academics should support students to understand the importance of ownership and authorship, and why they should not share work, neither to non-UWL tools, nor other students.

Academics should also emphasise the risks of submitting work that is not original and ask students to reflect on how unfair practice affects the academic community, other students submitting honestly, and ultimately their future career options.

Establish Expectations

Prior to assessments, academics should:

  • Be clear on what will be considered as a breach of Academic Integrity in each assignment.
  • Use rubrics to communicate what is expected.

Academics and students should make a habit of discussing expectations of Academic Integrity regularly, particularly before summative assessments are due. This includes establishing an understanding of codes of conduct and course requirements.

Where AI is permitted as part of an assessment, academics must be clear on the purpose and limitations of use.

In order to facilitate good practice, academics should ensure that the format and date of submission are clear, and that students have sufficient time to prepare and submit.

Course teams should work together to avoid bunching of assessments to the extent it is possible.

Engage with Authentic Assessments

Authentic Assessment will usually be specific to the context of the university and course. This makes it much more difficult for Essay Mills or AI to provide convincing work.

When preparing assessments academic staff are therefore encouraged to:

  • Create assessments that demonstrate ability to utilise, not just recall knowledge.
  • Reflect on what is the most suitable type of assessment and how it relates to real-world contexts.
  • Create assessments that allow students to demonstrate transferrable behaviours and skills.

In preparing students for future workplaces, this might include clearly defined use of Artificial Intelligence tools alongside a reflection of the performance and utility of any such tools.

Authentic Assessments may also include critical reflections on the process of creating work for assessment.

Recognise the limitations of Artificial Intelligence

The use of AI has clear limitations; understanding and communicating these can help in avoiding Academic Integrity being compromised. For example, currently available AI tools can:

  • Only simulate understanding.
  • ‘Hallucinate’ to create plausible sounding but factually incorrect answers.
  • Inherit bias and viewpoints from the training material.
  • Use information that is not always up to date.

For these reasons, AI cannot be considered a reliable source.

Identify Academic Misconduct

In addition to creating authentic assessments, academics should:

  • Use Turnitin contextually to identify plagiarism.
  • Ensure they have a good understand Academic Offences processes and penalties.
  • Report suspicions of Academic Misconduct.
  • Use oral examinations to determine authorship.

All automated tools have their limitations and recall that there is no ‘magic percentage’ on Turnitin that constitutes plagiarism. Each assessment and submission are different and must be interpreted uniquely.

When work is submitted and an AI tool has been used, it may not demonstrate the level of ability of the user or whether they have met their Learning Objectives. Consider whether use of the tool has given a student an unfair advantage, and whether they misrepresent the AI tool’s work as their own.

When conducting an oral examination as part of an investigation into academic misconduct, ask about the student’s work process, drafts, and how they found references.

 

More Resources

Templates and training for Academic Misconduct processes are available at the Academic Registry Staff Development SharePoint Site.

The QAA has compiled events and  relevant publications on their ChatGPT and Artificial Intelligence Page.

Inclusive and accessible Academic Integrity teaching resources are available on the QAA’s Collaborative Enhancement Project Page.