Taking Exams in the Age of AI: My Experience

The dilemma of assessing programming knowledge when artificial intelligence can solve exercises, analyze solutions, and answer quizzes even when students have not fully mastered the material. At the same time, AI can assist teachers with grading and reviewing submissions.

Introduction

Over the past few months, while preparing university assessments for technical subjects related to programming, scripting, and systems administration, I encountered a problem many instructors are likely facing: how can we assess a student’s knowledge when they have access to artificial intelligence tools capable of solving almost any exercise we give them?

In my case, the challenge has an additional dimension: the practical activities we assess are precisely the kind that are increasingly carried out with the assistance of artificial intelligence, using tools such as GitHub Copilot, ChatGPT, and other programming assistants (AI-assisted coding).

In other words, we are trying to assess knowledge through activities that are already routinely performed with AI assistance in the professional world.

Prohibiting these tools during practical work can distance us from real working conditions. But allowing them without changing our assessment methods makes it difficult to determine what the student has actually learned and what the tool contributed.

This issue becomes especially important when assessments are conducted remotely through learning platforms such as Moodle. For years, these platforms have made teachers’ work considerably easier: they allow us to manage question banks, generate different quizzes, set time limits, automate grading, and manage grades for large courses.

However, the emergence of AI tools that can not only produce code but also explain, modify, and justify it introduces a problem that cannot be solved simply by changing the questions or making them harder.

My experience preparing programming assessments led me to reconsider some of the assumptions on which we have traditionally built our exams. Paradoxically, my first solution was to return to a much older method: in-person assessment, either oral or written on paper.

But that solution left new challenges unresolved and led me to consider other possibilities, including using artificial intelligence itself to assist with assessment.

1. A contradiction between education and professional practice

In fields such as cloud computing, DevOps, systems administration, and software development, using artificial intelligence tools to generate code is becoming common practice.

A professional may use Copilot to write a Bash script, ask ChatGPT to identify a Python error, or ask an assistant to suggest improvements to an automation task. These tools can speed up work, automate repetitive processes, and help explore different implementation options.

Of course, using AI does not eliminate the need for technical knowledge. An automatically generated solution may contain errors, introduce security issues, or fail to account for specific operating conditions. That is why understanding, validating, and taking responsibility for the results remain essential in professional work.

In this context, prohibiting AI during programming practice may mean assessing students under conditions that are artificially different from those they will encounter in their professional lives.

But allowing AI without changing the assessment criteria also creates difficulties.

Consider a simple exercise: write a Bash script that takes an IP address or hostname, checks its availability, and displays a result. One student may develop the solution using their own knowledge. Another may enter the assignment into an AI tool and obtain a working script in seconds. Both could submit nearly identical results.

Can we conclude that both students acquired the same knowledge? Evidently not.

This brings us to a fundamental distinction: getting a correct solution is not necessarily the same as understanding how it works.

2. The knowledge we want to assess

Before considering how to administer exams, I think we need to establish what we intend to assess.

In subjects related to programming and systems administration, my goal is not necessarily for students to memorize every command or write every line of code without assistance. What I want to verify is that they have acquired three fundamental capabilities:

• Basic theoretical knowledge. Students should understand the concepts needed to develop and understand practical work. For example, anyone writing Bash scripts should know about variables, conditionals, loops, processes, permissions, exit codes, and other fundamental concepts of the language and operating system. They do not necessarily need to remember the exact syntax of every command, but they should understand the concepts that determine how it works.
• Solving practical tasks. Students should be able to solve the proposed tasks successfully, using whatever methods and tools they consider appropriate. They may use documentation, examples, search engines, and AI tools. What matters is building a solution that meets the requirements, works correctly, and can be verified.
• Understanding the solution. Students should fully understand the solution they submit and be able to answer questions about and justify relevant aspects of the code, configuration, design, and implementation decisions. It is not enough to submit a working script: they should explain why it works, what happens under conditions different from those anticipated, what limitations it has, what errors might occur, and how to address them. If the exercise is to configure an HTTPS service on a Linux instance, for example, they should be able to explain the purpose of certificates, private keys, ports, permissions, and the behavior observed during testing.

Ultimately, I do not consider it a problem for a student to use AI to build a solution, as long as they can demonstrate that they fully understand what they are submitting.

These capabilities are related, but they are not equivalent. And that is precisely where the problem arises: how can we verify them individually when AI tools can also explain and justify almost any solution?

3. Assessing how AI is used to solve the problem

So far, the approach has been to allow students to use artificial intelligence tools in their practical work while requiring them to fully understand the solutions they submit.

However, I think there is another aspect worth considering: if many of these problems are now solved professionally with AI assistance, perhaps we should also assess the process students use to reach a solution with these tools.

In other words, instead of assessing only the final product, we could also assess a student’s ability to work with AI effectively and critically.

Defining and framing the problem correctly

One fundamental skill is understanding the problem and translating it into clear requirements. Before asking an AI tool to generate a script, students should be able to identify what needs to be solved, what inputs the program will receive, what results are expected, and what constraints must be considered.

An incomplete or poorly framed request may produce a seemingly correct solution that does not meet the exercise’s objectives.

For example, if we want to develop a Bash script to check the availability of a set of servers, it may not be enough to ask AI to generate a program that runs ping. We need to consider how the list of servers will be provided, what it means for a server to be available, how errors will be handled, and how results will be presented.

The ability to define the problem correctly may be just as important as writing the code that solves it.

Interacting with AI and progressively improving the solution

Another skill is knowing how to use the tool during development. An AI’s first response is unlikely to always produce the best solution.

Students should be able to review the results, identify requirements that were overlooked, request corrections, and evaluate different alternatives.

This involves an iterative process: frame the problem, get a proposal, analyze it, identify shortcomings, and improve it. We could assess how the solution evolves through this process and what decisions the student makes.

The goal would not necessarily be to judge the complexity of the prompts, but to observe whether interacting with the tool effectively helps solve the problem.

Checking and validating generated code

This is probably one of the most important aspects.

An AI tool can generate syntactically correct and seemingly reasonable code, but that does not mean it will work properly under all circumstances.

Students should design and run tests, check the results, identify unanticipated conditions, and fix any errors they find.

In a scripting exercise, for example, they could assess whether the program handles invalid parameters, missing files, permission errors, or connectivity failures correctly.

In an infrastructure exercise, they should verify that a configuration not only works but also meets the security requirements and constraints of the assignment.

Responsibility for the validity of the result must remain with the person who develops and submits the solution, regardless of whether some of the work was generated automatically.

Understanding and justifying decisions

Finally, students should be able to explain why the chosen solution is appropriate. This includes justifying the tools used, design decisions, error-handling mechanisms, and possible alternatives.

They should also be able to identify limitations and recognize situations in which the implementation would need to be changed.

AI can assist with this analysis, but that does not remove the need to verify that students truly understand the decisions they present.

Evidence for assessing the process

One possibility would be to ask students to submit, along with the final product, a brief description of the AI-assisted development process: the initial problem statement, the main changes made, the issues found, and the tests used to validate the result.

They could also retain relevant excerpts from interactions with the tool, without submitting entire conversations or turning the number of prompts into a grading criterion.

In some cases, it might be useful to use Git repositories to record different versions of the solution and observe how it evolves.

However, we must recognize an important limitation: AI itself can also generate a convincing account of how the work was supposedly developed.

Therefore, a process description, a prompt history, or a sequence of commits are not, on their own, sufficient evidence of individual learning. They can even be reconstructed after the final solution has been obtained.

This evidence should primarily be used to assess the quality of the documented process and, when individual competencies need to be verified, be supplemented with some form of independent verification.

A new professional skill

All this leads me to believe that AI-assisted development should be explicitly included among the skills we teach and assess.

Knowing how to write code is not enough, but neither is knowing how to ask AI to write it. Students need to learn how to frame problems, use tools, interpret proposals, identify errors, verify results, and make well-founded technical decisions.

In this model, artificial intelligence does not necessarily replace learning to program: it changes the activities through which that learning is applied and demonstrated.

And this raises an additional question: are we preparing students to program the way people did before these tools appeared, or to develop solutions in the technological environment in which they are likely to work?

4. The limits of changing the types of questions

One initial alternative is to replace traditional programming exercises with activities that require more reasoning:

• Provide an existing script and ask students to explain how it works.
• Introduce errors and ask students to identify and fix them.
• Ask for modifications to incorporate new requirements.
• Present execution results and ask students to interpret them.
• Require students to justify specific implementation decisions.

At first glance, these activities seem to assess understanding rather than simply check whether students can produce code.

However, when we examine these possibilities, an obvious difficulty emerges: all of them can also be solved quickly by artificial intelligence tools.

Today’s models do more than generate code. They can also explain, debug, modify, and analyze logs, interpret errors, and justify technical decisions. Even a detailed, seemingly reasoned explanation may have been generated entirely by AI.

Therefore, replacing a programming exercise with an analysis or debugging task does not solve the problem of remote assessment. Nor does simply increasing the difficulty or generating multiple exam versions necessarily solve it.

These measures may be useful, but they cannot guarantee that responses reflect a student’s individual knowledge. The problem goes deeper than question design.

5. My first solution: going back to analog methods

After considering different alternatives, my first response was to separate solving the practical exercise from individually assessing knowledge.

The proposal consisted of two stages.

Practical work with freedom to use tools

Students could complete the exercise using documentation, search engines, AI assistants, Copilot, or any other available tool.

This way, the practical work would reflect real professional conditions. The goal would be to assess the solution produced: its functionality, quality, suitability for the requirements, and results.

Individual in-person assessment

The second stage would be an in-person quiz, with oral questions or written answers on paper, without technological assistance.

Its purpose would be to verify fundamental theoretical knowledge and understanding of the solution developed.

For example, if the practical assignment involved developing a Bash script to check the availability of different servers, the quiz could address concepts such as parameters, variables, conditional structures, exit codes, redirection, and error handling.

It could also include specific questions about the submitted script, how it behaves in particular cases, or the reasons behind certain implementation decisions.

The point would not be to require students to memorize every line of code, but to verify that they understand the mechanisms on which it relies and can explain and justify relevant technical decisions.

In this way, the practical assignment would assess the ability to solve problems using current tools, while the in-person exam would aim to verify individual knowledge.

Paradoxically, in a context of increasing digitization, the initial solution was to bring back some traditional assessment methods. Not out of nostalgia, but as a pragmatic decision.

6. The problem this solution does not solve

Although this combination clearly separates the two objectives, it has one important limitation: it does not solve the challenge of remote assessment or make full use of the advantages of digital learning platforms.

Moodle, for example, offers particularly useful features for managing assessments in large courses: question banks, random selection, categories, parameterized variants, time limits, attempt records, and automatic grading for different types of answers.

These capabilities considerably reduce administrative work and make it easier to assess large groups. Replacing all of this with paper quizzes means giving up many of those advantages.

Individual oral assessment, in turn, offers opportunities to explore a student’s understanding in greater depth, but it is difficult to scale when we need to assess thirty, fifty, or more students.

In addition, in fully remote courses, requiring students to be on campus may not be compatible with the nature of the course.

Therefore, my first solution partially addressed the pedagogical problem but introduced new operational difficulties. This led me to reframe the initial question:

Is it possible to retain the advantages of digital assessments while recreating sufficiently controlled conditions to verify individual knowledge?

7. The technological challenge: recreating the conditions of a traditional exam

A written in-person exam has a feature we often take for granted: the instructor can observe the environment in which it takes place. They can determine which materials are allowed, limit the use of devices, and monitor interaction among students.

That control is not perfect either, but it provides a reasonable level of confidence about assessment conditions.

When we move an exam to a computer, some of that control disappears. Students can open other windows, consult external resources, or turn to an AI assistant for answers.

In this context, technologies exist that try to recreate some of the restrictions of a traditional exam. One is Safe Exam Browser, a browser designed for controlled assessments that can integrate with Moodle and restrict access to certain applications and resources on the device.

In a classroom, using it alongside instructor supervision can provide conditions reasonably similar to a traditional written quiz while retaining the benefits of digital grading and administration.

The remote scenario, however, is considerably more complex. Even if we restrict the applications and browsing on the computer used for the exam, nothing necessarily prevents a student from consulting an AI tool on another device.

We can add video-call supervision, recordings, or remote proctoring systems, but these mechanisms do not offer absolute guarantees either. They also introduce other challenges related to privacy, accessibility, technical requirements, and the handling of personal data.

Therefore, we should distinguish between two different objectives:

• Reducing opportunities to use unauthorized external assistance.
• Guaranteeing that a response was produced exclusively by the student.

The first can be addressed through a combination of technological tools and supervision procedures. The second is much harder to ensure, especially when an exam takes place in a remote environment we do not control.

8. A practical alternative: limiting response time

Another possibility, especially for multiple-choice, true-or-false, and other closed-response questions, is to set sufficiently tight time limits.

The idea is relatively simple. If a student knows the answer, they should be able to identify it in a reasonably short time. By contrast, if they need to copy the question, consult a search engine or AI tool, interpret the answer, and return to the quiz, that process may take additional time.

Therefore, an exam with tight time limits could make systematic use of external resources more difficult.

But this raises a new challenge: determining how much time to allow.

A limit that is too generous may allow students to consult practically every answer. One that is too restrictive may disadvantage students who know the material but need more time to read, analyze, or respond.

In addition, current AI assistants are getting faster, and there are ways to consult information without manually copying each question.

For that reason, I do not consider time limits a guarantee against AI use, but rather a strategy that deserves experimental evaluation.

How to determine appropriate time limits

One possibility would be to measure the response times of students who know the material and compare them with the time needed to answer the same questions using AI.

This would make it possible to analyze whether there is a sufficiently significant difference to set useful limits.

We would also need to consider each question’s difficulty, length, reasoning requirements, accessibility needs, and differences between devices.

Digital quizzes could combine overall time limits, randomized questions, varied answer order, and browsing restrictions.

Moodle supports several of these settings, although not all of them provide an independent timer for each question.

The goal should not be to create a race against the clock, but to find conditions that encourage answers based on previously acquired knowledge without compromising the validity and fairness of the assessment.

9. The other side of AI: assisting teachers with grading

So far, the analysis has focused mainly on the problems AI introduces when students use it to complete assessments.

But there is another possibility I find especially interesting: using it as an assistant for teachers, particularly when reviewing and grading practical work and open-ended questions.

Multiple-choice and true-or-false quizzes can be graded automatically through platforms such as Moodle. However, essays, development exercises, and practical assignments usually require a much more labor-intensive manual review.

In large courses, this task can consume a considerable amount of time. This is where artificial intelligence could offer important benefits.

An initial automated review

Imagine a practical assignment in which students must configure an HTTPS service, generate certificates, run tests, and submit specific files and evidence.

An AI-assisted first review could check whether the submission includes the requested items, follows the required format, and addresses the required points in its explanations.

It could also identify potential technical inconsistencies, incomplete answers, or explanations that deserve closer review.

Naturally, AI should not simply decide whether an answer is correct based on how it looks.

For aspects that can be checked objectively, such as file structure, commands executed, or the results of specific tests, it would be preferable to use traditional automated validation methods.

AI could complement these methods by interpreting open-ended responses, documentation, and less structured technical evidence.

Preliminary grading of essay questions

Something similar could be applied to theoretical essay questions, especially when they were answered in an assessment where AI assistance was not allowed.

Given a prompt, a reference answer, and a grading rubric, an AI system could make an initial pass through the responses and flag:

• Whether the expected fundamental concepts are mentioned.
• Whether the explanation is technically correct.
• Whether important aspects are missing.
• Whether there are conceptual errors or contradictions.
• Which parts of the response the instructor should review.

It could even suggest a preliminary grade and individual feedback.

But I think it is important to maintain one distinction: assisting with grading does not mean automatically delegating the grading decision.

Models can also make mistakes, misinterpret a valid explanation, or evaluate conceptually equivalent answers differently.

For that reason, at least initially, I would use these results as a first pass to guide and simplify human review. The reliability of this assistance should be evaluated by comparing its results with instructors’ grading.

10. Comparing submissions: identifying possible copying

Another interesting application is comparing the responses of different students who completed the same exercise.

In a traditional assessment, an instructor may notice suspicious similarities between assignments, but identifying these patterns manually is difficult when there are many submissions.

An automated system could compare text, source code, file structures, and implementation decisions.

In programming, for example, it could analyze similarities beyond variable names or superficial formatting changes. It could also identify unusual errors repeated across different solutions.

In written assignments, it could compare sequences of explanations, examples, omissions, and shared conceptual errors.

These similarities do not necessarily prove that copying took place. Two students who correctly solved the same problem may produce very similar answers, especially when the exercise allows few reasonable alternatives.

Even the independent use of the same AI tool could produce similar solutions.

Therefore, automatic similarity detection should identify submissions that need review, not automatically establish academic misconduct.

The final decision would need to take into account the context, the characteristics of the exercise, and the students’ explanations.

11. The limits of detecting AI-generated answers

A natural question is whether we can use AI tools to identify work or responses that were also generated using artificial intelligence.

Commercial systems exist that try to detect writing patterns associated with text generated by language models. However, their results have significant limitations.

They may incorrectly identify human-written text as AI-generated, or fail to recognize text produced with technological assistance.

In addition, a short answer, a code fragment, or a highly standardized technical explanation may offer few useful features for this kind of analysis. Modifying or rephrasing text can also affect the results.

For this reason, I would not consider it reasonable to use an AI detector as conclusive evidence that a student violated exam rules.

At most, certain tools could provide supplementary information for an instructor’s review, provided their operation, limitations, and data handling are appropriate.

There is also an important distinction: if we explicitly allow AI for practical work, detecting its use in those submissions would make little sense.

By contrast, when an assessment requires individual answers without assistance, the question becomes how to verify that those conditions were respected.

Automated detection does not offer a sufficiently reliable solution on its own.

12. AI as support for assessing large courses

Combining these possibilities suggests an interesting approach to improving assessment without replacing instructor judgment.

In a first stage, students would complete practical activities using the permitted tools, including AI. Their submissions could undergo automated checks for functionality, format, and compliance with requirements.

Next, an AI-assisted system could conduct a preliminary review of explanations, identify omissions or inconsistencies, and compare submissions to flag similarities that deserve attention.

Finally, the instructor would review the results, especially cases that are unclear or require more careful judgment.

At the same time, assessments intended to verify individual knowledge could be conducted through controlled digital quizzes or additional supervision mechanisms.

This model would distinguish between roles:

• Traditional automation: checking objective conditions, formats, execution, and verifiable results.
• Artificial intelligence: assisting with the interpretation of open-ended answers, identifying possible omissions, and generating preliminary observations.
• Instructor: setting criteria, reviewing cases, resolving ambiguities, and taking responsibility for grades.

The expected result would be less repetitive grading work without abandoning individual academic assessment.

Of course, this model would also require validating the quality of AI-assisted grading, protecting student information, and establishing transparent criteria for using these tools.

13. A challenge that goes beyond individual instructors

The possibilities described raise questions that are difficult to resolve through the individual decisions of each instructor.

Universities and their academic leadership need to take part in the discussion.

Criteria need to be defined for the permitted use of AI tools, the conditions of remote assessments, supervision mechanisms, student privacy, and the validity of different methods of verifying learning.

The technological and human resources needed to implement new approaches should also be evaluated.

It does not seem reasonable to simultaneously require remote assessments, guarantees of individual performance, and fully automated grading without acknowledging the existing limitations.

The problem is not simply finding questions ChatGPT cannot answer. In fact, that strategy is likely to become increasingly unsustainable.

The challenge is to establish what we want to verify, what evidence we consider sufficient, and what assessment conditions we can realistically guarantee.

14. Final reflections

My experience preparing assessments in scripting and systems administration led me to a conclusion I initially did not expect.

At a time when artificial intelligence is transforming how software is developed, my first response for verifying individual knowledge was to bring back in-person assessment, either oral or written on paper.

This decision did not mean rejecting AI. On the contrary, it started from accepting that using it during practical work is consistent with current professional conditions.

But it also recognized that a working program, even one accompanied by a detailed explanation, is no longer sufficient evidence of individual understanding.

What we want to assess is whether students have acquired fundamental theoretical knowledge, can successfully complete a practical task using available tools, and fully understand and justify the solution they submit.

I would add one more objective: assessing how they use AI during the problem-solving process. If the problems in our courses are currently solved with artificial intelligence assistance, we should also teach students to frame requests correctly, critically analyze responses, improve proposed solutions, and validate their results.

In-person assessment offered a relatively simple way to verify some of this knowledge. However, it left an important problem unresolved: how to bring those conditions into the digital world without giving up the advantages of platforms such as Moodle or making assessment of large courses impractical.

There are different alternatives worth exploring: restricted browsers, remote supervision, carefully designed time limits, and supplementary individual checks.

At the same time, artificial intelligence itself creates opportunities to assist instructors with reviewing open-ended answers, checking submissions, and identifying similarities between assignments.

I do not yet have a definitive answer.

But I do believe we should stop framing this discussion exclusively in terms of whether students should be allowed to use AI.

Perhaps the solution is not to develop exams that are impossible for AI to solve, but to combine assessment of the result, the AI-assisted development process, individual understanding, and fundamental knowledge, while also making use of automation tools that can keep assessment manageable.

Artificial intelligence forces us to reconsider an assumption that seemed obvious for a long time: that correctly solving an exercise proves that the person submitting it has the knowledge needed to do so. Today, we can no longer take that relationship for granted.

The real challenge is no longer just determining whether a student can solve a problem without AI, but verifying that they have the knowledge to solve it responsibly with AI, understand the result, and take responsibility for the technical decisions involved.

At the same time, we need to find reliable, fair, and scalable ways to verify these abilities.

This is a discussion we are only beginning to have.

---

How are you addressing this problem in your courses? Do you assess the AI-assisted development process? Do you use AI tools to help with grading? Have you experimented with time-limited remote assessments, restricted browsers, or other mechanisms to verify individual knowledge? I am particularly interested in concrete experiences in technical programs and large courses.