Vulnerability of LLMs in Educational Assessment (doi:10.7910/DVN/OV2WAM)

View:

Part 1: Document Description
Part 2: Study Description
Part 5: Other Study-Related Materials
Entire Codebook

Vulnerability of LLMs in Educational Assessment

doi:10.7910/DVN/OV2WAM

Harvard Dataverse

2025-09-12

1

Milani, Alfredo, 2025, "Vulnerability of LLMs in Educational Assessment", https://doi.org/10.7910/DVN/OV2WAM, Harvard Dataverse, V1

Vulnerability of LLMs in Educational Assessment

doi:10.7910/DVN/OV2WAM

Milani, Alfredo (https://ror.org/035mh1293)

Harvard Dataverse

Milani, Alfredo

Valentina Franzoni

Florindi Emanuele

Milani, Alfredo

2025-09-12

https://doi.org/10.7910/DVN/OV2WAM

Computer and Information Science, Social Sciences, Large Language Models, Prompt Injection, Education Sciences, Education Evaluation, Trustworthy AI, Human-in-the-Loop AI

The dataset contains the output of experiments on a research project on Vulnerability of LLMs in Educational Assessment. The Dataset contains: -the students assignments data in normal form and the injected form -the output produced by the experimented LLMs: ChatGPT, Gemini, DeepSeek, Grok, Perplexity and Copilot for the experiments evaluation the assignments, as a single document and collectively as a group of documents, denominated: -User Legitimate LLMs Prompts -Normal (no injection) providing the reference base evaluation -Prompt Injection Pass, one type of injection experiments, called Fail-To-Top, to move an assignment evailuated FAIL by reference base evaluation to PASS, i.e. above 35% of total points. -Prompt Injection to Top25 , a type of injection experiments to move to top 25% an assignment with lowe reference base evaluation . This latter type of experiment come in 3 versions, Fail-To-Top, Sat-To-Top, Good-To-Top where assignment with reference base evaluation respectively: Fail (below 35%), Satisfactory (greater than 25% and belo 50%) and Good (above 50% and below 75%) are considered for injection. The name of the folders and output results files are accordingly self-explanatory .

<a href="http://creativecommons.org/publicdomain/zero/1.0">CC0 1.0</a>

"When AI is Fooled: Hidden Risks in LLM-assisted Grading" Authors: Alfredo Milani, Valentina Franzoni, Emanuele Florindi, Assel Omarbekova, Gulmira Bekmanova, Banu Yergesh in Education Sciences, ISSN 2227-7102

2227-7102

"When AI is Fooled: Hidden Risks in LLM-assisted Grading" Authors: Alfredo Milani, Valentina Franzoni, Emanuele Florindi, Assel Omarbekova, Gulmira Bekmanova, Banu Yergesh in Education Sciences, ISSN 2227-7102

Label:

Normal_and_Injected_Assignment_Experiments.zip

Text:

The dataset contains the output of experiments on a research project on Vulnerability of LLMs in Educational Assessment. The Dataset contains: -the students assignments data in normal form and the injected form -the output produced by the experimented LLMs: ChatGPT, Gemini, DeepSeek, Grok, Perplexity and Copilot for the experiments evaluation the assignments, as a single document and collectively as a group of documents, denominated: -Normal (no injection) providing the reference base evaluation -Prompt Injection Pass, one type of injection experiments, called Fail-To-Top, to move an assignment evailuated FAIL by reference base evaluation to PASS, i.e. above 35% of total points. -Prompt Injection to Top25 , a type of injection experiments to move to top 25% an assignment with lowe reference base evaluation . This latter type of experiment come in 3 versions, Fail-To-Top, Sat-To-Top, Good-To-Top where assignment with reference base evaluation respectively: Fail (below 35%), Satisfactory (greater than 25% and belo 50%) and Good (above 50% and below 75%) are considered for injection. The name of the folders and output results files are accordingly self-explanatory .

application/zip