{"@context":{"@language":"en","@vocab":"https://schema.org/","citeAs":"cr:citeAs","column":"cr:column","conformsTo":"dct:conformsTo","cr":"http://mlcommons.org/croissant/","rai":"http://mlcommons.org/croissant/RAI/","data":{"@id":"cr:data","@type":"@json"},"dataType":{"@id":"cr:dataType","@type":"@vocab"},"dct":"http://purl.org/dc/terms/","examples":{"@id":"cr:examples","@type":"@json"},"extract":"cr:extract","field":"cr:field","fileProperty":"cr:fileProperty","fileObject":"cr:fileObject","fileSet":"cr:fileSet","format":"cr:format","includes":"cr:includes","isLiveDataset":"cr:isLiveDataset","jsonPath":"cr:jsonPath","key":"cr:key","md5":"cr:md5","parentField":"cr:parentField","path":"cr:path","recordSet":"cr:recordSet","references":"cr:references","regex":"cr:regex","repeated":"cr:repeated","replace":"cr:replace","sc":"https://schema.org/","separator":"cr:separator","source":"cr:source","subField":"cr:subField","transform":"cr:transform","wd":"https://www.wikidata.org/wiki/"},"@type":"sc:Dataset","conformsTo":"http://mlcommons.org/croissant/1.0","name":"Replication Data for: Automated Quality Assessment for LLM-Based Complex Qualitative Coding: A Confidence-Diversity Framework","url":"https://doi.org/10.7910/DVN/GM8T8Q","creator":[{"@type":"Person","givenName":"zhilong","familyName":"zhao","affiliation":{"@type":"Organization","name":"https://ror.org/0530pts50"},"name":"zhao, zhilong"}],"description":"This replication package contains all code and data necessary to reproduce the results presented in \"Cross-Domain Quality Assessment for Complex Qualitative Analysis: Validating Confidence-Entropy Signals Across Legal, Political, and Medical Tasks\". Research Context: This study extends beyond accessible coding tasks to validate automated quality assessment for complex qualitative analysis requiring domain expertise and interpretive judgment across legal, political, and medical domains. Package Contents: - Core Scripts: reproduce_all_results.py (main reproduction script), generate_synthetic_data.py (data generator), validate_reproduction.py (result validation) - Data Files: Synthetic datasets matching paper statistics for SCOTUS legal reasoning (390 cases), Hyperpartisan political analysis (644 cases), and MTSamples medical classification (1,000 cases) - Expected Outputs: All LaTeX tables (Table 1-5), validation reports, and cross-domain statistical analyses Key Findings Reproduced: - Cross-domain signal effectiveness (Table 1): Perfect correlation reproduction across all domains (±0.005 accuracy) - Dual-signal weight optimization (Table 2): 6.6-113.7% improvements over single-signal baselines - Cross-domain transferability (Table 3): 88.9% success rate for weight transfer across domains - Intelligent triage efficiency (Table 5): 45.4% vs 44.6% effort reduction (0.8% difference) - Domain-specific patterns: Confidence signals are stronger in legal contexts, and entropy signals are more reliable in political/medical domains Validation Status: Successfully reproduces all core findings with statistical significance maintained across complex analytical tasks. Demonstrates automated quality assessment viability for scaling complex qualitative research beyond accessible coding tasks. Usage: Run ./run_complete_reproduction.sh for complete reproduction, or python3 reproduce_all_results.py for individual table generation. All dependencies included.","keywords":["Computer and Information Science","Social Sciences"],"license":"http://creativecommons.org/publicdomain/zero/1.0","datePublished":"2025-08-26","dateModified":"2025-08-27","includedInDataCatalog":{"@type":"DataCatalog","name":"Harvard Dataverse","url":"https://dataverse.harvard.edu"},"publisher":{"@type":"Organization","name":"Harvard Dataverse"},"version":"1.1","citeAs":"@data{DVN/GM8T8Q_2025,author = {zhao, zhilong},publisher = {Harvard Dataverse},title = {Replication Data for: Automated Quality Assessment for LLM-Based Complex Qualitative Coding: A Confidence-Diversity Framework},year = {2025},url = {https://doi.org/10.7910/DVN/GM8T8Q}}","distribution":[{"@type":"cr:FileObject","@id":"reproduction_package.zip","name":"reproduction_package.zip","encodingFormat":"application/zip","md5":"cf2c64c662be72d7ea11152c7c4845ad","contentSize":"124689","description":"","contentUrl":"https://dataverse.harvard.edu/api/access/datafile/12013838"}]}