ESG Disclosure Quality as Organizational Information Processing: Comparing Human Coding, Rule‐Based Automation, and LLM Semantic Scoring

Unofficial AI-generated summary based on the public title and abstract. Not an official translation.

🔬研究者:Provides empirical evidence that LLM-based scoring outperforms rule-based methods in convergence with human benchmarks, relevant for disclosure quality measurement methodology.

🏢実務担当者:Corporates and ESG raters should be aware that automated ESG scores are architecture-dependent and not direct extractions; LLMs offer better differentiation.

🏛政策担当者:Regulators developing digital disclosure frameworks should consider that the scoring method (human vs. automated) can systematically distort ESG quality assessments.

This study examines how different scoring architectures evaluate the same ESG disclosure text and produce different ESG disclosure‐quality scores. Using sustainability‐related reports published by Korean listed firms during 2020–2021, the study compares three approaches: Human‐ESG based on a structured nine‐item disclosure‐quality rubric, Rule‐ESG based on a rubric‐aligned deterministic dictionary system, and LLM‐ESG based on semantic scoring through a documented API protocol. All three approaches were applied to the same selected environmental, social, and governance disclosure segments. The analysis evaluates convergent validity, criterion validity, incremental validity, and structural diagnostics using Human‐ESG as a disclosure‐quality benchmark and KCGS ratings as a noisy institutional criterion. The findings indicate that LLM‐ESG shows greater convergence with the human‐coded benchmark and a stronger association with the KCGS institutional criterion than the rule‐based baseline, while producing different distributional properties. Structural diagnostics further suggest that the rule‐based architecture produces stronger upper‐end compression, whereas LLM‐based semantic scoring shows greater cross‐firm differentiation on several diagnostics. Additional robustness and influence diagnostics indicate that the main results are not driven by a small number of influential observations. The study contributes to ESG disclosure research by showing that disagreement in ESG evaluation can arise not only from institutional weighting differences, but also from the scoring architecture used to process disclosure text. The findings further suggest that automated ESG disclosure‐quality scores should be interpreted as architecture‐shaped outputs rather than direct extractions from sustainability reports.

• semanticscholar https://doi.org/10.1002/bse.71277first seen 2026-07-13 06:49:06
• scopus https://api.elsevier.com/content/abstract/scopus_id/105043885456first seen 2026-07-13 07:38:27