Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms
日本の上場企業におけるESGナラティブスコアリングでの高推論LLM導入の限界的便益：4モデルコンセンサス研究 (AI 翻訳)
Hiroyuki Kokubu
🤖 gxceed AI 要約
日本語
本論文は、ESGナラティブ開示の自動スコアリングにおける高推論LLMの有効性を、日本の上場企業10社のデータを用いて検証した。4モデルのコンセンサス設計により、高推論モデルと低推論モデルの間のスコア差は平均0.38（5点満点）と小さく、コストは約5.6倍であることを示した。結果として、高推論モデルの追加的便益は限定的であり、コスト効率の高いESGスコアリングパイプラインの構築には低推論モデルのコンセンサスが有効であると結論づけている。
English
This paper empirically examines the value of reasoning-heavy LLMs for automated ESG narrative scoring, using data from ten Japanese listed firms. Using a four-model consensus design, it finds that the mean score deviation between the reasoning-heavy model and reasoning-off counterparts is only 0.38 on a 5-point scale, while the cost is 5.6 times higher. The results suggest that reasoning-heavy deployment offers limited marginal benefit, and cost-effective ESG auto-scoring pipelines can rely on reasoning-off consensus. Implications for LLM deployment governance in accountability settings are discussed.
Unofficial AI-generated summary based on the public title and abstract. Not an official translation.
📝 gxceed 編集解説 — Why this matters
日本のGX文脈において
本論文は日本企業のESG開示に焦点を当てており、SSBJ基準や有価証券報告書におけるナラティブ開示の自動評価に関心がある実務者にとって示唆に富む。高コストな最先端LLMに過度に依存せず、複数の軽量モデルのコンセンサスで十分なスコアリング精度が得られる可能性を示しており、コスト抑制と開示品質向上の両立に貢献する。
In the global GX context
This study is directly relevant to the global push for cost-effective ESG disclosure evaluation using AI. As regulators like the ISSB and SEC emphasize narrative disclosures, automated scoring tools are becoming essential. The finding that reasoning-off LLM consensus can match reasoning-heavy model performance at a fraction of the cost has implications for firms and service providers seeking scalable, affordable ESG assessment solutions. It also contributes to the governance debate on appropriate LLM use in accountability settings.
👥 読者別の含意
🔬研究者:Researchers can leverage the four-model consensus methodology and the cost-benefit analysis framework for future work on LLM deployment in ESG contexts.
🏢実務担当者:Practitioners in sustainability teams and ESG rating agencies can use these findings to design cost-effective auto-scoring pipelines, avoiding overinvestment in expensive models.
🏛政策担当者:Policymakers overseeing disclosure standards may consider the implications of LLM-based scoring for regulatory compliance and the need for governance on model deployment.
📄 Abstract（原文）
Automated scoring of ESG narrative disclosures with large language models (LLMs) is gaining traction, yet whether reasoning-heavy frontier models add value commensurate with their cost remains empirically unsettled. We evaluate this question on a corpus of ten Japanese listed firms across three rubric axes — quantitative targets, progress-tracking infrastructure, and external-standard alignment — using a four-model consensus design that combines a reasoning-on frontier model with three reasoning-off contemporaries. Across 120 firm×axis×model scores, the pooled mean absolute deviation between the reasoning-on model and each reasoning-off counterpart is 0.38 on a 5-point scale; only 2% of pairwise comparisons reach a two-point deviation, and none exceeds two points. Per-firm cost accounting shows the reasoning-on arm alone costs roughly 5.6× as much as the three-provider reasoning-off ensemble, for outcomes that differ only within small margins. We conclude that in span-based ESG narrative scoring, reasoning-heavy deployment does not materially improve outcomes relative to reasoning-off consensus, while substantially increasing operational cost. We discuss implications for cost-effective ESG auto-scoring pipelines and LLM deployment governance in applied accountability settings. An earlier version of this work is available on SSRN (Abstract ID 6683303).
🔗 Provenance — このレコードを発見したソース
- arXiv https://arxiv.org/abs/2606.13693first seen 2026-07-17 23:33:03 · last seen 2026-07-18 00:30:05
🔔 こうした論文の新着を逃したくない方は キーワードアラート に登録（無料・3キーワードまで）。
gxceed は公開メタデータに基づく研究支援データセットです。要約・翻訳・解説は AI 支援で生成されています。 最終的な解釈・検証は利用者が原典資料に基づいて行うことを前提とします。