The Hidden Carbon Cost of Aligning AI: Carbon-Aware Scheduling for Reinforcement Learning from Human Feedback

AI調整の隠れた炭素コスト：人間のフィードバックからの強化学習におけるカーボンアウェアスケジューリング (AI 翻訳)

Saher Elsayed

ジャーナル2026-07-23#AI×ESGOrigin: US対象セクター: cross_sector

DOI: 10.1145/3811242.3819088

原典: https://doi.org/10.1145/3811242.3819088

🤖 gxceed AI 要約

日本語

本論文はRLHFのカーボンフットプリントを初めて段階別に実測し、PPOフェーズが総排出の59-64%を占めることを示した。提案するCarbonAware-RLHFはリアルタイムの系統炭素強度予測に基づきRLHFフェーズを低炭素時間帯にシフトし、38.9%の平均炭素削減を達成した。精度への影響は0.22pp以下の低下に留まり、優れた炭素-品質トレードオフを実現している。

English

This paper presents the first phase-resolved carbon footprint characterization of RLHF, finding that the PPO phase accounts for 59-64% of total training emissions. It introduces CarbonAware-RLHF, a scheduling framework that adaptively shifts RLHF phases to low-carbon windows using real-time grid carbon intensity forecasts, achieving a mean 38.9% carbon reduction with only a 0.22 percentage-point reward model accuracy degradation across a 90-day deployment in six US grid regions.

Unofficial AI-generated summary based on the public title and abstract. Not an official translation.

📝 gxceed 編集解説 — Why this matters

日本のGX文脈において

日本でも大規模言語モデルの開発が進む中、AIの環境負荷開示が課題となる可能性がある。本手法は日本の地域間で異なる系統炭素強度を活用したカーボンアウェアスケジューリングの実装指針となり、SSBJ等の開示基準におけるAI関連排出の測定・報告にも示唆を与える。

In the global GX context

As AI models become subject to climate disclosure regulations (e.g., ISSB, CSRD, SEC), this work provides a practical methodology for measuring and reducing AI training emissions. The carbon-aware scheduling framework offers a scalable approach for any organization running large-scale ML workloads, while the findings on geographic equity and Jevons paradox highlight structural limits relevant to global AI governance.

👥 読者別の含意

🔬研究者:Provides the first rigorous carbon measurement of RLHF training phases and a reproducible scheduling method, enabling further research in carbon-efficient ML.

🏢実務担当者:Offers an open-source scheduling tool (CarbonAware-RLHF) that can be integrated into AI training pipelines to reduce carbon footprint with minimal quality loss.

🏛政策担当者:Highlights the need for carbon accounting standards for AI and the potential of scheduling-based interventions, while cautioning against rebound effects.

📄 Abstract（原文）

Reinforcement Learning from Human Feedback (RLHF) has become the dominant paradigm for aligning large language models (LLMs) with human values, yet its environmental cost remains nearly invisible in both academic discourse and practitioner tooling. This paper presents the first phase-resolved empirical characterization of RLHF’s carbon footprint, combining Marginal Operating Emission Rate (MOER) based carbon accounting, real-time grid carbon intensity (CI) signals, and a multi-region 90-day deployment study, a scope and resolution not addressed by prior machine-learning (ML) carbon measurement work. We find that the Proximal Policy Optimization (PPO) phase alone accounts for 59–64% of total training emissions across all model scales, reaching 78.9 kgCO2eq for a single LLaMA-65B alignment run. We introduce CarbonAware-RLHF, an open-source scheduling framework that adaptively pauses, reorders, or temporally shifts RLHF phases toward low-carbon windows using real-time CI forecasts. In a 90-day deployment across six United States (US) Independent System Operator (ISO) grid regions (312 jobs, three independent seeds per condition), CarbonAware-RLHF achieves a mean carbon reduction of 38.9% with only a − 0.22 pp (percentage-point) reward model accuracy degradation and no statistically significant change on MT-Bench safety evaluation, the best carbon-quality trade-off among all evaluated strategies. Carbon savings correlate strongly with grid CI variability (coefficient of determination R2 = 0.91), providing a practical deployment heuristic. We discuss geographic equity, carbon accounting standards, and the Jevons paradox as structural challenges that technical scheduling alone cannot resolve. All data, code, and reproducibility artifacts are released at https://github.com/Saher-Elsayed/carbonaware-rlhf.

🔗 Provenance — このレコードを発見したソース

openalex https://doi.org/10.1145/3811242.3819088first seen 2026-07-26 05:26:59

🔔 こうした論文の新着を逃したくない方はキーワードアラートに登録（無料・3キーワードまで）。

gxceed は公開メタデータに基づく研究支援データセットです。要約・翻訳・解説は AI 支援で生成されています。最終的な解釈・検証は利用者が原典資料に基づいて行うことを前提とします。