GitHub adds organization-level model rules for Copilot
GitHub now lets enterprise owners target which Copilot models are available to specific organizations, giving teams finer-grained model governance without changing enterprise-wide defaults.
-
-
LLM Confidence Calibration Breaks Down on Hard Tasks, New Benchmark Finds
A preregistered study reports that LLM confidence is not a reliable proxy for correctness: models are overconfident on hard tasks and underconfident on easy ones. The authors introduce LifeEval to test calibration across difficulty levels.
-
Goal-level energy accounting for agentic AI
A new arXiv paper argues that agent energy benchmarks should measure completed goals, not single model calls. It proposes Energy per Successful Goal (EpG) and Orchestration Overhead Index (OOI) to capture retries, tool use, and orchestration costs.
-
Google Tensor SDK beta adds LiteRT deployment for Pixel 10 on-device AI
Google has moved Tensor ML SDK to beta with LiteRT integration, letting developers convert, compile, deploy, and run models on Pixel 10’s TPU for lower-latency, private on-device AI.