• bitcoinBitcoin(BTC)$61,182.002.02%
  • ethereumEthereum(ETH)$1,719.942.29%
  • tetherTether(USDT)$1.000.05%
  • binancecoinBNB(BNB)$558.422.19%
  • usd-coinUSDC(USDC)$1.000.03%
  • rippleXRP(XRP)$0.930.04%
  • solanaSolana(SOL)$70.490.20%
  • tronTRON(TRX)$0.3052740.14%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.062.22%
  • HyperliquidHyperliquid(HYPE)$62.994.04%
  • zcashZcash(ZEC)$795.681.33%
  • moneroMonero(XMR)$489.752.88%
  • dogecoinDogecoin(DOGE)$0.0554540.95%
  • leo-tokenLEO Token(LEO)$9.170.38%
  • whitebitWhiteBIT Coin(WBT)$58.721.96%
  • chainlinkChainlink(LINK)$8.833.04%
  • USDSUSDS(USDS)$1.000.00%
  • cardanoCardano(ADA)$0.1453110.84%
  • Ethena USDeEthena USDe(USDE)$1.000.04%
  • stellarStellar(XLM)$0.127927-0.28%
  • daiDai(DAI)$1.000.00%
  • bitcoin-cashBitcoin Cash(BCH)$198.520.11%
  • nearNEAR Protocol(NEAR)$2.995.07%
  • litecoinLitecoin(LTC)$40.070.60%
  • avalanche-2Avalanche(AVAX)$6.701.55%
  • CantonCanton(CC)$0.072532-5.28%
  • tether-goldTether Gold(XAUT)$3,552.890.35%
  • Ripple USDRipple USD(RLUSD)$1.000.00%
  • uniswapUniswap(UNI)$4.72-0.01%
  • USD1USD1(USD1)$1.000.06%
  • the-open-networkGram (prev. Toncoin)(GRAM)$0.982.66%
  • suiSui(SUI)$0.662.35%
  • Global DollarGlobal Dollar(USDG)$1.000.01%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.09%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.01%
  • RainRain(RAIN)$0.003228-9.26%
  • hedera-hashgraphHedera(HBAR)$0.0514440.61%
  • BittensorBittensor(TAO)$191.172.73%
  • shiba-inuShiba Inu(SHIB)$0.0000040.06%
  • crypto-com-chainCronos(CRO)$0.040720-0.89%
  • quant-networkQuant(QNT)$141.961.57%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.120.00%
  • Circle USYCCircle USYC(USYC)$1.140.00%
  • okbOKB(OKB)$82.404.61%
  • MemeCoreMemeCore(M)$0.75-2.46%
  • Falcon USDFalcon USD(USDF)$1.000.06%
  • United StablesUnited Stables(U)$1.000.11%
  • pax-goldPAX Gold(PAXG)$3,557.250.35%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
  • aaveAave(AAVE)$98.77-1.01%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Exclusive: OpenAI reveals Performance Factory, a Codex-powered software optimization system that keeps ChatGPT fast

November 19, 2026
in AI & Technology
Reading Time: 8 mins read
A A
Exclusive: OpenAI reveals Performance Factory, a Codex-powered software optimization system that keeps ChatGPT fast
ShareShareShareShareShare

When a software update slows down an application, engineers need to go through multiple stages, including inspecting metrics, tracing requests across services, identifying the responsible code, and testing potential solutions. 

With AI agents taking on the main burden of writing and changing code, manual performance review of code changes becomes one of the key bottlenecks of the software development lifecycle.

YOU MAY ALSO LIKE

The Winners Will Be Measured in Hours Without Human Help – Unite.AI

Which Is Better for Managing Work? – Unite.AI

To address this challenge, OpenAI developed the Performance Factory, a new, internal AI system powered by Codex that automates the investigation and mitigation of performance regressions. OpenAI is already using Performance Factory to keep ChatGPT fast.

According to an exclusive engineering report shared with VentureBeat, the system tracked more than 1,700 investigations and identified the root causes of over 600 cases in just over three months. That’s an average of 17 automated investigations per day.

The Performance Factory provides a glimpse into how coding agents can evolve from solving individual tasks into building automated workflows that coordinate between AI agents and human engineers. This approach can help engineering teams leverage the speed of AI-assisted software development while reserving human attention for decisions that require judgment.

From manual debugging to automated workflows

OpenAI’s performance engineering team initially relied on manual investigations. When a service slowed down, engineers examined logs and traces, looked at code changes, and debugged live systems. The team developed more than 30 specialized tools to streamline these investigations.

Codex changed how engineers worked with these tools. Instead of manually searching through codebases and telemetry, engineers could ask Codex why a service had become slower. The agent could investigate logs and use tools to provide the answer. But engineers were responsible for steering the process.

To reduce this dependence, the team developed a reusable Codex skill called perf-forensics, which captures the performance team’s investigation procedures for the agent to reproduce.

After testing and refining the skill, OpenAI could give Codex a performance alert and receive a diagnosis in a single run.

At this point, the engineers became intermediaries between monitoring systems and Codex. They would copy alerts into Codex and then paste the findings into Slack. So, naturally, the team simply connected the alerts to cloud-based Codex sessions. 

But the investigations ended with a diagnosis that required review and implementation by an engineer. Meanwhile, the ChatGPT performance team was dealing with regressions almost daily and struggling to keep track of them.

In comments provided to VentureBeat, OpenAI’s performance team explained that it initially built a regression tracking system to manage the growing workload. This led to the realization that each tracked regression could become a task that Codex would automatically investigate and advance through different stages, such as identifying the root cause, proposing a fix, and verifying recovery.

This became the foundation of the Performance Factory’s task-based architecture.

How the Performance Factory works

The Performance Factory organizes engineering work into tasks that move through predefined workflows. A task might begin with a performance alert and progress through investigation, root cause identification, code changes, review, and verification.

Each task maintains its state and findings independently so that different agents can work on the same problem at different stages. According to the OpenAI performance team, the task record contains a free-form field where agents leave notes, along with structured metadata such as when a regression began and when its root cause was identified.

Instead of copying large amounts of profiling data into the task, agents record the relevant metrics, queries, and high-level findings. The next agent receives the task record and can rerun those queries if it needs to verify the results. This keeps the information passed between agents manageable while preserving access to the underlying evidence.

OpenAI Performance Factory (source: OpenAI)

The system has four main components.

  1. The task service stores tasks and their progress. Engineers define workflows through configuration files and provide markdown prompts that determine what the agents must do at each stage.

  2. A dispatch loop periodically checks on the tasks. When a task is ready to advance, it starts a Codex session and assigns the task to it. The agent records its findings and updates the task when it completes its work.

  3. The third component is an internal API for running Codex in the cloud. Agents operate in environments that have access to OpenAI’s code repositories and observability infrastructure. They are designed in a way that lets OpenAI control permissions and track agent actions. 

  4. Finally, subscriptions connect tasks to external events. For example, if an agent opens a pull request and needs an engineer’s review, the task can pause until it receives the engineer’s feedback on GitHub.

Performance Factory in action

OpenAI’s first concrete implementation of Performance Factory is the Regression Workflow, which focuses on performance regressions. Regression Workflow kicks into action when a configured alert fires. It creates a task that agents investigate using the perf-forensics skill that the OpenAI team had previously developed.

The agent first determines what changed by examining the affected metrics and identifying when performance started deteriorating and which requests were affected. 

After homing in on the problem the agent tries to establish the cause by examining several artifacts, including code changes, logs, and traces.

However, OpenAI found that agents sometimes stopped at the first plausible explanation, even when the evidence did not fully support it.

To address this problem, the team introduced an “adversarial review” process that uses a separate Codex session to challenge the original diagnosis. The reviewer receives the entire task record but starts with a fresh context. Its instructions explicitly tell it to find gaps in the diagnosis, verify the supporting evidence, and test alternative explanations. This allows the reviewer to devote its entire session to challenging the original agent’s conclusions.

For example, in one investigation, an agent attributed failing ChatGPT conversations to errors in image downloads. The reviewing agent noticed that the internal service responsible for locating files had already recovered while image requests continued failing. The reviewer therefore reopened the investigation because the original diagnosis did not explain the continuing failures.

According to the OpenAI performance team, the adversarial reviewer initially sent back a substantial number of diagnoses for further investigation. The team used this feedback to improve the original investigator’s instructions, reducing the number of cases returned for additional work. Currently, the adversarial review sends back around 25% of diagnoses.

Once a diagnosis passes adversarial review, Codex proposes a fix and opens a pull request. An engineer reviews the code and decides whether to merge it.

The results show the gap between diagnosing problems and implementing reliable fixes. According to the OpenAI performance team, of the more than 600 investigations that reached a root cause, around 400 progressed to a proposed fix. Approximately 50 fixes that were eventually shipped were influenced by or matched agent suggestions, while around 15 shipped fixes were the exact pull requests authored by Codex.

The team acknowledges that these results fall short of its ambitions for automated remediation. However, much of the work of resolving regressions involves detecting them, identifying their causes, tracking their progress, and verifying recovery. The Performance Factory has automated much of this process, while improving the quality of proposed fixes remains a priority. OpenAI has also completed a workflow evaluation suite that it plans to use to improve agent-generated fixes.

The OpenAI performance team found that Codex performs best when the cause of a regression is clear and the fix is relatively contained. For example, a code change might cause an application to serialize the same conversation several times while processing a request. Codex can identify the redundant work and propose serializing the conversation once and reusing the result.

More complex changes remain challenging, particularly when a new feature adds latency and requires architectural modifications. Agents sometimes propose overly complicated fixes or introduce new problems. For example, adding a cache might improve response times but create stale data. OpenAI has therefore added adversarial review to proposed fixes as well.

After an engineer deploys a fix, another agent checks whether the performance issue has been addressed. If the problem persists, the workflow sends the task back for further investigation. 

Another challenge OpenAI had to address was managing the volume of agent-generated findings. Engineers were initially given a user interface full of investigation results, and they had to determine which findings mattered.

The system now makes it easier for engineers to navigate results by grouping related incidents, ranking their estimated impact, and generating concise reports with graphs and metrics. This helps engineers evaluate proposed changes without reconstructing each investigation from raw agent output.

Lessons for enterprise AI teams

Engineers of agentic workflows can take away a few key lessons from OpenAI’s work on Performance Factory. The practices are applicable to agentic systems beyond performance engineering.

Avoid AI fatigue. Where humans must review an AI agent’s output, try to make sure that the most pertinent information is presented to them to avoid overwhelming them with too much content (aka AI slop).

Preserve evidence without overloading agent context. OpenAI’s approach of storing high-level findings and reproducible queries in task records allows agents to hand off work without transferring large volumes of raw telemetry. This pattern can help teams build multi-stage workflows that remain manageable as investigations become more complex.

Evaluate agents and their interactions with humans. OpenAI backtests investigations against historical incidents, measuring how accurately agents identify when problems started, their impact, and potential causes. These evaluations require care because historical evidence can expire, while information collected after an incident might reveal the correct answer. The team also evaluates agent-generated summaries separately to determine whether engineers can understand the evidence and proposed actions.

Monitor the agents themselves. OpenAI runs a daily agent that examines recent agent logs for recurring tool failures and permission problems. When it finds an issue that repeatedly blocks investigations, the Performance Factory can create a task and propose a fix for an engineer to review. Evaluations then check whether the change resolves the problem.

From reactive maintenance to continuous optimization

The Performance Factory currently focuses on responding to configured alerts, but OpenAI plans to expand it into proactive optimization.

The team is exploring ways for agents to compare CPU profiles, memory data, and request traces across releases to discover slowdowns that monitoring alerts might miss. It is also developing workflows for tasks such as service rightsizing, where agents can identify inefficient autoscaling configurations.

OpenAI is investigating whether agents could eventually deploy some low-risk fixes without human approval.

According to the OpenAI performance team, automating regression investigations has allowed engineers to spend more time on long-term performance improvements instead of constantly responding to incidents. The team also hopes to turn what engineers learn from individual investigations into reusable workflows that can improve services across the organization.

This approach could enable engineering teams to preserve and apply performance knowledge across projects and services. Lessons from one investigation could inform how agents identify and resolve similar problems months later, reducing repetitive debugging work and allowing engineers to focus on more strategic optimizations.

Credit: Source link

ShareTweetSendSharePin

Related Posts

The Winners Will Be Measured in Hours Without Human Help – Unite.AI
AI & Technology

The Winners Will Be Measured in Hours Without Human Help – Unite.AI

November 22, 2026
Which Is Better for Managing Work? – Unite.AI
AI & Technology

Which Is Better for Managing Work? – Unite.AI

November 22, 2026
What Does The Blue Snowflake Icon Mean For Your Tesla Battery?
AI & Technology

What Does The Blue Snowflake Icon Mean For Your Tesla Battery?

November 22, 2026
Google AI Previews Tool Retrieval in the Gemini API for Large Tool Libraries
AI & Technology

Google AI Previews Tool Retrieval in the Gemini API for Large Tool Libraries

November 22, 2026
Next Post
I BOUGHT THE DIP ON NVIDIA STOCK TODAY!

I BOUGHT THE DIP ON NVIDIA STOCK TODAY!

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
El-Sayed responds to Trump’s attack on his name and faith

El-Sayed responds to Trump’s attack on his name and faith

November 16, 2026
LIVE NOW: NVIDIA Q3 EARNINGS REPORT 2026

LIVE NOW: NVIDIA Q3 EARNINGS REPORT 2026

November 18, 2026
How Upward Comparison Undermines Cooperation | Chioceology Podcast Clip

How Upward Comparison Undermines Cooperation | Chioceology Podcast Clip

November 16, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!