Watchtower

Between January 13 and January 17th of 2025, OpenAI once again changed the o1 system card on their website. This time, they made dozens changes throughout the text. Notably, they have not announced these changes, nor have they acknowledged it on the system card itself — the "last updated" timestamp still reads "December 5, 2024."

The most notable differences are:

• Finally adding the evaluation results from the full o1 model to the website
• Acknowledging that some evaluations, including those conducted by third-party evaluators like METR, were on an early checkpoint of the model (and not the version that released).

There are also many smaller but substantive changes throughout.

These changes follow increased awareness of issues with the system card first highlighted by Zvi Mowshowitz on his blog. It also follows The Midas Project's outreach to OpenAI concerning an undisclosed change to the system card made a week ago.

A full PDF of the changes, on the website and PDF system card respectively, can be found below. Note that some large sections which appear to be removed/added have simply been moved.