For OpenAI, o1 represents a step toward its broader goal of human-like artificial intelligence. More practically, it does a better job at writing code and solving multistep problems than previous models. But it’s also more expensive and slower to use than GPT-4o. OpenAI is calling this release of o1 a “preview” to emphasize how nascent it is.
The training behind o1 is fundamentally different from its predecessors, OpenAI’s research lead, Jerry Tworek, tells me, though the company is being vague about the exact details. He says o1 “has been trained using a completely new optimization algorithm and a new training dataset specifically tailored for it.”
OpenAI taught previous GPT models to mimic patterns from its training data. With o1, it trained the model to solve problems on its own using a technique known as reinforcement learning, which teaches the system through rewards and penalties. It then uses a “chain of thought” to process queries, similarly to how humans process problems by going through them step-by-step.
At the same time, o1 is not as capable as GPT-4o in a lot of areas. It doesn’t do as well on factual knowledge about the world. It also doesn’t have the ability to browse the web or process files and images. Still, the company believes it represents a brand-new class of capabilities. It was named o1 to indicate “resetting the counter back to 1.”
I think this is the most important part (emphasis mine):
As a result of this new training methodology, OpenAI says the model should be more accurate. “We have noticed that this model hallucinates less,” Tworek says. But the problem still persists. “We can’t say we solved hallucinations.”
I can answer math questions really really fast. Not correct though, but like REALLY fast!
😋
It scores 83% on a qualifying exam for the international mathematics olympiad compared to the previous model’s 13% so…
When you say previous model, you mean gemini with alpha geometry (an actual RL method)? Which scored a silver?
I mean not only google did it before, they also released their details unlike openai’s “just trust me bro, its RL”.
Openai also said that we should reserve 25k tokens for this “reasoning” and they will be charged the same as output tokens which is exorbitantly high (60$ for 1m tokens).
And the cherry on top is that they won’t even give us these “reasoning” tokens. How the hell am I supposed to improve my prompts if I can’t even see it? How would I reduce the hallucinations without it?
My personal experience is that, it does have an extra reasoning thing going for itself but in no way does it make openai’s tactics tolerable. The quality does not increase enough to justify its cost per token, let alone their “reasoning tokens” BS.
I’m the same with any programming question as long as the answer is Hello World
That’s a flat out lie, I use it for code all the time and it’s fantastic at writing useful functions if you tell it what you want. It’s also fantastic if you ask it to explain code or options for problem solving.