OpenAI’s next big AI model has ‘entered the AGI era’
The Verge·1 min read
For OpenAI, this could be seen as good PR in one small way — showing how powerful its models can be in an ever-intensifying AI race — but it also damaged OpenAI’s reputation as far as reliability. OpenAI made sure to say in a release that Astra is the company’s “most aligned model yet” and helps people “delegate complex work while maintaining oversight.” Jakub Pachocki, OpenAI’s chief scientist, spoke to reporters about difficulties with keeping AI models aligned with human interests, saying that “progress in intelligence does not guarantee progress in alignment,” and he said that monitoring AI systems is becoming more and more challenging. (Researchers have recently raised alarms about reports that OpenAI allows Astra to utilize “opaque recurrence,” or render its chain of thought — a “mental scratchpad” that researchers rely on to detect if a model is scheming against its human evaluators — unreadable.)
