Grok 4 Claims AI Supremacy—But Does It Deliver?

Elon Musk’s xAI has officially released Grok 4, the latest version of its large language model (LLM), and the internet is buzzing with claims that it has “crushed every AI benchmark.” Touted as a breakthrough in artificial intelligence, Grok 4 is earning praise from enthusiasts as a state-of-the-art system. But how much of the hype is real?

Benchmark Breakdown: Fact vs. Fiction

Grok 4 has performed strongly across academic and technical benchmarks:

• Humanity’s Last Exam (HLE): Grok 4 Heavy scored 44.4%—well above GPT-4o (~25%) and Gemini (~21%).
• Intelligence Index: Grok 4 scored 73, ahead of GPT-4o (70) and Claude Opus (64), according to Artificial Analysis.
• ARC-AGI: On abstract reasoning tasks, Grok 4 scored 16.2%, nearly double Claude Opus’s 8.5%.
• SWE-Bench: The coding-focused Grok 4 Code scored 72–75% on software engineering tasks—leading all competitors.
• VendingBench: In business simulations, Grok 4 generated 5x more revenue than its nearest rival.

Architecture and Features

One key innovation is Grok 4’s multi-agent architecture, where multiple specialized AI agents collaborate. This boosts performance on complex tasks beyond what single-agent models can handle.

Grok 4 also accesses real-time data from X (formerly Twitter), offering a major advantage over static models like GPT-4o and Claude Opus, which rely on periodic updates.

Pricing and Access

Grok 4 is available through two subscription tiers:

• Grok 4 (Standard): Included in X Premium+ at $30/month.
• SuperGrok Heavy: $300/month for early access to Grok 4 Heavy.

Developer pricing includes:

• $3 per million input tokens
• $15 per million output tokens
• 256,000-token context window

Hype vs. Reality

While Grok 4 leads many benchmarks, some claims—like “beyond human in every academic discipline”—are not supported by current data. Experts say Grok 4 excels in reasoning and coding but doesn’t universally outperform humans or dominate all tasks.

In multimodal areas like image and audio processing, OpenAI’s GPT-4o and Google’s Gemini 2.5 Pro remain competitive leaders.

Benchmark Comparison

Final Verdict

Grok 4 is a strong competitor in today’s AI race—particularly for tasks involving logic, software, and current events. Its multi-agent architecture and live data feed give it an edge in dynamic use cases. However, claims of absolute dominance are overstated. The broader AGI race is still ongoing, and Grok 4’s success is context-specific, not universal.

(With inputs from Artificial Analysis and independent benchmark databases.)