The Efficiency Revolution Accelerates
While we reported earlier this week that AI models now run on 15-year-old hardware, a new technical achievement pushes the boundaries even further. According to Hacker News, developers have achieved 4.74 tokens per second running the massive Qwen3.5-397B model using just 5.9GB of RAM—a feat that would have been unthinkable even months ago.
This represents a dramatic leap beyond the 4GB laptop demonstrations we covered. Running a 397-billion parameter model—comparable in scale to the largest proprietary systems—with such minimal memory requirements fundamentally changes the economics of AI deployment.
Community Developers Double Down on Specialization
TheLocalDrummer announced the release of four new specialized models on Reddit: Skyfall 31B v4.1, Valkyrie 49B v2.1, Anubis 70B v1.2, and Anubis Mini 8B v1. These releases, described as "significant upgrades" matching the sound of Gen 4.0 models, continue the trend we identified of open source AI fragmenting into specialized niches.
The gaming and adventure-themed branding suggests developers are targeting specific use cases rather than competing directly with general-purpose models—a strategy that aligns with the community's shift toward lightweight, purpose-built tools.
Performance Reality Check
Not all efficiency gains are universal. jnmi235 shared detailed benchmarks of Mistral-Small-4-119B running on an RTX Pro 6000, revealing important limitations:
- Generation speed drops from 131.3 tokens/second (1K context, single user) to 42.8 tokens/second (256K context, two users)
- Time to first token increases from 0.5 seconds to over 2 minutes at maximum context length
- Performance becomes impractical beyond 256K context with multiple users
These benchmarks provide crucial context to the Qwen3.5 breakthrough—efficiency gains don't always translate across all models or use cases.
The Bigger Picture
Hugging Face released their official "State of Open Source" report for Spring 2026, providing industry-wide perspective on these developments. While the specific details weren't available in our sources, the timing suggests these efficiency breakthroughs and specialized releases represent broader trends in the ecosystem.
The contrast between the Qwen3.5 efficiency achievement and Mistral's performance limitations illustrates a key dynamic: the open source community is simultaneously pushing boundaries in multiple directions—extreme efficiency for massive models and practical performance limits for production deployments.
What This Means
These developments mark a significant evolution from last week's democratization milestone. We're not just seeing AI run on old hardware—we're witnessing fundamental breakthroughs in how efficiently these models can operate. The 5.9GB RAM requirement for a 397B parameter model suggests that hardware constraints may soon become irrelevant for even the largest models.
Combined with the community's continued focus on specialized variants, the open source ecosystem appears to be solving the accessibility problem from multiple angles: making large models radically more efficient while creating smaller, purpose-built alternatives for specific use cases.