Enthusiast Runs 284B DeepSeek on a PC Without Any GPU
Watching artificial intelligence hardware prices spiral into madness makes any clever software hack feel like pure rebellion against the silicon monopoly.
The open-source project GitHub repository called llama-modchip dropped under the Apache 2.0 license, featuring a suite of custom patches for ggml and llama.cpp. The mod was engineered to squeeze massive sparse Mixture-of-Experts models straight onto a regular consumer processor without relying on dedicated graphics cards or virtual memory swapping.
The developer tested the setup on DeepSeek-V4-Flash, an architecture boasting 284 billion total parameters with 13 billion actively routing per token. Running on a laptop-grade 16-core AMD Ryzen 9 9955HX with 64 gigabytes of system memory, the peak memory footprint capped at roughly 58 gigabytes even across an astronomical context window of 256,000 tokens. The system ingested prompts at 44 tokens per second and generated output between 9 and 15 tokens per second—just fast enough to match standard reading speed, assuming the user hasn't downed three double espressos.
Fitting this behemoth into ordinary silicon relied on a double play. The model itself computes like a mid-tier network because each token only queries a fraction of the available expert layers. The author achieved the RAM squeeze by aggressively compressing routed experts down to 1.56 bits per weight using a customized IQ1_S quantization format, which compressed over 284 billion weights into a file footprint hovering just above 50 gigabytes.
The catch lies in the creator's definition of lossless performance, which solely guarantees that raw mathematical output matches an unpatched build running the exact same heavily degraded weights. Compressing network values down to 1.56 bits inevitably degrades reasoning capabilities, and the proprietary files refuse to load inside standard environments like Ollama or LM Studio.
The relentless price gouging across the memory and hardware markets might eventually collide with ruthless community-driven optimization. Whether running a heavily compressed frontier brain locally beats paying monthly subscriptions to cloud providers will decide the future of decentralized computing.
Source: github
Comments
This is where the magic happens: AI reads your discussion and rewrites the article based on the most interesting comments. Each strong comment adds points to the meter below. Once the meter is full, the article updates live — no page reload needed.