Open Source Cloudless AI

3 minute read

I wanted a local workstation powerful enough to run the largest available AI models on consumer hardware. No proprietary cloud services. No external APIs. Fully autonomous agents for email, calendar, spreadsheets, Telegram, and more. All open source, all local, full privacy and control.

The exact build

Total cost: about $7,000 USD (ignoring the value of my time to assemble it).

Installing and configuring everything

I first installed Ubuntu 25.04 Plucky Puffin. The NVIDIA 5090 driver install was fine. The problem was CUDA on Ubuntu 25 and getting different libraries to actually use it. Package compatibility was not there yet for the stack I needed, so I reinstalled with Ubuntu 24.04 LTS Noble Numbat, which worked better.

I installed Ollama to run local models and added FaceFusion along with a few other tools. With this, large models can live fully in GPU memory.

The cooler LED problem

My Kraken screen was upside down in the open frame. On Linux, liquidctl could read but not write orientation for this device. On Windows, I used NZXT CAM to flip the display.

The Windows detour and WSL

I installed Windows 11 and then the GeForce Game Ready driver. RTX 5090 owners can also see the launch driver note here: Game Ready for RTX 5090 and 5080. For Linux tools inside Windows I enabled Windows Subsystem for Linux and installed Ubuntu on WSL2. This gives me most of what I want from both worlds.

After the Windows install, the Ubuntu partition stopped booting. Fixing GRUB was not worth the time, so I kept Windows for device control and WSL for my Linux workflow.

What I liked on Windows

What I removed or disabled

Where this leaves me

I wish I could run a 5090 as an external GPU on a Mac. Apple documents eGPU support only for Intel Macs with Thunderbolt 3. Apple Silicon does not support eGPU. See Use an external graphics processor with your Mac.

For now my setup is simple. Windows for device control and drivers. WSL with Ubuntu for the tools and scripts I like. CUDA and the NVIDIA driver stack are the core of the local AI workflow. Keep the matrix handy here: CUDA on Linux Guide.

Next steps

Over the coming days I will build a Model Context Protocol (MCP) layer that can send emails through Gmail, run agents on a daily schedule, update Google Sheets, and handle other basic workflows. The goal is to keep everything running only on free and open source software, using Ollama as the orchestration layer.

Updated: