Are you concerned about privacy? Adopt an AI (on your computer)
For privacy, cost, and sustainability, in many cases it is advisable to avoid large cloud artificial intelligence tools and keep data at home
BarcelonaEvery time we ask an artificial intelligence bot to summarize, transcribe a meeting, or improve a text, what we write travels to the servers of companies like Google, Microsoft, Anthropic, and OpenAI, is processed in their enormous data centers, and is returned to us in the form of a response. It seems like magic, but it has a cost: economic, environmental, and, above all, privacy. Fortunately, a viable alternative already exists that changes this scheme: running the AI model directly on our own device, without anything leaving home.
The idea of running a large language model (LLM) on our computer might have seemed reserved for programmers. But the reality is that, with the right tools and minimally up-to-date hardware, anyone can have their own AI assistant running completely autonomously. The two main arguments for doing so: the data doesn't go to the cloud and there are no monthly fees to pay.
An apparently free service
Large cloud-based AI services are often offered for free or at a seemingly low price: the usual €20 per month. But the business model that sustains them is, to a large extent, the exploitation of the data you input. Every query you make, every document you upload, every private conversation you transcribe leaves a trace on the servers of companies operating under US legislation or other jurisdictions. Furthermore, the data centers that run cloud AI consume vast amounts of energy and water; a lightweight model executed locally, on the other hand, consumes only the same energy as the computer you already have on.
There are groups that cannot afford to send data to an external cloud: doctors working with clinical records, lawyers managing confidential documentation, executives with sensitive information. They all ask themselves the same question: where is the information stored and for how long? With a local model, the answer is that everything remains on your own hard drive or on the company's server, under the user's control. It also eliminates the dependence on an internet connection and the cost per query or monthly subscription.
Local AI is not a phenomenon exclusive to desktop computers. The What can be done with local AI
A local model does not have the same capacity as cloud-based ones. Small models on your own device are useful for everyday tasks such as summaries, translations, and text writing, while very complex reasoning or massive data analysis may require a remote model: a cloud-based LLM can be between 80 and 150 times larger – in terms of parameters – than a local one.
One of the use cases where local execution is particularly competent is audio transcription. With Whisper, an OpenAI voice recognition model, you can transcribe a meeting or an interview directly to your computer, without the recording leaving the device. Softcatalà has published guides for installing and using Whisper locally, with good performance in Catalan. Other well-adapted tasks include summarizing long documents, improving and correcting texts, and translating content.
Your phone already does it
On-device AI is not an exclusive phenomenon of desktop computers. The latest high-end smartphones already incorporate AI functions processed directly by the device's chip. Google and Samsung paved this way: Pixel phones transcribe audio in real-time without an internet connection; Samsung's Galaxy AI incorporates functions such as advanced photo editing, filtering suspicious calls, and real-time translation, thanks to the Gemini Nano model integrated directly into the operating system.
Apple Intelligence promises the same philosophy and is expected to progressively expand local capabilities, as it aims to make privacy its central differentiating argument.
How to adopt AI
The main requirement to run a language model on your own computer is RAM memory. Softcatalà's heroes have published a very complete guide to install Ollama, an open-source software that simplifies the download, management, and execution of models directly on the user's machine, without relying on external services. Ollama works on Windows, macOS, and Linux, and has a graphical web interface that allows you to dialogue with the model in natural language.
With 8 GB of RAM, Softcatalà recommends installing Google's Gemma 3 model in the 4 billion parameter version; with 16 GB, you can opt for the 12 billion version. This Gemma 3 family is Softcatalà's main recommendation for general use in Catalan, followed by the European Mistral Small model. Speaking of language, it is common for models poorly trained in Catalan to mix the two languages in their responses, but Google's Gemma almost always avoids this problem.
Regarding the computer, the best scenario is Macs with Apple Silicon chips (M1, M2, M3, and later): the unified memory architecture, shared between CPU and GPU, is particularly suitable for running LLMs. It is no coincidence that in recent years industry professionals have massively switched from Linux computers to Macs. On Windows and Linux computers with NVIDIA or AMD graphics cards with sufficient video memory, performance is also very good; without a dedicated graphics card, the model runs on the CPU: it can be done, but it is considerably slower.
On-premise AI is not the solution for all situations: tasks requiring very large models or access to real-time updated information will continue to call for the cloud. But everyday uses are increasingly common – summarizing a document, transcribing an interview, correcting a text – where having AI at home is already an affordable, private, and fee-free option.
