How to adopt an AI on your computer and why it's worth doing
For privacy, cost and sustainability, in many cases it is convenient to avoid large cloud artificial intelligence tools and keep data at home
BarcelonaEvery time we ask an artificial intelligence conversational bot to summarize, transcribe a meeting, or improve a text, what we write travels to servers of companies like Google, Microsoft, Anthropic, and OpenAI, is processed in their enormous data centers, and is returned to us in the form of a response. It seems like magic, but it has a cost: economic, environmental, and, above all, privacy. Fortunately, a viable alternative already exists that flips this scheme: running the AI model directly on our own device, without anything leaving home.
The idea of running a large language model (LLM) on our computer might have seemed reserved for programmers. But the reality is that, with the right tools and minimally up-to-date hardware, anyone can have their own AI assistant running completely autonomously. The two main arguments for doing so: data doesn't go to the cloud and there's no need to pay monthly fees.
An only apparent gratuity
Large AI services in the cloud are usually offered for free or at a seemingly low price: the usual €20 per month. But the business model that sustains them is, to a large extent, the exploitation of the data you enter. Every query you make, every document you upload, every private conversation you transcribe leaves a trace on servers of companies operating under US legislation or other jurisdictions. Furthermore, the data centers that run cloud AI consume enormous amounts of energy and water; a lightweight model run locally, on the other hand, consumes only the same energy as the computer you already have running.
There are groups that cannot afford to send data to an external cloud: doctors working with clinical histories, lawyers managing confidential documentation, executives with sensitive information. They all ask themselves the same question: where is the information stored and for how long? With a local model, the answer is that everything stays on your own hard drive or the company's server, under the user's control. It also eliminates the dependence on an internet connection and the cost per query or monthly subscription.
On-device AI is not an exclusive phenomenon of desktop computers. The What can be done with local AI
A local model does not have the same capacity as cloud-based ones. Small models on your own device are useful for everyday tasks such as summaries, translations, and text writing, while very complex reasoning or massive data analysis may require a remote model: a cloud-based LLM can be between 80 and 150 times larger – in terms of parameters – than a local one.
A local execution is particularly competent in audio transcription. With Whisper, a voice recognition model from OpenAI, you can transcribe a meeting or an interview directly on your computer, without the recording leaving the device. Softcatalà has published guides to install and use Whisper locally, with good performance in Catalan. Other well-adapted tasks include summarizing long documents, improving and correcting texts, and translating content.
Your phone already does it
On-device AI is not a phenomenon exclusive to desktop computers. The latest high-end smartphones already incorporate AI functions processed directly by the device's chip. Google and Samsung paved the way: Pixel phones transcribe audio in real-time without an internet connection; Samsung's Galaxy AI incorporates features such as advanced photo editing, filtering suspicious calls, and real-time translation, thanks to the Gemini Nano model integrated directly into the operating system.
Apple Intelligence promises the same philosophy and is expected to progressively expand local capabilities, as it aims to make privacy its central differentiating argument.
How to adopt AI
The main requirement for running a language model on your own computer is RAM memory. The heroes of Softcatalà have published a very complete guide to install Ollama, an open-source software that simplifies the download, management, and execution of models directly on the user's machine, without relying on external services. Ollama works on Windows, macOS, and Linux, and has a web-based graphical interface that allows you to interact with the model in natural language.
With 8 GB of RAM, Softcatalà recommends installing Google's Gemma 3 model in the 4 billion parameter version; with 16 GB, you can opt for the 12 billion version. This Gemma 3 family is Softcatalà's main recommendation for general use in Catalan, followed by the European Mistral Small model. Speaking of language, it is common for models not well-trained in Catalan to mix the two languages in their responses, but Google's Gemma almost always avoids this problem.
Regarding the computer, the best scenario is that of Macs with Apple Silicon chips (M1, M2, M3 and later): the unified memory architecture, shared between CPU and GPU, is particularly suitable for running LLMs. It's no coincidence that in recent years industry professionals have massively switched from Linux computers to Macs. On Windows and Linux computers with NVIDIA or AMD graphics cards with enough video memory, performance is also very good; without a dedicated graphics card, the model runs on the CPU: it can be done, but it is considerably slower.
Local AI is not the solution for all situations: tasks requiring very large models or access to real-time updated information will continue to demand the cloud. But increasingly, there are more and more everyday uses – summarizing a document, transcribing an interview, correcting a text – where having AI at home is already an affordable, private, and subscription-free option.