Open-Weight vs Closed Models: A Practical Decision Guide for Teams Building with LLMs

A few years ago, building with large language models meant one thing: sending requests to a handful of proprietary APIs. Today the landscape is far more varied. Families of open-weight models, from Meta's Llama series to Mistral's releases and many others hosted on Hugging Face, have become capable enough for a wide range of production tasks. Meanwhile, closed frontier models continue to lead on many of the hardest reasoning and multimodal benchmarks.

So the question I hear most often from developers I mentor is not "can AI do this?" but "which kind of model should we bet on?" It is a good question, and it deserves a better answer than tribal loyalty. Let us start by clarifying terms.

An open-weight model is one whose trained parameters you can download and run yourself. That does not automatically make it open source in the traditional sense. Many open-weight models ship with custom licences that restrict certain uses, impose user-count thresholds or forbid using outputs to train competing models. Training data and full training code are frequently not released.

The Open Source Initiative has worked on a definition of open source AI that addresses this gap, emphasising access to the information needed to study and modify a system, not just the weights. For builders, the practical lesson is simple: read the licence before you build a business on a model. "Free to download" and "free to use commercially in any way" are different claims.

A closed model, by contrast, is accessed only through a provider's API or managed platform. You never see the weights; you rent capability per token or per request, under the provider's terms of service.

The differences show up across the whole stack:

• Customisation: with open weights you can fine-tune deeply. Parameter-efficient methods like LoRA train small adapter matrices instead of all weights, and QLoRA combines this with 4-bit quantisation so that sizeable models can be fine-tuned on a single GPU. Closed providers increasingly offer fine-tuning too, but within the limits they choose.
• Inference control: self-hosting lets you choose quantisation levels, batching strategies, context lengths and serving frameworks. You can trade a little quality for large gains in speed and cost.
• Data flow: with self-hosting, prompts and documents never leave your infrastructure. With an API, they transit to the provider, subject to its data retention and processing terms.
• Capability ceiling: closed frontier models often still hold an edge on complex multi-step reasoning, long-context synthesis and multimodal tasks, although the gap varies by task and changes with every release cycle.
• Operational burden: an API is someone else's on-call rotation. Self-hosting means you own GPU capacity planning, autoscaling, model updates, security patches and monitoring.

Technical reports such as the Llama 2 paper and the Mistral 7B paper are worth reading not just for benchmarks but for how they describe architecture choices like grouped-query attention, which directly affect serving cost.

Rather than asking which approach is better, score your use case on a few dimensions.

1. Data sensitivity and residency. If you handle patient records, financial data or government information with strict residency rules, self-hosting or a regionally hosted deployment may be the only acceptable option. In health tech, this consideration often dominates everything else.
2. Task difficulty. Classification, extraction, summarisation of structured documents and routing are often handled well by smaller open models, especially after fine-tuning. Open-ended reasoning across messy inputs may justify a frontier API.
3. Volume and cost curve. APIs are cheap at low volume and have no fixed cost. Self-hosting has high fixed cost but low marginal cost once GPUs are well utilised. Model the crossover point with your real traffic, not guesses.
4. Latency and connectivity. For offline-first or on-premises deployments, which are common in parts of Africa where connectivity is unreliable, a local model may be a requirement rather than a preference.
5. Team maturity. If nobody on the team has run GPU workloads in production, factor in the learning curve honestly.
6. Vendor risk. APIs can change pricing, deprecate models or alter behaviour. Open weights you have downloaded will behave the same tomorrow.

In practice, many mature systems use both. A common architecture puts a router in front of several models: a small, fast, self-hosted model handles the bulk of routine requests, while hard or ambiguous cases escalate to a frontier API. Embeddings and rerankers, which are often small, are frequently self-hosted even when generation is not.

def route(request):
    if request.contains_sensitive_data:
        return local_model          # never leaves our VPC
    if classifier.difficulty(request) < 0.6:
        return local_model          # cheap and fast
    return frontier_api             # reserved for hard cases

The key enabler is an abstraction layer. Wrap every model behind a common interface, keep prompts in version control and maintain an evaluation suite that you can run against any candidate model. When a new open model is released, you can benchmark it on your tasks in an afternoon instead of debating it for weeks.

Both paths carry risk. Open weights can be fine-tuned to remove safety behaviours, which places more responsibility on whoever deploys them; you inherit the job of content filtering and abuse monitoring. Licences may also change between versions. Closed models concentrate power in a few companies, can create dependency on foreign infrastructure and may be priced in currencies that hurt teams earning in naira or cedis.

There is also a sovereignty dimension. For African governments and enterprises, the ability to run capable models on local infrastructure, adapt them to local languages and keep data within national borders is increasingly seen as strategic. Open-weight models make that possible in a way that pure API access does not, but only if local compute and skills exist to use them.

• Open-weight means downloadable parameters, not necessarily open source; always read the licence.
• Decide on data sensitivity, task difficulty, volume, latency, team maturity and vendor risk, not on hype.
• Parameter-efficient fine-tuning such as LoRA and QLoRA makes customising open models accessible to small teams.
• Most production systems benefit from a hybrid router that sends routine work to small models and hard cases to frontier APIs.
• Build a model-agnostic abstraction and your own evaluation suite so switching models is cheap.
• For emerging markets, open weights support data residency, offline deployment and local adaptation.

1. LoRA: Low-Rank Adaptation of Large Language Models — arXiv
2. QLoRA: Efficient Finetuning of Quantized LLMs — arXiv
3. Llama 2: Open Foundation and Fine-Tuned Chat Models — arXiv
4. Mistral 7B — arXiv
5. Open Source AI — Open Source Initiative
6. Hugging Face Hub — Hugging Face

Cover image: original illustration created for this article. Cited works remain the property of their respective authors and publishers.