How should I abstract the LLM provider (Ollama → Bedrock/OpenAI)?

Short answer: build a narrow port/adapter (Strategy) around your capability, not around the provider. A small interface plus real implementations selected by config; nothing more.

Your real danger here is over-abstraction: people try to turn this into a generic “AI framework” and burn months on it. All you need is one thin interface.

1. Define the interface by capability, not by vendor. A tiny contract like complete(prompt, opts): Result is enough. Ollama, Bedrock and OpenAI become three implementations of it; config/env picks which one runs. Don’t leak provider-specific params into the core logic — keep the interface small and capability-based.
2. Tolerate differences with explicit timeouts + a cheap fallback. In production the provider slows down or hits a rate limit; put a clear timeout on every call and fall back to a cheap/local option when needed. This frees the architecture from being hostage to one provider’s bad day.
3. Budget tokens at the edge. Each model has a different context window; truncate/summarize the input at the edge so it fits the model. Differences like “small window locally, big in prod” should be config, not an if in business logic.
4. Put streaming behind the same interface. If you need it, keep streaming support optional in the same contract; the caller shouldn’t know the provider’s stream API.
5. Keep a golden-prompt test suite. Run the same prompt set against each provider; automate the output-quality and regression comparison. That’s what catches what broke when you swap providers.

Bottom line: I’d build one small interface, three real implementations and config-based selection — anything more is YAGNI. Local Ollama is for dev cost and fast iteration; treat the production differences (context window, latency, rate limits, cost) as configuration, not as branches in business logic. The narrower the abstraction stays, the sturdier it is.