Hosting Your Own Local LLM with LM Studio, Ollama, and SearXNG

I could have just used ChatGPT, Claude, Gemini, or whatever new model is currently being renamed into a product tier. That would have been easier.

But easy is not always the point.

There is something deeply satisfying about running a model on your own machine, watching the tokens stream back from a server sitting two feet away from you, and knowing that the whole thing works even when the cloud is having a bad day. Local LLMs are not magic, and they are definitely not free once you count the cost of hardware, electricity, and the time you spend debugging weird model names. But they are yours in a way a subscription is not.

For my own setup, I wanted three things:

1. A local model server that speaks an OpenAI-compatible API.
2. A second option in case I want a lighter command-line workflow.
3. A self-hosted search endpoint so the assistant can look things up without depending entirely on a big search API.

That led me to LM Studio, Ollama, and SearXNG.

The best part is that all three can be made to work nicely with askk, my small macOS ask bar. The trick is not the model itself. The trick is giving askk endpoints that match what it expects.

The Shape of the Setup

The architecture is simple:

askk
  -> local LLM server
     -> LM Studio at http://127.0.0.1:1234/v1
     -> or Ollama at http://localhost:11434/v1

askk
  -> SearXNG search server
     -> http://localhost:8080/search?q=...&format=json

The model server handles chat. SearXNG handles web search. askk sits on top and gives me a small, keyboard-first way to use both.

The important part is that the model server must expose:

GET  /v1/models
POST /v1/chat/completions

LM Studio supports those endpoints through its OpenAI-compatible API. Ollama supports them too through its OpenAI compatibility layer. SearXNG just needs JSON output enabled.

Phase 1: Hosting the Model with LM Studio

LM Studio is the friendliest route if you want a desktop app, a model browser, and a local server without turning the whole afternoon into a package-manager side quest.

You install the app, download a model, load it, and start the server. That is the happy path, and for once the happy path is actually pretty close to reality.

Official docs:

• LM Studio Developer Docs
• LM Studio OpenAI-compatible endpoints
• LM Studio REST API

1. Install LM Studio

Download LM Studio from the official site:

https://lmstudio.ai/

Open the app and search for a model. For a general desktop assistant, I usually prefer something that is small enough to respond quickly but still good enough to follow instructions. A 7B or 8B model is a good starting point. If you have a machine with more memory, you can obviously go bigger.

The boring advice is also the correct advice: start smaller than you think. A model that responds quickly is more useful than a giant model you avoid using because every answer feels like waiting for a render.

2. Download and Load a Model

Inside LM Studio:

1. Go to the model search area.
2. Download a model that fits your machine.
3. Load the model into memory.
4. Keep the model loaded before testing the API.

This last step matters. If askk can reach the server but no model is loaded, the connection can still feel broken from the user's side. The model server is up, but there is nobody home.

3. Start the Local Server

In LM Studio, start the local server. The usual local address is:

http://localhost:1234

LM Studio's OpenAI-compatible base URL is:

http://localhost:1234/v1

You can test it with:

curl http://localhost:1234/v1/models

If you get a JSON response with model data, the server is alive.

Then test chat completions:

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "use-the-model-id-from-lm-studio",
    "messages": [
      { "role": "user", "content": "Say hello in one sentence." }
    ],
    "temperature": 0.7
  }'

The model ID has to match what LM Studio reports. This is one of those tiny details that causes a lot of fake debugging. The server can be fine, the endpoint can be fine, and the whole thing can still fail because the model name is not the one the server knows.

Optional: Run LM Studio Headless

LM Studio also has a headless path through llmster and the lms command. This is useful if you want a server-like setup instead of clicking around the app.

The LM Studio docs show this install path:

curl -fsSL https://lmstudio.ai/install.sh | bash

Then the basic flow is:

lms daemon up
lms get <model>
lms server start --port 1234

I would still start with the desktop app first. Once you know which models work well on your machine, moving to the headless route is much less mysterious.

Make LM Studio Compatible with askk

askk expects an OpenAI-compatible model server. LM Studio is the most direct match.

In askk, use this model endpoint:

http://127.0.0.1:1234

or:

http://localhost:1234

askk normalizes the address and talks to the /v1 API. So entering the base server address is enough. Under the hood, askk checks:

http://127.0.0.1:1234/v1/models

and sends chat messages to:

http://127.0.0.1:1234/v1/chat/completions

If the connection test fails, check these in order:

1. LM Studio server is running.
2. A model is loaded.
3. The server is on port 1234.
4. The model appears in GET /v1/models.

This is the cleanest setup for askk today because it is exactly the API shape askk was built around.

Phase 2: Hosting the Model with Ollama

Ollama feels different from LM Studio. LM Studio is a workshop with buttons and model cards. Ollama is more like a small engine you start from the terminal and forget about until you need it.

That can be a good thing.

Official docs:

• Ollama Quickstart
• Ollama CLI Reference
• Ollama API Introduction
• Ollama OpenAI compatibility

1. Install Ollama

Download Ollama from the official site:

https://ollama.com/download

After installing, open a terminal and check that it responds:

ollama

You can also start the service manually:

ollama serve

On many desktop installs, Ollama runs in the background automatically. But knowing ollama serve exists is helpful when you are trying to figure out whether the server is actually running or you are just shouting at an empty port.

2. Pull a Model

Pick a model and pull it:

ollama pull llama3.2

Or use another model you prefer:

ollama pull qwen3:8b

Then run it once:

ollama run llama3.2

If that opens an interactive chat, the local model side is working.

3. Use the Native Ollama API

Ollama's native API is served at:

http://localhost:11434/api

A simple native API test looks like this:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    { "role": "user", "content": "Say hello in one sentence." }
  ],
  "stream": false
}'

This is useful for Ollama-native apps, but it is not the endpoint shape askk wants.

For askk, the more important piece is Ollama's OpenAI-compatible API.

4. Use Ollama's OpenAI-Compatible API

Ollama also exposes OpenAI-style endpoints at:

http://localhost:11434/v1

Test the model list:

curl http://localhost:11434/v1/models

Then test chat completions:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2",
    "messages": [
      { "role": "user", "content": "Say hello in one sentence." }
    ],
    "temperature": 0.7
  }'

The OpenAI compatibility docs also mention that some tools expect OpenAI-ish names like gpt-3.5-turbo. If you ever run into a tool that hardcodes a model name, Ollama can copy a model to a new local alias:

ollama cp llama3.2 gpt-3.5-turbo

For askk, you usually do not need that. askk asks the server for available models and uses what it finds.

Make Ollama Compatible with askk

In askk, use this model endpoint:

http://localhost:11434

askk will normalize it to:

http://localhost:11434/v1

That means askk can call:

GET  http://localhost:11434/v1/models
POST http://localhost:11434/v1/chat/completions

If askk says the model is unavailable, pull at least one model first:

ollama pull llama3.2

If askk can list models but responses are poor or slow, the problem is probably not the connection. It is usually the model choice, quantization, context size, or your machine's available memory.

For a smoother askk experience, I would choose a model that can answer quickly over the biggest model your machine can technically load. A local assistant should feel close at hand. If every small question turns into a benchmark, you will stop asking it things.

Phase 3: Configuring SearXNG for Search

The model is only half the story. Local models are great, but they do not magically know what happened five minutes ago. If you want current information, you need search.

SearXNG is a good fit because it is self-hostable, does not require you to build a search index from scratch, and can return JSON results that an app like askk can feed into the model.

Official docs:

• SearXNG Documentation
• SearXNG container installation
• SearXNG search settings
• SearXNG Search API

1. Start with Docker Compose

The official SearXNG docs recommend Compose for containerized installs. In this repo, I already keep a small compose setup under searxng/, and it follows the same general shape:

services:
  core:
    image: docker.io/searxng/searxng:latest
    ports:
      - 8080:8080
    volumes:
      - ./core-config/:/etc/searxng/

  valkey:
    image: docker.io/valkey/valkey:9-alpine

The practical version is:

cd searxng
cp .env.example .env
mkdir -p core-config
docker compose up -d

Then open:

http://localhost:8080

If the web UI loads, the service is running.

2. Enable JSON Output

This is the important askk detail.

SearXNG does not necessarily allow every output format by default. The official Search API docs say that JSON, CSV, and RSS output depend on the formats enabled in settings.yml; if a requested format is not enabled, SearXNG can return 403 Forbidden.

In your SearXNG config file:

searxng/core-config/settings.yml

make sure the search.formats list includes json:

search:
  formats:
    - html
    - json

Then restart:

docker compose restart core

Test JSON search:

curl "http://localhost:8080/search?q=askk&format=json"

If you see JSON with a results array, you are in business.

3. Tune Engines Without Turning It Into a Religion

SearXNG can aggregate a lot of engines. That is both its strength and the source of many little annoyances. Some engines rate-limit aggressively. Some return weird snippets. Some work beautifully for a week and then start throwing errors.

My advice is to start boring:

search:
  safe_search: 0
  formats:
    - html
    - json

Then tune engines only after the basic loop works.

For a personal instance, I would avoid exposing it publicly unless I had a real reason. If you put it on the internet, you need to think about rate limits, bot protection, abuse, logs, reverse proxy headers, and updates. For askk, a local instance is enough:

http://localhost:8080

That keeps the moving parts smaller.

4. Update the SearXNG Container

When you want to update:

docker compose down
docker compose pull
docker compose up -d

The SearXNG docs also recommend reviewing updated compose templates when you update, because container settings can change over time.

This is not the fun part of self-hosting, but it is the part that keeps future-you from wondering why something broke six months later.

Make SearXNG Compatible with askk

askk expects a SearXNG base URL, not the full search path.

Use:

http://localhost:8080

askk builds the search URL itself:

http://localhost:8080/search?q=<query>&format=json&categories=general

So do not paste /search?q=... into askk. Just paste the base address.

The required SearXNG setting is:

search:
  formats:
    - html
    - json

In askk's settings:

1. Set Web Search to Auto or Always.
2. Set SearXNG host to localhost.
3. Set port to 8080.
4. Leave HTTPS off for a local Docker instance.
5. Test the connection.

If askk shows no results, test this directly:

curl "http://localhost:8080/search?q=test&format=json&categories=general"

If that returns 403, JSON is not enabled. If it returns an empty result set, your SearXNG instance is reachable but the configured engines are not giving useful results. If it does not connect at all, the container probably is not running or the port mapping is different.

The Setup I Would Actually Use

If I wanted the least painful local askk setup today, I would start with this:

LM Studio
  server: http://127.0.0.1:1234
  API:    OpenAI-compatible /v1

SearXNG
  server: http://localhost:8080
  JSON:   enabled in settings.yml

Then, once that works, I would add Ollama as the second model backend:

Ollama
  server: http://localhost:11434
  API:    OpenAI-compatible /v1

LM Studio is easier to see and debug. Ollama is easier to script and keep around. SearXNG gives the whole setup a way to look outside the model's training data.

None of this makes local AI perfect. Smaller models hallucinate. Bigger models are slow. Search results can be noisy. Docker will occasionally remind you that computers are just a pile of agreements held together by logs.

But when the setup works, it feels different. You press a shortcut, ask a question, and the answer comes from your own machine, with your own search instance nearby. It is not fully independent from the internet, but it is a lot less rented than the usual AI workflow.

And that is the part I like.

Not because it is the most practical option every time. It is not.

But because it makes the computer feel like mine again.

Quick Reference

LM Studio

# Base URL for askk
http://127.0.0.1:1234

# What askk calls internally
http://127.0.0.1:1234/v1/models
http://127.0.0.1:1234/v1/chat/completions

Docs:

• LM Studio Developer Docs
• OpenAI Compatibility Endpoints

Ollama

ollama pull llama3.2
ollama run llama3.2

# Base URL for askk
http://localhost:11434

# What askk calls internally
http://localhost:11434/v1/models
http://localhost:11434/v1/chat/completions

Docs:

• Ollama Quickstart
• Ollama OpenAI compatibility

SearXNG

search:
  formats:
    - html
    - json

# Base URL for askk
http://localhost:8080

# What askk calls internally
http://localhost:8080/search?q=<query>&format=json&categories=general

Docs:

• SearXNG container installation
• SearXNG search settings
• SearXNG Search API