Building a Kong gateway foundation for APIs, LLMs and MCP

From ingress controller to integration boundary

Kong has been part of my Kubernetes homelab for some time. Initially, its role was relatively conventional: terminate TLS, provide ingress and apply policies before traffic reached a workload.

That role has changed.

As I added LLM integrations, MCP servers and automation around n8n and OpenClaw, I started seeing the same problem from different directions. Each new integration could easily introduce another direct connection, another credential and another piece of application-specific configuration.

That does not scale particularly well.

I now use Kong as a common integration boundary between consumers and three different types of services:

These are three logical gateway boundaries running on the same Kong platform.

The distinction is important. I do not want every application to understand how RouterOS authentication works, where an LLM provider lives or how an MCP server is secured. Consumers should understand the gateway contract.

Kong deals with what happens behind that contract.

The architecture

The current architecture looks roughly like this:

The solid paths are the integrations I am already using. The dashed n8n paths represent where I want to take this foundation next. n8n already has controlled network access towards Kong, so the gateway becomes the natural integration point as I add more automation.

This starts to give me something much more useful than a collection of individual applications.

Open WebUI provides interaction with models and tools. OpenClaw can reason about events and use the LLM gateway. n8n provides workflow orchestration. Gatus performs deterministic monitoring. Kong sits between those consumers and the capabilities they need.

One runtime, three clear contracts

I deliberately separated the configuration into API, LLM and MCP responsibilities:

services/gateway/kong/
|-- api-gateway/
|   `-- routeros-upstream/
|-- bootstrap/
|-- core/
|-- llm-gateway/
`-- mcp-gateway/
    |-- kubernetes-upstream/
    `-- nmap-upstream/

core contains the shared Kong capability: Helm deployment, TLS, IngressClass, network policy and common configuration.

Everything above that belongs to a logical gateway.

This means an upstream integration can own the Kubernetes Service abstraction, route, Kong plugins, consumer configuration and required secrets together.

That makes the repository structure reflect the architecture.

It also gives me a repeatable onboarding pattern. A new service gets a gateway contract, an upstream definition, its authentication and transformation policy, required secrets and finally a consumer path. Flux then reconciles that configuration instead of me manually modifying a running gateway.

RouterOS became the API gateway use case

RouterOS is a good example of why I wanted this architecture.

Gatus performs checks against my MikroTik devices. Technically, I could allow Gatus to connect directly to the RouterOS REST API and provide it with RouterOS credentials.

I deliberately do not want that relationship.

The current flow is:

Gatus
  |
  | HTTPS + Kong consumer credential
  v
api-gw.${DOMAIN}/mikrotik/...
  |
  | authentication
  | rate limiting
  | request transformation
  v
Kong
  |
  | RouterOS authentication
  v
MikroTik RouterOS REST API

Gatus now calls the same api-gw contract that another authorized workload could use later.

The RouterOS credential stays behind the gateway. Kong retrieves the required configuration through External Secrets and injects the upstream authentication when forwarding the request. The consumer does not need the RouterOS username and password.

That changes the trust relationship.

Instead of:

workload -> RouterOS

I now have:

workload -> controlled gateway -> RouterOS

It also means that access policy, rate limiting, logging and eventually identity can be enforced at the integration boundary.

Gatus itself is now a Kong consumer and its WireGuard monitoring requests use:

https://api-gw.${DOMAIN}/mikrotik/...

rather than addressing Kong through an internal Kubernetes service URL.

For me, this is one of the most useful outcomes of the refactoring. RouterOS becomes a protected upstream service instead of something every workload needs to know how to access.

The LLM gateway hides the provider

The LLM boundary solves a similar problem at another layer.

Open WebUI and OpenClaw consume an OpenAI-compatible endpoint through:

https://llm-gw.${DOMAIN}/openai/v1

The consumer knows the gateway.

It does not need to own the upstream provider integration.

Behind that endpoint I can apply authentication, model mapping, request transformation and other AI-specific gateway policy. The Kong configuration already contains model normalization and the different paths needed for my AI integrations.

This gives me an abstraction I increasingly value.

The application asks for an LLM capability. The gateway decides how that capability is delivered.

That gives me room to experiment with different models and providers without making every workflow dependent on their individual API implementation.

For n8n in particular, this becomes interesting. A workflow should eventually be able to call llm-gw as just another controlled internal capability instead of carrying provider-specific credentials and URLs.

MCP turns tools into another service boundary

The third boundary is mcp-gw.

I currently have two MCP use cases behind it:

mcp-gw.${DOMAIN}/kubernetes
mcp-gw.${DOMAIN}/nmap

The Kubernetes MCP server exposes controlled Kubernetes functionality while the Nmap MCP server exposes network discovery capabilities.

Open WebUI can use both as tools.

This creates another useful separation. The AI-facing application does not have to communicate directly with the MCP workloads. It talks to the gateway and Kong applies the required authentication and routing.

There is even a small protocol translation involved in the current configuration. Open WebUI sends its MCP credential as an Authorization: Bearer token while the Kong configuration uses key authentication internally. A Kong pre-function converts that representation at the boundary.

That detail belongs in the gateway.

It should not become application logic.

GitOps also defines the dependency order

One of the less visible improvements is how the configuration is reconciled.

Some Kong resources reference credentials generated through External Secrets. Kong’s admission process can validate those references before the generated Kubernetes Secret exists.

That creates a dependency problem during a clean deployment.

I now handle this explicitly with separate Flux bootstrap stages.

For example, the RouterOS bootstrap waits until the required ExternalSecrets are ready. Only then are the RouterOS route, plugins and Gatus consumer applied. MCP follows the same model: its bootstrap secrets are reconciled first and the MCP gateway resources follow afterwards.

This is more than making Flux happy.

It means the Git repository describes both the desired configuration and the order required to safely construct it.

That matters if I want the environment to remain reproducible.

The gateway is becoming my integration layer

What interests me most about this setup is not Kong itself.

It is what the gateway now enables.

I am gradually creating three reusable capabilities:

API  -> interact with systems
LLM  -> reason about information
MCP  -> expose controlled tools

Applications and automation can combine them without requiring direct knowledge of every backend.

An n8n workflow could receive an event, use the LLM gateway to interpret it, retrieve information through an MCP tool and finally execute an action through an API exposed behind api-gw.

OpenClaw can use the same foundation for agent-style workflows.

Open WebUI provides an interactive interface on top of the same LLM and MCP capabilities.

Gatus already demonstrates that this model is not AI-specific at all. It consumes a normal API through the same controlled gateway pattern.

That is the part I am most interested in developing further. The Kubernetes cluster is no longer only hosting individual services. It is starting to expose reusable capabilities that I can compose into personal automation and, more importantly, use to experiment with patterns that are also relevant in professional environments.

Static API keys are the next boundary to remove

The current model still has one thing I want to improve: static consumer keys.

They work, but operationally they introduce another secret lifecycle.

A credential needs to be generated, stored in Vault, synchronized through External Secrets and made available to the workload. Rotation means coordinating those components again.

My next step is therefore machine-to-machine identity using an OIDC client credentials flow.

Instead of:

workload
   |
   | long-lived API key
   v
Kong

I want to move towards:

workload
   |
   | client identity
   v
Identity Provider
   |
   | short-lived access token
   v
Kong
   |
   | policy based on identity / audience / scope
   v
upstream service

That would move the gateway from authenticating possession of a shared secret towards authenticating a workload identity.

It would also reduce the amount of API-key material I need to distribute through Vault and External Secrets Operator.

The API, LLM and MCP gateway boundaries give me a useful place to introduce that capability because the consumers already depend on stable gateway contracts rather than directly on the upstream systems.

Way forward

The important part is that I now have somewhere to add these controls.

I do not have to redesign the relationship between every application and every backend independently.

Kong has become the control point.

A foundation rather than another service

This started as a cleanup of my Kong configuration.

It ended up making the architecture much clearer.

I now have one shared Kong runtime with three explicit contracts for APIs, LLMs and MCP services. RouterOS is available through a controlled API boundary, my LLM clients no longer need to depend directly on model providers, and Kubernetes and Nmap capabilities are exposed through an MCP gateway.

More importantly, I have a repeatable place to onboard the next service.

That is the foundation I was looking for.

The interesting work from here is not adding more gateway configuration. It is using these capabilities together: automation in n8n, reasoning through LLMs, controlled tool execution through MCP and protected access to APIs.

And eventually, replacing static gateway keys with workload identity.

The implementation and refactoring are documented in homelabs PR #234.

Seeing the gateway path from Open WebUI

The architecture is useful on paper, but I also want to see the complete path working from a real consumer. Open WebUI gives me that validation: the conversation uses the LLM through Kong while tool calls are routed through the MCP gateway to the individual tool servers.

Open WebUI uses the LLM path together with the Kubernetes MCP server to retrieve a live overview of the four-node cluster.

The same setup can hand a request to the Nmap MCP server. In this example it scans k8s-master01 and identifies the services exposed on the node. A follow-up request for networknet.nl is rejected by the tool’s target restrictions, which is also useful evidence: exposing a tool through MCP does not mean giving it unrestricted reach.

The Nmap MCP integration scans an internal Kubernetes node, while an external-domain request is rejected by the tool boundary.