Configuring Open WebUI to a Model in Cake Projects

Overview

When you deploy a model in Cake, it runs successfully in Ray Serve, but it will not automatically appear in Open WebUI. To make it available in Open WebUI for testing or demos, you need to add the model’s internal service endpoint manually.

1. Deploy Your Model

1. Go to your Project in Cake.

2. Navigate to the Models tab.

3. Deploy your model (e.g., financerag-deepseek-r1-0528-qwen3-8b).

4. Confirm the model card shows as Running with:

   • Ray Service URL

   • Open WebUI URL

   • Internal & Published Endpoints

✔️ If the model is running here, you’re ready to link it to Open WebUI.

2. Locate the Internal Endpoint

1. On the model card, copy the Internal Endpoint (this is the Ray Serve internal service URL).

Example: http://financerag-deepseek-r1-0528-qwen3-8b.svc.cluster.local:8000/v1

This is what Open WebUI uses to talk to your model.

3. Configure Open WebUI

1. Go to the Open WebUI Admin Settings page: https://open-webui.prod.aws.cake.ai/admin/settings/connections

2. Click Add Connection.

3. Fill in:

   • Endpoint URL: paste the Internal Endpoint from step 2

   • External: ✅ make sure this box is checked (this ensures Open WebUI uses the endpoint correctly)

4. Save the connection.

4. Verify in Open WebUI

1. Open WebUI at your project’s Open WebUI URL (from the model card).

2. Select your newly added model from the dropdown.

3. Run a test message (e.g., hi) to confirm it responds.