Service status

Jump to models

Operational The upstream inference gateway is reachable.

18 responding Every model is probed with one minimal request every 15 minutes; the last sweep ran 7 min ago. “Responding” means the model answered that probe (it checks that the model replies, not that the answer is correct) and it is a recorded result, not a call made when you loaded this page.

These models run on CCS's NVIDIA GH200 Grace Hopper servers. Each server pairs an H100-class GPU (96 GB HBM3) with a 72-core Grace CPU and ~480 GB of LPDDR5X - about 576 GB of unified memory over NVLink-C2C.

Models

13 live · notes updated 2026-10-26 · service status

Reasoning models spend tokens thinking before they answer

A reasoning model uses part of your max_tokens budget on internal reasoning. If the budget is small it can spend ALL of it thinking and return no answer text (the content field comes back empty or null) -- which looks like a broken service but is not. How much each model needs differs a lot; each model's card states the budget measured for it. When in doubt, leave max_tokens unset or switch thinking off where the card says how. The reasoning itself arrives in a separate reasoning_content field; a reply may begin with blank lines, and content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.

chat
General instruction-following. Ask it something, get an answer.
reasoning
Thinks step by step before answering. Better at hard problems, slower, and it spends tokens on the thinking.
vision
Accepts images alongside text.
embeddings
Turns text into vectors for search and RAG. Does NOT answer questions.

Qwen3.6 35B-A3B (no-think)

Responding
chat

Same model, thinking switched off. The fastest thing here.

ccs/Qwen/Qwen3.6-35B-A3B-FP8-nothink
Size35B / 3B per token active ArchMoE Contextabout 390 pages (approximately 253,952 tokens)
no thinkingtoolsimages
Good for
  • Interactive chat and autocomplete where latency is what you feel
  • Claude Code and similar tools that dislike a thinking preamble
  • Short, well-specified tasks that do not need deliberation
Detail, limits and gotchas
Notes
  • Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • Thinking is switched off on this id, so max_tokens is all answer. It is the same model as ccs/Qwen/Qwen3.6-35B-A3B-FP8, which thinks first.
  • Accepts images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. Verified 2026-10-05 on code and terminal screenshots.
  • Measured fastest of all eight: about 0.10s to a short non-streaming reply.
  • Not a different model: the same weights as ccs/Qwen/Qwen3.6-35B-A3B-FP8, with thinking switched off.
Deployment
  • Alias of ccs/Qwen/Qwen3.6-35B-A3B-FP8, same weights, same GPU, reasoning disabled.
  • Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 39msUptime 24h 100.0% (96 checks)

Llama 3.1 8B

Responding
chat

Small, quick, dependable. Built for simple work.

ccs/llama3.1:8b
Size8B ArchDense Contextabout 190 pages (approximately 122,880 tokens)
no tool calling
Good for
  • Classification, extraction, summarising, rewriting
  • High-volume jobs where per-request latency matters
  • A cheap first pass before escalating to a bigger model
Detail, limits and gotchas
Watch out
  • No tool calling: a request that carries tools returns HTTP 400.
  • Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Notes
  • Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • No thinking phase, so what you set as max_tokens is what you get as answer.
  • /v1/completions returns a raw continuation of the text you send, suited to code autocomplete.
Deployment
  • Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 34msUptime 24h 100.0% (96 checks)

Qwen3.6 35B-A3B

Responding
reasoning

Big-model quality at small-model speed.

ccs/Qwen/Qwen3.6-35B-A3B-FP8
Size35B / 3B per token active ArchMoE Contextabout 390 pages (approximately 253,952 tokens)
thinks firsttoolsimages
Good for
  • Agentic and tool-using workflows -- this is what it was built for
  • Coding and code review
  • Anything where you want reasoning quality without a 120B latency bill
Detail, limits and gotchas
Notes
  • Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • Answer budget: it thinks first, and the thinking counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) a 200-line script review came back empty at 4,096 (2 of 6 calls) and nothing was empty from 8,192 up; the longest reply used 7,607 tokens. Allow at least 8,192, more for long reviews, or leave max_tokens unset. For no thinking, use the -nothink id.
  • Tool calling is supported.
  • Accepts images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. Verified 2026-10-05 on code and terminal screenshots.
  • Only 3B of the 35B parameters run per token, which is why it replies quickly for its size.
  • It thinks before answering. Budget max_tokens accordingly -- see the reasoning-token warning below.
  • Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
  • Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it only for assistant turns after your latest message, that is within the current tool loop. To keep it for the whole conversation, send chat_template_kwargs {"preserve_thinking": true} in the request body. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
Deployment
  • 256 experts, 8 routed + 1 shared active per token
  • Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 46msUptime 24h 100.0% (96 checks)

Qwen3.8 27B

Responding
reasoning

Dense 27B that reads images too. Thinks before it answers.

ccs/Qwen/Qwen3.8-27B-FP8
Size27B ArchDense, hybrid attention (Gated DeltaNet + attention), with a vision encoder Contextabout 390 pages (approximately 253,952 tokens)
thinks firsttoolsimages
Good for
  • Reasoning tasks where you want a dense model rather than an MoE
  • Reading screenshots, figures and scanned pages, and answering questions about them
  • Long inputs: its input limit is about 390 pages
  • Tool-using workflows
Detail, limits and gotchas
Notes
  • Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • A very long prompt slows other requests to this model while it is read in: measured 2026-10-05, one ~160,000-token prompt slowed other requests to about 0.4 tokens a second for about 40 seconds.
  • It thinks before answering by default, and the thinking counts inside max_tokens. With thinking on, allow at least 16,384 tokens. Some long code reviews and hard screenshots thought for more than 20,000 tokens in our tests; for those, allow more or leave max_tokens unset. Measured 2026-10-05.
  • If the budget runs out while it is still thinking, the reply is HTTP 200 with empty content and finish_reason "length". Check finish_reason, not only the status code.
  • For short answers, switch thinking off: send chat_template_kwargs {"enable_thinking": false} in the request body. With thinking off, small budgets are fine. reasoning_effort has no effect through this service.
  • Images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. For reading or describing a screenshot, thinking off with max_tokens 2,048 worked well in our tests.
  • Tool calling is supported.
  • Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
  • Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it for every turn of the conversation. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
  • Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Deployment
  • Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 98msUptime 24h 100.0% (96 checks)

DeepSeek-R1 32B (distill)

Responding
reasoning

Reasoning-first. Shows its working.

ccs/deepseek-r1:32b
Size32B ArchDense Contextabout 190 pages (approximately 122,880 tokens)
thinks firstno tool calling
Good for
  • Maths, logic and step-by-step problems
  • Cases where seeing the reasoning is the point
Detail, limits and gotchas
Watch out
  • NO TOOL CALLING. Sending `tools=` returns HTTP 400. This is empirically verified, not a guess -- it is served without tool support.
  • In our tests (2026-10-04), its accuracy at finding specific facts dropped on long documents, and it sometimes gave confident but incorrect answers.
  • Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Notes
  • Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • Answer budget: it always reasons first, and the reasoning counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) no answer came back empty at 4,096 or more; the longest reply used 3,811 tokens. Allow at least 8,192, or leave max_tokens unset.
  • Writes long. It will use the whole max_tokens budget on reasoning if you let it.
  • It always reasons before answering; there is no switch to turn that off.
  • Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
  • Earlier thinking is never kept across turns, even when your client sends it back as reasoning_content: the model sees only its earlier answers. Measured through this service 2026-10-26.
Deployment
  • Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 47msUptime 24h 100.0% (96 checks)

GPT-OSS 120B

Responding
reasoning

The largest model here. Reach for it when quality matters more than speed.

ccs/gpt-oss:120b
Size117B / 5.1B per token active ArchMoE Contextabout 190 pages (approximately 122,880 tokens)
thinks firsttools
Good for
  • Hard reasoning where you can afford the wait
  • Long-form writing and analysis
  • A second opinion when a smaller model looks wrong
Detail, limits and gotchas
Notes
  • Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut. A request whose prompt plus max_tokens goes past the full window is not refused; the reply stops when the window is full (measured 2026-10-04).
  • When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
  • Answer budget: it always reasons first, and the reasoning counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) no answer came back empty at 4,096 or more, but one was cut off at 4,096; the longest reply used 4,510 tokens. Allow at least 8,192, or leave max_tokens unset. There is no switch to turn the reasoning off here.
  • Tool calling is supported.
  • Supports adjustable reasoning effort upstream; not currently exposed as a parameter here.
  • A large model that uses a full GPU on its own.
  • Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
  • Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it only within the current tool loop; thinking from before its last final answer is dropped. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
  • Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Deployment
  • Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 7 min agoLatency 33msUptime 24h 100.0% (96 checks)

Qwen3 Embedding 8B

Responding
embeddings

Turns text into vectors. It does NOT answer questions.

ccs/Qwen/Qwen3-Embedding-8B
Size8B ArchDense embedding model Contextabout 50 pages per item (approximately 33,000 tokens) Dimensions4096 Endpoint/v1/embeddings
Good for
  • RAG -- embed your documents, embed the question, retrieve the closest chunks
  • Semantic search over a corpus
  • Clustering, deduplication, similarity scoring
Detail, limits and gotchas
Watch out
  • Pass `encoding_format: "float"`. It used to be required -- without it the request failed with HTTP 400 -- and since the upstream upgrade on 2026-08-16 it succeeds either way. Keep sending it so your code does not depend on which upstream version happens to be deployed.
  • Sending a chat request to this model returns an error (HTTP 404). Three failed requests to it within a minute put it into a 30-second cooldown for everyone (HTTP 429, "try again in 30 seconds"), so fix the endpoint rather than retrying.
Not for
  • Chat or question answering. It has no answer to give -- it returns numbers, not text.
  • Do not send it to /v1/chat/completions. Use /v1/embeddings. A chat request to this model does not only fail for you -- it takes the model out for everyone for about 30 seconds.
Notes
  • Limits: each input can be up to 32,768 tokens; there is no reply, so nothing is kept back. An input over the limit gets a clear error and no result; nothing is silently cut. The limit is per input, so a batch of many inputs is fine as long as no single one is over.
  • Send at most 256 inputs per request (a 256-input batch measured fine on 2026-10-05; much larger batches can fail).
  • The dimensions parameter is ignored: every vector has 4096 dimensions (verified 2026-10-05).
  • Returns a 4096-dimension vector per input. Size your vector store for that.
  • Typical RAG shape: embed chunks once and store them, embed each query at ask time, retrieve top-k by cosine similarity, then pass those chunks to a chat model as context.
Deployment
  • Limit is about 50 pages per item (approximately 33,000 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.

Upstream model card

Last check 8 min agoLatency 65msUptime 24h 100.0% (96 checks)

Live, not yet described

These are available through the API. We have not written notes for them yet.

Responding ccs/Qwen/Qwen3.8-27B-FP8-nothink Responding ccs/dgx-spark/Qwen/Qwen3-Reranker-4B Responding ccs/dgx-spark/google/gemma-4-26B-A4B-it Responding ccs/dgx-spark/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B Responding ccs/dgx-spark/whisperx/large-v3 Responding ccs/dgx-spark/whisperx/large-v3-turbo

Using the API

The gateway speaks the OpenAI API. Point any OpenAI-compatible client at the base URL below, authenticate with a Bearer token, and pass one of the model ids above as model.

Base URL https://llm.ccs.uky.edu/v1

This host issues no keys to people. For a durable personal key, use Locksmith. If you are running an Open OnDemand session or a batch job, a short-lived key is issued to the job automatically and revoked when the job ends. There is nothing to do.

curl
curl https://llm.ccs.uky.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ccs/llama3.1:8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
Python (OpenAI SDK)
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.ccs.uky.edu/v1",
    api_key="YOUR_KEY",
)

resp = client.chat.completions.create(
    model="ccs/llama3.1:8b",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Never share your key or embed it in published code. To check a key and compare models without installing anything, use Try it. See the full documentation for streaming, embeddings, and client integrations.