OperationalThe upstream inference gateway is reachable.
18 respondingEvery model is probed with one minimal request every 15 minutes;
the last sweep ran 7 min ago. “Responding”
means the model answered that probe (it checks that the model
replies, not that the answer is correct) and it
is a recorded result, not a call made when you loaded this page.
These models run on CCS's NVIDIA GH200 Grace Hopper servers. Each server pairs an H100-class GPU (96 GB HBM3) with a 72-core Grace CPU and ~480 GB of LPDDR5X - about 576 GB of unified memory over NVLink-C2C.
Reasoning models spend tokens thinking before they answer
A reasoning model uses part of your max_tokens budget on internal reasoning. If the budget is small it can spend ALL of it thinking and return no answer text (the content field comes back empty or null) -- which looks like a broken service but is not. How much each model needs differs a lot; each model's card states the budget measured for it. When in doubt, leave max_tokens unset or switch thinking off where the card says how. The reasoning itself arrives in a separate reasoning_content field; a reply may begin with blank lines, and content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
chat
General instruction-following. Ask it something, get an answer.
reasoning
Thinks step by step before answering. Better at hard problems, slower, and it spends tokens on the thinking.
vision
Accepts images alongside text.
embeddings
Turns text into vectors for search and RAG. Does NOT answer questions.
Qwen3.6 35B-A3B (no-think)
Responding
chat
Same model, thinking switched off. The fastest thing here.
Interactive chat and autocomplete where latency is what you feel
Claude Code and similar tools that dislike a thinking preamble
Short, well-specified tasks that do not need deliberation
Detail, limits and gotchas
Notes
Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
Thinking is switched off on this id, so max_tokens is all answer. It is the same model as ccs/Qwen/Qwen3.6-35B-A3B-FP8, which thinks first.
Accepts images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. Verified 2026-10-05 on code and terminal screenshots.
Measured fastest of all eight: about 0.10s to a short non-streaming reply.
Not a different model: the same weights as ccs/Qwen/Qwen3.6-35B-A3B-FP8, with thinking switched off.
Deployment
Alias of ccs/Qwen/Qwen3.6-35B-A3B-FP8, same weights, same GPU, reasoning disabled.
Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
High-volume jobs where per-request latency matters
A cheap first pass before escalating to a bigger model
Detail, limits and gotchas
Watch out
No tool calling: a request that carries tools returns HTTP 400.
Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Notes
Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
No thinking phase, so what you set as max_tokens is what you get as answer.
/v1/completions returns a raw continuation of the text you send, suited to code autocomplete.
Deployment
Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
Agentic and tool-using workflows -- this is what it was built for
Coding and code review
Anything where you want reasoning quality without a 120B latency bill
Detail, limits and gotchas
Notes
Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
Answer budget: it thinks first, and the thinking counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) a 200-line script review came back empty at 4,096 (2 of 6 calls) and nothing was empty from 8,192 up; the longest reply used 7,607 tokens. Allow at least 8,192, more for long reviews, or leave max_tokens unset. For no thinking, use the -nothink id.
Tool calling is supported.
Accepts images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. Verified 2026-10-05 on code and terminal screenshots.
Only 3B of the 35B parameters run per token, which is why it replies quickly for its size.
It thinks before answering. Budget max_tokens accordingly -- see the reasoning-token warning below.
Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it only for assistant turns after your latest message, that is within the current tool loop. To keep it for the whole conversation, send chat_template_kwargs {"preserve_thinking": true} in the request body. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
Deployment
256 experts, 8 routed + 1 shared active per token
Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
Last check 7 min agoLatency 46msUptime 24h 100.0% (96 checks)
Qwen3.8 27B
Responding
reasoning
Dense 27B that reads images too. Thinks before it answers.
ccs/Qwen/Qwen3.8-27B-FP8
Size27BArchDense, hybrid attention (Gated DeltaNet + attention), with a vision encoderContextabout 390 pages (approximately 253,952 tokens)
thinks firsttoolsimages
Good for
Reasoning tasks where you want a dense model rather than an MoE
Reading screenshots, figures and scanned pages, and answering questions about them
Long inputs: its input limit is about 390 pages
Tool-using workflows
Detail, limits and gotchas
Notes
Limits: full window 262,144 tokens; input limit 253,952 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
A very long prompt slows other requests to this model while it is read in: measured 2026-10-05, one ~160,000-token prompt slowed other requests to about 0.4 tokens a second for about 40 seconds.
It thinks before answering by default, and the thinking counts inside max_tokens. With thinking on, allow at least 16,384 tokens. Some long code reviews and hard screenshots thought for more than 20,000 tokens in our tests; for those, allow more or leave max_tokens unset. Measured 2026-10-05.
If the budget runs out while it is still thinking, the reply is HTTP 200 with empty content and finish_reason "length". Check finish_reason, not only the status code.
For short answers, switch thinking off: send chat_template_kwargs {"enable_thinking": false} in the request body. With thinking off, small budgets are fine. reasoning_effort has no effect through this service.
Images: pass an image_url content part, the standard OpenAI vision shape; a base64 data URI works. For reading or describing a screenshot, thinking off with max_tokens 2,048 worked well in our tests.
Tool calling is supported.
Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it for every turn of the conversation. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Deployment
Limit is about 390 pages (approximately 253,952 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
NO TOOL CALLING. Sending `tools=` returns HTTP 400. This is empirically verified, not a guess -- it is served without tool support.
In our tests (2026-10-04), its accuracy at finding specific facts dropped on long documents, and it sometimes gave confident but incorrect answers.
Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Notes
Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut.
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
Answer budget: it always reasons first, and the reasoning counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) no answer came back empty at 4,096 or more; the longest reply used 3,811 tokens. Allow at least 8,192, or leave max_tokens unset.
Writes long. It will use the whole max_tokens budget on reasoning if you let it.
It always reasons before answering; there is no switch to turn that off.
Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
Earlier thinking is never kept across turns, even when your client sends it back as reasoning_content: the model sees only its earlier answers. Measured through this service 2026-10-26.
Deployment
Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
Limits: full window 131,072 tokens; input limit 122,880 tokens (input only, the whole conversation added together); 8,192 tokens are kept for the reply. A prompt over the input limit gets a clear "too long" error (HTTP 400); nothing is silently cut. A request whose prompt plus max_tokens goes past the full window is not refused; the reply stops when the window is full (measured 2026-10-04).
When it is busy, requests wait their turn rather than being refused; a second request near the full window waits for the first. Nothing is cut.
Answer budget: it always reasons first, and the reasoning counts inside max_tokens. In our 2026-10-05 test (three text tasks, two runs each) no answer came back empty at 4,096 or more, but one was cut off at 4,096; the longest reply used 4,510 tokens. Allow at least 8,192, or leave max_tokens unset. There is no switch to turn the reasoning off here.
Tool calling is supported.
Supports adjustable reasoning effort upstream; not currently exposed as a parameter here.
A large model that uses a full GPU on its own.
Reasoning arrives in a separate reasoning_content field, not in content. A reply may begin with blank lines. content can be null when the reply carries a tool call or is cut off at max_tokens, so client code should handle null.
Earlier thinking: when your client sends a previous assistant turn's reasoning back as reasoning_content (agent tools such as Goose and OpenCode do; Open WebUI and most chat apps send the answer text only), the model keeps it only within the current tool loop; thinking from before its last final answer is dropped. Returned thinking counts toward the input limit, so a long thinking-on agent session reaches the too-long error sooner. Measured through this service 2026-10-26.
Not an embedding model: a /v1/embeddings request returns an error (HTTP 404). Embeddings come only from a model listed as an embedding model on /status.
Deployment
Limit is about 190 pages (approximately 122,880 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
Last check 7 min agoLatency 33msUptime 24h 100.0% (96 checks)
Qwen3 Embedding 8B
Responding
embeddings
Turns text into vectors. It does NOT answer questions.
ccs/Qwen/Qwen3-Embedding-8B
Size8BArchDense embedding modelContextabout 50 pages per item (approximately 33,000 tokens)Dimensions4096Endpoint/v1/embeddings
Good for
RAG -- embed your documents, embed the question, retrieve the closest chunks
Semantic search over a corpus
Clustering, deduplication, similarity scoring
Detail, limits and gotchas
Watch out
Pass `encoding_format: "float"`. It used to be required -- without it the request failed with HTTP 400 -- and since the upstream upgrade on 2026-08-16 it succeeds either way. Keep sending it so your code does not depend on which upstream version happens to be deployed.
Sending a chat request to this model returns an error (HTTP 404). Three failed requests to it within a minute put it into a 30-second cooldown for everyone (HTTP 429, "try again in 30 seconds"), so fix the endpoint rather than retrying.
Not for
Chat or question answering. It has no answer to give -- it returns numbers, not text.
Do not send it to /v1/chat/completions. Use /v1/embeddings. A chat request to this model does not only fail for you -- it takes the model out for everyone for about 30 seconds.
Notes
Limits: each input can be up to 32,768 tokens; there is no reply, so nothing is kept back. An input over the limit gets a clear error and no result; nothing is silently cut. The limit is per input, so a batch of many inputs is fine as long as no single one is over.
Send at most 256 inputs per request (a 256-input batch measured fine on 2026-10-05; much larger batches can fail).
The dimensions parameter is ignored: every vector has 4096 dimensions (verified 2026-10-05).
Returns a 4096-dimension vector per input. Size your vector store for that.
Typical RAG shape: embed chunks once and store them, embed each query at ask time, retrieve top-k by cosine similarity, then pass those chunks to a chat model as context.
Deployment
Limit is about 50 pages per item (approximately 33,000 tokens) of input: every message in the conversation added together, including the system prompt and anything retrieved. The reply you ask for does not count. Go over it and you get a clear error naming the limit: nothing is sent, and nothing is silently dropped.
The gateway speaks the OpenAI API. Point any OpenAI-compatible client
at the base URL below, authenticate with a Bearer token, and pass one
of the model ids above as model.
Base URLhttps://llm.ccs.uky.edu/v1
This host issues no keys to people. For a durable personal key, use
Locksmith. If you are running an
Open OnDemand session or a batch job, a short-lived key is issued to
the job automatically and revoked when the job ends. There is
nothing to do.
Never share your key or embed it in published code. To check a key
and compare models without installing anything, use
Try it. See the
full documentation for
streaming, embeddings, and client integrations.