Design chat messages
Build clear text conversations within Router's supported message and context limits.
Use roles deliberately
| Role | Use it for |
|---|---|
system | Stable instructions, tone, boundaries, and response format guidance. |
user | The current request and user-provided context. |
assistant | Earlier model replies needed to continue a conversation. |
Router accepts between 1 and 16 messages. Message content must be text, and the combined UTF-8 message content cannot exceed the edition’s max_prompt_bytes (at most 196,608 bytes). Bytes only bound the request size: input that exceeds the edition’s context_length in tokens is rejected with context_length_exceeded, never truncated.
Keep durable instructions stable
Put product-wide behavior in one concise system message. Keep changing user context in user messages. This makes a conversation easier to inspect and reduces duplicated prompt text.
{ "model": "gemma3:1b", "messages": [ { "role": "system", "content": "You are a concise product support assistant. Say when information is missing." }, { "role": "user", "content": "Explain the difference between online providers and free capacity." } ]}Manage conversation history
Send only the history the next answer needs. For longer conversations, summarize older turns in your own application before the request reaches Router. Preserve facts, decisions, and unresolved questions in the summary.
Control output length
Set max_tokens to the smallest useful ceiling. It defaults to 512, Router’s platform ceiling is 2,048 output tokens, and the selected model edition can be lower. Input and output share the context window, so a long prompt leaves less room for output. A response ending with finish_reason: "length" reached max_tokens or filled the context window and may be incomplete.
Choose a temperature
temperature accepts values from 0 through 2. Lower values usually suit extraction, support, and repeatable formatting. Higher values permit more variation. The exact effect depends on the provider runtime and model.
Next, learn how to stream the answer or inspect the chat-completions request schema.