Overview
Cohere builds enterprise LLMs focused on retrieval and reranking. The Automize connector covers Chat, Embed, Rerank, Classify, Tokenize, Detokenize, and Model metadata via the Cohere REST API.
- Regions: Single global API at api.cohere.com. EU residency available on Enterprise plans via a separate api.cohere-eu.com host – set the connector's base URL if you've been provisioned onto it.
- Plans: Trial keys throttle to 5 requests/minute on Chat and 100/minute on Embed – fine for prototyping, will rate-limit fast in any production flow. Production keys remove the per-minute caps and unlock command-r-plus and embed-english-v3.0.
Setting up the connection
Auth type: Bearer Token.
- In the Cohere dashboard, open API Keys and create a Production key. Trial keys exist for quickstart use but rate-limit aggressively.
- Copy the key (it's shown once) and paste it as the API token in Automize.
- The connector sends Authorization: Bearer <key> and X-Client-Name: Automize so requests show up correctly in Cohere's usage dashboard.
Find your credentials at https://dashboard.cohere.com/api-keys.
Rate limits
Production: 10,000 chat req/min, 2,000 embed req/min (96 texts/request), 1,000 rerank req/min. Trial: 5 chat / 100 embed. Hard cap on output tokens per Chat call (~4k) regardless of plan. Connector retries on 429 with exponential backoff.
Data model
Cohere's model family + endpoints:
Chat command-r, command-r-plus, command (legacy)
Embed embed-english-v3.0, embed-multilingual-v3.0,
embed-english-light-v3.0 (lower-dim, cheaper)
Rerank rerank-english-v3.0, rerank-multilingual-v3.0
Classify classify (few-shot, no fine-tune step)
Tokenize deterministic – same tokens across calls
Embed input_type taxonomy (REQUIRED on v3 embed models):
search_document docs you're indexing for retrieval
search_query queries you're matching against the index
classification inputs you'll feed to Classify
clustering inputs you'll cluster
Using the wrong input_type silently degrades retrieval quality –
v3 embeddings train asymmetric encoders. document/query vectors
live in compatible subspaces; cross-classifying produces garbage.
Chat connectors (RAG/grounding):
web-search Cohere's hosted web search
retrieval your own RAG via documents: [{title, snippet}]
tool-use function calling via tools: [{name, schema}]
Citations always come back in chat.citations[] when connectors fire.Available operations
8 operations available. Click any row to jump to its detail.
| Operation | Category | What it does |
|---|---|---|
| Chat | General | Generate a chat completion (Command R/R+) |
| Classify | General | Classify inputs into provided categories |
| Detokenize | General | Convert token ids back to text |
| Embed | General | Generate embeddings for text or images |
| Get Model | General | Get model details |
| List Models | General | List available models |
| Rerank | General | Rerank documents against a query (great for retrieval-comparator pipelines) |
| Tokenize | General | Tokenize a piece of text |
Operations
Chat
Generate a chat completion (Command R/R+)
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Messages messages | No | text | |
Temperature temperature | No | text | |
Max Tokens max_tokens | No | text | |
P p | No | text | |
K k | No | text | |
Tools tools | No | text | |
Documents documents | No | text | |
Citation Options citation_options | No | text |
Tips
- Pass message for single-turn or chat_history + message for multi-turn. Use connectors=[{'id':'web-search'}] for grounded answers with citations.
- model defaults to command-r – use command-r-plus for higher quality at ~3x cost.
- stream=true is supported by the API but the connector resolves the full response (use SSE-aware actions elsewhere if you need token-level streaming).
Classify
Classify inputs into provided categories
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Inputs inputs | No | text | |
Examples examples | No | text | |
Preset preset | No | text |
Tips
- Few-shot classification – pass examples (text + label pairs, 2+ per label, 5+ total). No fine-tuning step needed.
- Best for short text (<512 tokens). Long documents – embed + cluster instead.
Detokenize
Convert token ids back to text
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Tokens tokens | No | text |
Tips
- Inverse of Tokenize – token IDs back to text. Useful for debugging tokenisation edge cases (special characters, emoji, non-Latin scripts).
Embed
Generate embeddings for text or images
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Texts texts | No | text | |
Images images | No | text | |
Input Type input_type | No | text | |
Embedding Types embedding_types | No | text | |
Truncate truncate | No | text |
Tips
- input_type is REQUIRED (search_document, search_query, classification, clustering). Mismatched input_type silently degrades retrieval quality.
- Up to 96 texts per call, 512 tokens each. Use embed-english-v3.0 for production retrieval; embed-multilingual-v3.0 covers 100+ languages.
Get Model
Get model details
| Parameter | Required | Type | Description |
|---|---|---|---|
Model Id model_id | Yes | text |
Tips
- GET /v1/models/{model_name} – capabilities, context_length, max_output_tokens, finetuned (bool), tokenizer.
- Pin the exact model name in production; -latest aliases drift when Cohere ships new versions.
List Models
List available models
No input parameters.
Tips
- Returns every model + endpoint pairing accessible to the workspace. endpoint field distinguishes chat/embed/rerank/classify variants.
- tokenizer_url field lets you download the BPE table for offline tokenisation – handy for client-side token counting.
Rerank
Rerank documents against a query (great for retrieval-comparator pipelines)
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Query query | No | text | |
Documents documents | No | text | |
Top N top_n | No | text | |
Rank Fields rank_fields | No | text | |
Return Documents return_documents | No | text |
Tips
- Pass a query + documents[] (up to 1000) and get back relevance-sorted indexes. This is the cheapest way to lift retrieval quality on top of any vector DB.
- top_n limits returned results; total documents scanned is unchanged. Returns relevance_score in [0, 1].
Tokenize
Tokenize a piece of text
| Parameter | Required | Type | Description |
|---|---|---|---|
Model model | No | text | |
Text text | No | text |
Tips
- Returns the token IDs + the string fragment per token for any input text. Deterministic per model – same input always gives same tokens.
- Use for prompt budgeting: count tokens before sending Chat to predict cost + stay under context-length caps.
FAQ
- Trial key vs Production key – what's actually different?
- Trial keys are free, throttle at 5 chat/min, and bar commercial use per Cohere's TOS. Production keys are billed per token, no rate cap (other than the per-minute production limits), and clear for production traffic. Use trial for prototyping only.
- Why are my embeddings worse than I expected?
- Check input_type. Embedding a query with input_type=search_document (instead of search_query) produces vectors in the wrong subspace – they look fine but rank terribly. The v3 models train asymmetric document/query encoders.
- Rerank pricing vs running a bigger retrieval?
- Rerank is per-document, per-query. For top-K=10 over 100 candidates, you're charged for 100 documents. Cheaper than running a 7B reranker yourself in almost every case; more expensive than the embed search step before it.
- Do I get token usage back?
- Yes – the chat response includes a meta.tokens block with input_tokens, output_tokens, and prompt_tokens. Log these against your Automize run for cost attribution.