Replicate integration

Overview

Replicate runs open-source ML models behind a single HTTP API – image generation (Flux, SDXL), audio (Whisper, MusicGen), video (Wan), and thousands of community-published models. The Automize connector covers Predictions (sync + async), Trainings, Deployments, model metadata, and search.

• Regions: Hosted at api.replicate.com. Inference runs on Replicate's GPU pool – no region selection at the API surface. Cold-start latency varies by model popularity.
• Plans: Pay-per-second of GPU time. No free tier, but small models on CPU billing tiers are sub-cent per call. Deployments (always-warm instances) bill per-second whether they're processing or not.

Setting up the connection

Auth type: API Key.

1. In Replicate → Account → API Tokens → Create token (starts with r8_).
2. Paste it as the API key. Connector sends Authorization: Token <key> (note: 'Token', not 'Bearer').
3. Set up billing under Account → Billing – every prediction call returns 402 until a card is on file.

Find your credentials at https://replicate.com/account/api-tokens.

Rate limits

Burst of 600 req/min on the prediction-create endpoint per token. No hard ceiling on parallel running predictions; Replicate's pool queues them. Long-running predictions count GPU-seconds, not API calls, against your bill.

Data model

Replicate's compute graph:

    Account
      ├─ Models (public or private)
      │    └─ Versions     (immutable SHA-pinned snapshots)
      │         └─ Schema  (OpenAPI for input/output)
      ├─ Predictions
      │    ├─ status       starting → processing → succeeded | failed | canceled
      │    ├─ input        (matches the version's input schema)
      │    ├─ output       (URL(s) for file outputs, string/JSON for others)
      │    ├─ logs         (model stdout, streamed)
      │    └─ metrics      (predict_time = GPU-seconds billed)
      ├─ Trainings        (paid version-creation for trainable models)
      │    └─ destination → new model/version on your account
      └─ Deployments      (always-warm hosted version, billed continuously)

Identifier shapes:

    Model      "owner/model-name"           (e.g. "stability-ai/sdxl")
    Version    64-hex-char SHA              (immutable; pin in code)
    Prediction "abc123xyz"                  (alphanum, ~13 chars)
    Training   same shape as Prediction
    Deployment "owner/deployment-name"

Pinning versions matters: a model maintainer can push a new version
with breaking input changes; the floating "latest" then silently
starts producing 400s. Always SHA-pin in production. Run Model
resolves "latest" for you (convenient but fragile).

Available operations

12 operations available. Click any row to jump to its detail.

Operations

Cancel Prediction

Cancel a running prediction

Tips

• Stops a running prediction. Already-billed GPU seconds aren't refunded. Use for runaway loops or wrong-input mistakes caught mid-flight.

Run Deployment

Run a prediction against a deployment (private capacity)

Tips

• POST /v1/deployments/{owner}/{name}/predictions – same async shape as Create Prediction, but routes through your warm deployment instead of the shared pool.
• No cold-start penalty (deployment is always live). Trade-off: deployment GPU is billed whether you're using it or not – only worth it for sustained traffic.

Create Prediction

Run a model – supply version+input or model owner/name+input

Tips

• Async – returns prediction_id and a webhook URL (if provided). Poll Get Prediction or set webhook=<your URL>.
• version is the model version SHA. Get it from List Models or Get Model. Pinning a version makes reruns deterministic; using the floating 'latest' breaks pipelines silently when the maintainer pushes a breaking input change.
• input is the model's input schema – varies per model. Get Model returns the OpenAPI schema for the latest version.

Create Training

Fine-tune a base model on your data

Tips

• Trainings are paid version-creation for models that support fine-tuning (typically LoRA on diffusion models or full fine-tunes on text models).
• destination='<your-account>/<new-model-name>' creates the trained model under your account. Make sure the destination model exists (create empty via Replicate UI first).
• Async – poll Get Training. On success, the trained version SHA is in output.version.

Get Model

Get model details (latest version, schema, etc.)

Tips

• GET /v1/models/{owner}/{name} – model metadata + latest_version + the version's OpenAPI input/output schema.
• The schema field is the source of truth for what 'input' shape to send. Don't read it from blog posts or community examples – those go stale when the maintainer pushes.

Get Prediction

Get prediction status and output

Tips

• Poll every 1-5s while status='starting' or 'processing'. Terminal states: succeeded, failed, canceled.
• logs field streams stdout from the model – useful for debugging slow runs.
• metrics.predict_time is GPU seconds billed. Sum across runs for cost reporting.

Get Training

Get training status and output

Tips

• GET /v1/trainings/{id} – same status flow as predictions (starting → processing → succeeded | failed | canceled).
• On success, output.version is the SHA of the newly trained version under your destination model.

List Deployments

List deployments on your account

No input parameters.

Tips

• Deployments are always-warm hosted versions of a model – no cold start, billed per second the instance is up. Use for low-latency UX flows (image gen behind a 'Generate' button).

List Public Models

List public models on Replicate

No input parameters.

Tips

• Paginated list of all public + your own private models. cursor-based pagination via 'next' URL.
• Each entry includes the latest_version (SHA) so you can grab the runnable version without a second call.

List Predictions

List recent predictions

No input parameters.

Tips

• Account-wide prediction history. Cursor pagination. Filter status= to find failures.
• Predictions are retained ~30 days; older runs disappear (logs + outputs both gone).

Run Model

Run a model by owner/name (latest version)

Tips

• Higher-level wrapper that creates a prediction AND waits for completion. Convenient for short-running models (<60s); the connector times out for long-running ones – use Create Prediction + poll for those.

Search Models

Full-text search across public models

Tips

• Free-text search across the Replicate registry. Returns ranked matches by name + description.
• Combine with Get Model to drill into the version + schema before kicking off a prediction.

FAQ

Cold-start latency – how bad?

Highly variable. Popular models (Flux, SDXL, Whisper) usually warm in seconds. Niche community models can take 30-120s to boot. Deployments eliminate this at the cost of always-on billing.

How do I get model output URLs to persist?

Output URLs from predictions are signed for 1 hour. Download immediately if you need persistence – store the bytes in S3/R2/Drive on your side.

Webhook delivery – retries?

Replicate delivers each event once. For at-least-once guarantees, set webhook_events_filter=['completed'] and reconcile via Get Prediction on a timer – webhook + poll is the standard pattern.

Fine-tuning – which models?

Mostly diffusion LoRAs (Flux, SDXL) and a handful of text models that publish training entry points. Get Model returns 'trainings' link on supported models; absent = no fine-tuning available.

Run a model end-to-end synchronously?

Use Run Model for predictions that finish in <60s. Otherwise create + poll. The synchronous wrapper has a hardcoded server-side timeout you'll hit on slow models.

Related connectors

• Openai
• Anthropic
• Elevenlabs
• Heygen