LLM Inference

Get started

1. Create a developer account and verify your email.
2. Open your Overview, accept the current LLM agreement, and complete the required BAA. Your signer can complete it through DocuSign; no Medsender account review is needed for LLM access.
3. Create a key in LLM API Keys and save the secret shown once.
4. Open Billing and add credits through Stripe. Billing setup is automatic. Key creation does not require payment first.
5. Run the example below with your key. A successful response confirms your first completion.

Production faxing has a separate approval process. Fax approval is not required for LLM inference.

LLM keys make real inference requests and consume credits. Staging and production use separate hosts; there is no simulated or free test-key mode.

Inference requires available credits and an unfrozen billing organization. If a request is refused for insufficient credits, top up in Billing and retry after the balance updates.

Endpoint and authentication

POST /v1/chat/completions is the only route the gateway serves. Every other path is refused at the edge.

Send your key in the Authorization header using the Bearer scheme. Create keys in the portal at LLM Keys.

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gemini-3.5-flash","messages":[{"role":"user","content":"Say hello."}],"max_tokens":32}'

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gemini-3.5-flash","messages":[{"role":"user","content":"Say hello."}],"max_tokens":32}'

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gemini-3.5-flash","messages":[{"role":"user","content":"Say hello."}],"max_tokens":32}'

Models and pricing

Prices are in USD per one million tokens. Sign in to the portal Overview to see pricing for your account.

Making a request

A complete chat completion using the default gemini-3.5-flash model: a system message that sets the behavior, a user message that asks the question, and a cap on the generated length. Sign in to the portal Overview to generate the same example for any other model.

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "gemini-3.5-flash",
  "messages": [
    {
      "role": "system",
      "content": "You are a clinical intake assistant. Answer in one sentence."
    },
    {
      "role": "user",
      "content": "Why would a primary care office send a referral to a cardiologist?"
    }
  ],
  "max_tokens": 256
}'

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "gemini-3.5-flash",
  "messages": [
    {
      "role": "system",
      "content": "You are a clinical intake assistant. Answer in one sentence."
    },
    {
      "role": "user",
      "content": "Why would a primary care office send a referral to a cardiologist?"
    }
  ],
  "max_tokens": 256
}'

curl https://llm.medsender.com/v1/chat/completions \
  -H "Authorization: Bearer $MEDSENDER_LLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "gemini-3.5-flash",
  "messages": [
    {
      "role": "system",
      "content": "You are a clinical intake assistant. Answer in one sentence."
    },
    {
      "role": "user",
      "content": "Why would a primary care office send a referral to a cardiologist?"
    }
  ],
  "max_tokens": 256
}'

Using the OpenAI SDK

The endpoint is OpenAI-compatible, so the official openai SDKs work once you override the base URL. Shown for Python and Node/TypeScript, the languages with an official SDK; in any other language, call the endpoint over plain HTTP as above.

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.medsender.com/v1",
    api_key=os.environ["MEDSENDER_LLM_API_KEY"],
)

completion = client.chat.completions.create(
    model="gemini-3.5-flash",
    messages=[
        {"role": "system", "content": "You are a clinical intake assistant. Answer in one sentence."},
        {"role": "user", "content": "Why would a primary care office send a referral to a cardiologist?"},
    ],
    max_tokens=256,
)
print(completion.choices[0].message.content)

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.medsender.com/v1",
    api_key=os.environ["MEDSENDER_LLM_API_KEY"],
)

completion = client.chat.completions.create(
    model="gemini-3.5-flash",
    messages=[
        {"role": "system", "content": "You are a clinical intake assistant. Answer in one sentence."},
        {"role": "user", "content": "Why would a primary care office send a referral to a cardiologist?"},
    ],
    max_tokens=256,
)
print(completion.choices[0].message.content)

# pip install openai
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://llm.medsender.com/v1",
    api_key=os.environ["MEDSENDER_LLM_API_KEY"],
)

completion = client.chat.completions.create(
    model="gemini-3.5-flash",
    messages=[
        {"role": "system", "content": "You are a clinical intake assistant. Answer in one sentence."},
        {"role": "user", "content": "Why would a primary care office send a referral to a cardiologist?"},
    ],
    max_tokens=256,
)
print(completion.choices[0].message.content)

// npm install openai
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://llm.medsender.com/v1',
  apiKey: process.env.MEDSENDER_LLM_API_KEY,
});

const completion = await client.chat.completions.create({
  model: 'gemini-3.5-flash',
  messages: [
    { role: 'system', content: 'You are a clinical intake assistant. Answer in one sentence.' },
    { role: 'user', content: 'Why would a primary care office send a referral to a cardiologist?' },
  ],
  max_tokens: 256,
});
console.log(completion.choices[0].message.content);

// npm install openai
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://llm.medsender.com/v1',
  apiKey: process.env.MEDSENDER_LLM_API_KEY,
});

const completion = await client.chat.completions.create({
  model: 'gemini-3.5-flash',
  messages: [
    { role: 'system', content: 'You are a clinical intake assistant. Answer in one sentence.' },
    { role: 'user', content: 'Why would a primary care office send a referral to a cardiologist?' },
  ],
  max_tokens: 256,
});
console.log(completion.choices[0].message.content);

// npm install openai
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://llm.medsender.com/v1',
  apiKey: process.env.MEDSENDER_LLM_API_KEY,
});

const completion = await client.chat.completions.create({
  model: 'gemini-3.5-flash',
  messages: [
    { role: 'system', content: 'You are a clinical intake assistant. Answer in one sentence.' },
    { role: 'user', content: 'Why would a primary care office send a referral to a cardiologist?' },
  ],
  max_tokens: 256,
});
console.log(completion.choices[0].message.content);

Request schema and limits

The body takes exactly three fields. Any other field is rejected rather than ignored.

A request is refused when it exceeds any of these:

• 32 messages per request
• 8,192 input tokens
• max_tokens of 131,072
• 1 MiB of request body (1,048,576 bytes)

Errors

Most failures return the OpenAI-compatible error envelope: an error object with message, type, param, and code. Branch on the status and, where the table names one, on code. Two shapes differ and are shown below: a 400 carries a plain string, and a 402 carries only code and message plus billing fields. A code not listed here means the problem is on our side; do not retry it, and contact support.

Requests time out after 60 seconds and return upstream_error. A shorter prompt and a smaller max_tokens finish sooner.

{
  "error": {
    "message": "Rate limit exceeded for this API key.",
    "type": "invalid_request_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

{
  "error": {
    "message": "Rate limit exceeded for this API key.",
    "type": "invalid_request_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

{
  "error": {
    "message": "Rate limit exceeded for this API key.",
    "type": "invalid_request_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

{
  "error": {
    "code": "insufficient_credits",
    "message": "not enough credits for this request",
    "balance_usd_micro": 1000,
    "top_up_url": "https://developer.medsender.com/billing"
  }
}

{
  "error": {
    "code": "insufficient_credits",
    "message": "not enough credits for this request",
    "balance_usd_micro": 1000,
    "top_up_url": "https://developer.medsender.com/billing"
  }
}

{
  "error": {
    "code": "insufficient_credits",
    "message": "not enough credits for this request",
    "balance_usd_micro": 1000,
    "top_up_url": "https://developer.medsender.com/billing"
  }
}

{
  "detail": {
    "error": "unsupported request fields: stream"
  }
}

{
  "detail": {
    "error": "unsupported request fields: stream"
  }
}

{
  "detail": {
    "error": "unsupported request fields: stream"
  }
}

How requests are billed

• Credits are held for the worst case when a request is accepted and settled to actual usage when it completes.
• A request the provider fails is never charged.
• A request you cancel or disconnect from mid-flight is charged the full held amount, since the provider may have generated the whole response.
• Cached input tokens are billed at the cached rate where the model lists one.

Not supported

These are rejected, not ignored: a request that uses one fails instead of quietly falling back, so do not build against them.

• Streaming (stream) and stream_options
• Tools and function calling
• response_format
• The user field
• The Responses API
• Embeddings

Key lifecycle

Each key carries its own spending allowance and throughput. The limits are per key, so splitting traffic across keys is how you separate one workload from another.

• Does not expire. A key stays valid until you revoke it.
• Spends against a rolling 30-day allowance. If you expect to need more than that allows, contact us before you launch.
• Allows 60 requests per minute, 100,000 tokens per minute, and 4 concurrent requests.
• Counts toward a ceiling of 50 active keys per account. Revoked keys do not count, but a key creation still in progress does.
• Revoking takes effect immediately and is permanent. There is no un-revoke, so a revoked key is replaced by a new one.

Create, review, and revoke keys at LLM Keys.