Setup Guides

• Google Vertex AI: uses service account credentials. Best for enterprise setups with GCP infrastructure.
• Google AI Studio: uses a simple API key. Easier to set up.

​

Option 1: Google Vertex AI

​

Google Cloud Setup

1. Enable the Vertex AI API in your Google Cloud Platform.
2. Go to “Service Accounts” in the Google Cloud Console IAM Settings.

3. Click on “Create Service Account”.
4. Give the Service Account a name.

5. Assign the “Vertex AI User” Role.

6. Create the Service Account.

7. You are brought back to the Service Account overview.

8. On the overview page, click on “Manage keys”.

9. Create a new JSON key.

10. Download and open the JSON file.

​

Langdock Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Use the prebuilt Langdock config or set up manually. Set the SDK to Google Vertex.

3. Fill in the connection fields:
   • Service Account Email: paste the client_email value from your JSON key file (e.g. my-sa@my-project.iam.gserviceaccount.com)
   • Service Account Private Key: paste the private_key value from your JSON key file (including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----)
   • Region: your Vertex AI region (e.g. europe-west3, us-central1). This determines which Vertex AI endpoint is used.
   • Model ID: the model ID from the Vertex portal (e.g. gemini-2.5-flash, gemini-2.5-pro)
4. Click Save and test the model.

​

Option 2: Google AI Studio

1. Get an API key from Google AI Studio.
2. Go to Workspace Settings -> Models and click Add Custom Model. Select Google AI Studio as the SDK.
3. Paste your API key and set the Model ID.
4. Click Save and test the model.

​

Imagen (Image Generation)

1. An AWS account with Bedrock access enabled
2. IAM credentials with Bedrock permissions
3. Model access enabled in your AWS Bedrock console
4. Admin access to your Langdock workspace

​

AWS Setup

​

1. Enable Model Access

1. Go to the AWS Bedrock Console.
2. Navigate to Model access in the left sidebar.
3. Click Manage model access and enable the models you need.
4. Wait for access to be granted (this may take a few minutes).

​

2. Create IAM Credentials

1. Go to the AWS IAM Console.
2. Navigate to Users and click Create user.
3. Give the user a descriptive name (e.g. langdock-bedrock-access).
4. Attach the AmazonBedrockFullAccess policy, or create a custom policy with minimum required permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Resource": "*"
        }
    ]
}

5. Go to the user’s Security credentials tab, click Create access key, select Third-party service, and save both the Access Key ID and Secret Access Key.

​

Langdock Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Select Bedrock as the SDK.

3. Fill in the connection fields:
   • Access Key ID: your AWS Access Key ID
   • Secret Access Key: your AWS Secret Access Key
   • Region: your AWS region (e.g. us-east-1, eu-central-1)
   • Model ID: use the Bedrock model identifier (see below)
   • Context Size: set according to the model (see the model configuration tables)
4. Click Save and test the model by sending a prompt before making it visible to all users.

​

Model IDs

​

Cross-Region Inference Profiles

​

Supported Regions

• US East (N. Virginia): us-east-1
• US West (Oregon): us-west-2
• EU (Frankfurt): eu-central-1
• EU (Ireland): eu-west-1
• EU (Paris): eu-west-3
• Asia Pacific (Tokyo): ap-northeast-1
• Asia Pacific (Sydney): ap-southeast-2

​

Network Configuration

​

Troubleshooting

1. A Mistral account at console.mistral.ai
2. An API key from the Mistral platform
3. Admin access to your Langdock workspace

​

Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Fill in the connection fields:
   • SDK: select Mistral
   • Base URL: leave empty to use the default (https://api.mistral.ai/v1) or specify a custom endpoint
   • Model ID: use the official model identifier (see below)
   • API Key: paste your Mistral API key
3. Click Save and test the model.

​

Model IDs

​

Using Mistral from Azure

​

Configuration Notes

• Mistral models support tool calling natively.
• The default API endpoint https://api.mistral.ai/v1 is used automatically when no custom Base URL is provided.
• Mistral models are known for strong multilingual capabilities, particularly in European languages.

​

Troubleshooting

1. A DeepSeek account at platform.deepseek.com
2. An API key from the DeepSeek platform
3. Admin access to your Langdock workspace

​

Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Fill in the connection fields:
   • SDK: select DeepSeek
   • Base URL: https://api.deepseek.com/v1
   • Model ID: see table below
   • API Key: paste your DeepSeek API key
   • Region: US
3. For reasoning models (R1), enable Always show reasoning to surface the model’s thinking in the UI.
4. Click Save and test the model.

​

Model IDs

​

Configuration Notes

• DeepSeek models are hosted in the US region only.
• DeepSeek R1 is a reasoning model — enable Always show reasoning to see its reasoning steps in the UI. DeepSeek models do not support image analysis.
• The base URL must include the /v1 path: https://api.deepseek.com/v1.

​

Troubleshooting

1. A Perplexity account at perplexity.ai
2. An API key from your Perplexity API settings
3. Admin access to your Langdock workspace

​

Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Fill in the connection fields:
   • SDK: select Perplexity
   • Base URL: leave empty to use the default (https://api.perplexity.ai)
   • Model ID: see table below
   • API Key: paste your Perplexity API key
   • Region: US
3. Click Save and test the model.

​

Model IDs

​

Configuration Notes

• Perplexity models include built-in web search capabilities, so they always have access to current information.
• The API endpoint https://api.perplexity.ai is automatically used when no custom Base URL is provided.
• Sonar Pro models provide more detailed responses with better source citations.
• Reasoning variants are best for complex analytical tasks that benefit from step-by-step thinking.
• Perplexity models do not support image analysis.

​

Troubleshooting

• vLLM: high-throughput inference server for large language models
• LiteLLM: proxy server providing a unified interface to 100+ LLM providers
• Ollama: run large language models locally
• Text Generation Inference (TGI): Hugging Face’s inference server
• LocalAI: self-hosted, OpenAI-compatible API
• Custom deployments: any service implementing the OpenAI chat completions API

1. A running OpenAI-compatible inference endpoint accessible over HTTPS
2. The base URL of your endpoint
3. The model ID/name as configured in your inference server
4. An API key (if your endpoint requires authentication)
5. Admin access to your Langdock workspace

​

Setup

1. Go to Workspace Settings -> Models and click Add Custom Model.
2. Fill in the connection fields:
   • SDK: select OpenAI Compatible
   • Base URL: your endpoint URL (e.g. https://your-server.com/v1). Required.
   • Model ID: the exact model identifier as configured in your inference server
   • API Key: your authentication key, or leave empty if not required
   • Context Size: the context window size of your model in tokens
3. Click Save and test the model.

​

Example Configurations

​

Common Use Cases

• Data privacy: run models on your own infrastructure so prompts and responses stay within your network.
• Cost optimization: running open-source models on your own hardware can significantly reduce costs for high-volume use cases.
• Custom fine-tuned models: deploy models fine-tuned for specific tasks or domains with vLLM or similar servers.
• Multi-provider abstraction: use LiteLLM as a proxy to route requests to different providers from a single interface.

​

Troubleshooting