Skip to main content

​
1. Set up the models and the keys in Langdock

You need different types of models for the platform to work. To add models, go to the model settings in the workspace settings. Here are the model types necessary to cover all functionalities:

​
1.1 Embedding Model

  • Embedding models process documents and allow the model to search uploaded documents
  • The platform requires an embedding model with 1536 dimensions for vector storage. Any OpenAI or Azure-compatible embedding model with 1536 dimensions will work.

​
1.2 Backbone Model

  • The backbone model is used for:
    • Generating chat titles in the sidebar
    • Summarizing conversation history for context management
    • Acting as a last-resort fallback when the main model and its fallback model both fail
    • Helper tasks like prompt optimization and OCR
  • Use a fast, cost-effective model for this. It also serves as the backbone for Deep Research.
The backbone model is a separate model you need to set up. Even if you already added the same model as a completion model, please set up another instance. This model will be set as a backbone model afterward.

​
1.3 Image Generation Model

  • Add at least one image generation model so users can generate images from chat.
  • Check the adding models page for supported image generation models and setup instructions.

​
1.4 Completion Models

  • These are the models your users select in the chat. Add the models you want to make available from the providers you have keys for.
  • Make sure to also add the models needed for Deep Research. The backbone model you set up in 1.2 already covers the Deep Research backbone.
  • We support models hosted by Microsoft Azure, AWS Bedrock, Google Vertex AI, Google AI Studio, OpenAI, Anthropic, Mistral, DeepSeek, Perplexity, Black Forest Labs, Replicate, and any OpenAI-compatible endpoint.
  • For quotas, anything between 200k and 500k TPM (tokens per minute) should be good to cover usage of ~200 users. For your most-used model, you might need a quota of 500k to 1 million tokens.
For the main models, we recommend setting up multiple deployments in different regions. If a model has an error in one region, Langdock automatically retries the call in a different region.
Checklist: You should have set up the following:
  • 1x Embedding model (1536 dimensions)
  • 1x Backbone model (separate instance from your completion models)
  • 1 or more image generation models
  • Models required for Deep Research
  • Completion models from the providers you want to offer

​
2. Reach out to the Langdock team

After you have set up all the models you need, reach out to the Langdock team. We will align with you on a timeslot to turn on BYOK on our side. Usually, this should be done in the late afternoon or evening when fewer users are active. There should not be any downtime; this is a precautionary step to ensure no disruptions during the switch. Please ensure that you or someone who can set up the models is available. We will check that an engineer is also available on our side.

​
3. Test the models

Please make sure that all of the models work correctly. Here is how you can test the models:
  • Completion models: Send a prompt to each model you can select in the interface (e.g., “write a story about dogs”).
  • Embedding model: Create a Knowledge Folder, upload a file to it, then start a chat and @mention the Knowledge Folder to ask a question about the file. You should receive an answer based on the file content.
  • Image model: Ask any model to generate an image. You should see an image generated by the model.
  • Backbone model: Write a message in a new chat and check whether a chat title is generated after sending the prompt. (Please ensure that strict mode is disabled for this model)
Please contact the Langdock team if there are any issues here.