​
1. Provide the required API keys

You need different models for the platform to work. Here are the models necessary to cover all functionalities:
  1. Embedding Model
  • Embedding models process documents and allow the model to search uploaded documents.
  • We currently require the provision of ADA v2 (text-embedding-ada-002).
  1. Backbone Model
  • For chat titles and planning steps of models that are not efficient in tool calling, a backbone model as a foundation is required.
  • We recommend GPT-4o mini (gpt-4o-mini) for this purpose.
  1. Image Generation Model
  • We currently require Dall-E 3 (dall-e-3).
  1. Completion Models
  • On top of these must-have models, you can add the completion models for your users like GPT-4o, o1, Claude 3.5 Sonnet, Gemini 1.5 Pro etc.
  • We support all models hosted by Microsoft Azure, AWS, Google Vertex and OpenAI.
  • For quotas, anything between 200k and 500k should be good to cover usage of ~200 users. For GPT-4o, the most-used model, you might need a quota of 500k to 1 Mio tokens.

​
2. Reach out to Langdock team

After you have received all keys you need, please reach out to the Langdock team. We will agree with you on a timeslot to turn on BYOK on our side. After that, you can add the different models. Usually, this is done in the evening, when less users are active, as there might be a downtime of a few minutes during the model switch.

​
3. Set up your models

After BYOK is activated, you can add the different models in the model settings. Here is a guide on how to add models.

​
4. Test the models

Please make sure that all of the models work correctly. Here is how you can test the models:
  • Completion models: Send a prompt to each model you can select in the interface (e.g. “write a story about dogs”).
  • Embedding model: Upload a file and ask a question about it (e.g. “what is in the file”). The upload should work and you should receive an answer based on the file.
  • Image model: Ask any model to generate an image. You should see an image generated by the model in the background.
  • Backbone model: Select a model which does not support native tool calling (like Llama or Mistral) and ask to search the web (e.g. “search the web for Langdock”). If you do not have any model without tool calling, send a message in a new chat and observe if the conversation name in the bar on the left changes after a few seconds.
Please do not hesitate to contact the Langdock team if there are any issues here.