Sending a request in prompt mode
Written by
Updated at May 2, 2024
To generate text in prompt mode, send a request to the model using the completion method.
Getting started
Get API authentication credentials as described on the Authentication with the Yandex Foundation Models API page.
Send a request to the model
-
Create a file with the request body, e.g.,
body.json
:{ "modelUri": "gpt://<folder_ID>/yandexgpt-lite", "completionOptions": { "stream": false, "temperature": 0.1, "maxTokens": "1000" }, "messages": [ { "role": "system", "text": "Translate the text" }, { "role": "user", "text": "To be, or not to be: that is the question." } ] }
Where:
-
modelUri
: ID of the model to generate the response. The parameter contains the ID of a Yandex Cloud folder or the ID of a model fine-tuned in DataSphere. -
completionOptions
: Request configuration options:stream
: Enables streaming of partially generated text. It may take either thetrue
orfalse
value.temperature
: With a higher temperature, you get more creative and randomized response from the model. This parameter accepts values between0
and1
, inclusive. The default value is0.3
.maxTokens
: Sets a limit on the model's output in tokens. The maximum number of tokens per generation depends on the model. For more information, see Quotas and limits in YandexGPT API.
-
messages
: List of messages that set the context for the model:-
role
: Message sender's role:user
: Used to send user messages to the model.system
: Used to set request context and define the model's behavior.assistant
: Used for responses generated by the model. In chat mode, model's responses tagged with theassistant
role are included in the message to save the conversation context. Do not send user messages with this role.
-
text
: Text content of the message.
-
-
-
To send the request to the model, run this command:
export FOLDER_ID=<folder_ID> export IAM_TOKEN=<IAM_token> curl --request POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer ${IAM_TOKEN}" \ -H "x-folder-id: ${FOLDER_ID}" \ -d "@<path_to_json_file>" \ "https://llm.api.cloud.yandex.net/foundationModels/v1/completion"
Where:
FOLDER_ID
: ID of the folder for which your account has theai.languageModels.user
role or higher.IAM_TOKEN
: IAM token received before starting.
The service will respond with the generated text:
{ "result": { "alternatives": [ { "message": { "role": "assistant", "text": "Быть или не быть - вот в чём вопрос." }, "status": "ALTERNATIVE_STATUS_FINAL" } ], "usage": { "inputTextTokens": "28", "completionTokens": "10", "totalTokens": "38" }, "modelVersion": "06.12.2023" } }