Throughout OpenAI’s dev day, one of many major announcements was the power to obtain a JSON from the chat completion API. Nevertheless, there aren’t a number of clear examples of how to do that as most examples give attention to operate calls.
Our goal is easy: given a question, we wish to obtain a solution in JSON format.
How can we obtain this?
There are three essential steps.
Modify your immediate
Your immediate should explicitly specify that the response ought to be in JSON format and it’s good to outline the construction of the JSON object.
Given this work historical past, what is the total years of expertise?
Work historical past consists of begin and finish date (or current), title, and firm identify.
If the top date is "Current", use the present date. At the moment is November 2023.
Return the reply in JSON format with the sphere "experienceInMonths"
and worth as a quantity.
Take note of the final sentence of the immediate.
Move response_format
When calling the API, specify the response_format.
const res = await this.openAI.chat.completions.create({
mannequin: 'gpt-3.5-turbo-1106',
temperature: 0.0,
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
response_format: {
kind: 'json_object', // specify the format
},
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: workHistory },
],
});
It is essential to switch the immediate as effectively. Simply altering the response kind to JSON may lead to a JSON of an arbitrary construction.
See this remark from the OpenAI API:
**Necessary:** when utilizing JSON mode, you **should** additionally instruct the mannequin to
produce JSON your self through a system or person message. With out this, the mannequin might
generate an never-ending stream of whitespace till the era reaches the token
restrict, leading to elevated latency and the looks of a "caught" request.
Additionally word that the message content material could also be partially minimize off if
`finish_reason="size"`, which signifies the era exceeded `max_tokens`
or the dialog exceeded the max context size.
Parse JSON response
As soon as we obtain the response, the content material continues to be textual content (string kind), however we are able to now parse it as JSON.
const content material: string = get(res, 'selections[0].message.content material');
strive {
return JSON.parse(content material)['experienceInMonths'];
} catch (error) {
this.logger.warn('Calculating whole expertise for a member didn't work');
return -1;
}
It is good apply to wrap JSON.parse in a strive…catch assertion in case we obtain an invalid JSON construction.
You could find a playground example here.