Controlling LLM Output Format and Parsing Results

Method 1: Using Function Calling Function Calling is a feature from OpenAI for GPT API that lets the LLM decide whether to call a specific function before outputting the final result. For example, if a user asks about today's weather, the LLM will first output an intermediate result telling you to call a weather-related function with the parameter 'today'. You can then call the weather function, get the result, and tell the LLM to output the final result to the user. This feature wasn't originally meant for controlling output format, but it gives us a standard JSON format when telling us which function to call, even with GPT-3.5, providing fairly stable JSON output. We can leverage this to control LLM output format. We can define the content we want ChatGPT to output as a function, but we don't actually need to execute the function, just get the output result from the LLM. For example, if I want ChatGPT to output an Object: { "name": "John", "age": 30, "city": "New York" } When calling GPT, define a function with parameters matching the desired JSON format: { "name": "getUserInfo", "description": "Get user information", "parameters": { "type": "object", "properties": { "name": { "type": "string", "description": "User's fullname" }, "age": { "type": "number", "description": "User's age" }, "city": { "type": "string", "description": "User's city" } }, "required": ["name", "age", "city"] } } Then when calling GPT, you can get a stable JSON output. The limitation is that the API must support Function Calling. Method 2: Using few-shot with output format examples If the API doesn't support Function Calling, we can use few-shot by providing one or more output format examples for the LLM to follow. For instance, when translating, I might have the LLM translate twice - once literally and once freely - then use the free translation. In this case, I don't need JSON format; I can simply use special characters to separate the two results and split by those characters to get the free translation. For JSON format, few-shot can also be used, but with GPT-3.5, stability isn't great and sometimes the format isn't followed. Ensure that your response can be parsed by Python json, use the following format as an example: { "name": "John", "age": 30, "city": "New York" }