You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently when using OutputGuardrails, if I issue a reprompt and my AI Service uses structured output, that output info isn't sent in the reprompts.
For example, if my original request looks like this:
{
"model" : "llama3.2",
"messages" : [ {
"role" : "system",
"content" : "You are a helpful, respectful, and honest assistant named \"Parasol Assistant\".\n\nYou work for Parasol Insurance.\n\nYour role is to generate business professional emails, with a subject, for clients notifying them of changes in their claim status.\nBe polite and concise.\n\nYou will always return a JSON response with the following structure:\n{\n\"subject\": \"string\",\n\t\"body\": \"string\"\n}\n\nPlease ONLY include the JSON. Do NOT begin the response with things like:\n- Here is your JSON\n\nAll your email bodies should end with the following text, EXACTLY as it appears below:\n\nSincerely,\nParasoft Insurance Claims Department\n\n--------------------------------------------\nPlease note this is an unmonitored email box.\nShould you choose to reply, nobody (not even an AI bot) will see your message.\nCall a real human should you have any questions. 1-800-CAR-SAFE."
}, {
"role" : "user",
"content" : "Start the email body with 'Dear Marty McFly,'\nMake sure to include the claim number (CLM195501) and that the claim's status has been changed to \"Denied\".\n\n\nAnswer using the following information:\n4.3.\n\nAn adjuster may be assigned to inspect the vehicle and assess damages.\n\n5. Premiums and Deductibles\n5.1. Premiums are calculated based on factors including driver history, vehicle type, and coverage\n\nselected.\n5.2. Deductibles apply to collision and comprehensive coverage as specified on the declarations\n\npage.\n\n6. Cancellation\n6.1. The policyholder may cancel at any time with pro-rata refund of premiums.\n6.2. The insurer reserves the right to cancel within 30 days notice, or immediately in cases of\n\nnon-payment or fraud.\n\n7. Definitions\n7.1. "Insured vehicle" refers to the automobile listed on the declarations page.\n\n7.2. "Accident" means a sudden, unexpected event resulting in damage or injury.\n\n8. Additional Provisions\n8.1. Coverage extends to other drivers listed on the policy.\n8.2. Rental car coverage may be included if specified on the declarations page.\n8.3. Roadside assistance may be available if specified on the declarations page.\n\n9. Contact Information\nFor claims or inquiries:\nPhone: 800-CAR-SAFE\nEmail: [email protected]\n\nParasol Insurance Company\nPolicy Document\n\n1. Coverage\n1.1. This policy provides coverage for the insured vehicle as specified in the declarations page.\n1.2. Coverage types may include:\n\n1.2.1. Liability (Bodily Injury and Property Damage)\n1.2.2. Collision\n1.2.3. Comprehensive\n1.2.4. Personal Injury Protection (PIP)\n1.2.5. Uninsured / Underinsured Motorist\n\n1.3. Specific coverage limits and deductibles are listed on the declarations page.\n\n2. Policy Term\n2.1. The policy term is six months from the date of inception, unless otherwise specified.\n2.2. Renewal is not automatic and must be initiated by the policyholder.\n\n3. Exclusions\n3.1. Damage caused by intential acts or gross negligence is not covered.\n3.2. Normal wear and tear is excluded from coverage.\n3.3. Damage resulting from racing is excluded\n3.4. Damage resulting from using the vehicle for commercial purposes without proper endorsement\n\nis excluded.\n3.5. Driving under the influence of alcohol or drugs voids coverage for that incident.\n\n4. Claim Process\n4.1. Claims should be filed as soon as possible after an incident.\n4.2. Policyholders must provide all relevant information, including police reports when applicable.\n4.3. An adjuster may be assigned to inspect the vehicle and assess damages."
} ],
"options" : {
"temperature" : 0.3,
"top_k" : 40,
"top_p" : 0.9
},
"format" : {
"type" : "object",
"properties" : {
"subject" : {
"type" : "string"
},
"body" : {
"type" : "string"
}
},
"required" : [ "subject", "body" ]
},
"stream" : false
}
notice the format and required elements.
If an output guardrail issues a reprompt, the format and required elements are missing:
{
"model" : "llama3.2",
"messages" : [ {
"role" : "system",
"content" : "You are a helpful, respectful, and honest assistant named \"Parasol Assistant\".\n\nYou work for Parasol Insurance.\n\nYour role is to generate business professional emails, with a subject, for clients notifying them of changes in their claim status.\nBe polite and concise.\n\nYou will always return a JSON response with the following structure:\n{\n\"subject\": \"string\",\n\t\"body\": \"string\"\n}\n\nPlease ONLY include the JSON. Do NOT begin the response with things like:\n- Here is your JSON\n\nAll your email bodies should end with the following text, EXACTLY as it appears below:\n\nSincerely,\nParasoft Insurance Claims Department\n\n--------------------------------------------\nPlease note this is an unmonitored email box.\nShould you choose to reply, nobody (not even an AI bot) will see your message.\nCall a real human should you have any questions. 1-800-CAR-SAFE."
}, {
"role" : "user",
"content" : "Start the email body with 'Dear Marty McFly,'\nMake sure to include the claim number (CLM195501) and that the claim's status has been changed to \"Denied\".\n\n\nAnswer using the following information:\n4.3.\n\nAn adjuster may be assigned to inspect the vehicle and assess damages.\n\n5. Premiums and Deductibles\n5.1. Premiums are calculated based on factors including driver history, vehicle type, and coverage\n\nselected.\n5.2. Deductibles apply to collision and comprehensive coverage as specified on the declarations\n\npage.\n\n6. Cancellation\n6.1. The policyholder may cancel at any time with pro-rata refund of premiums.\n6.2. The insurer reserves the right to cancel within 30 days notice, or immediately in cases of\n\nnon-payment or fraud.\n\n7. Definitions\n7.1. "Insured vehicle" refers to the automobile listed on the declarations page.\n\n7.2. "Accident" means a sudden, unexpected event resulting in damage or injury.\n\n8. Additional Provisions\n8.1. Coverage extends to other drivers listed on the policy.\n8.2. Rental car coverage may be included if specified on the declarations page.\n8.3. Roadside assistance may be available if specified on the declarations page.\n\n9. Contact Information\nFor claims or inquiries:\nPhone: 800-CAR-SAFE\nEmail: [email protected]\n\nParasol Insurance Company\nPolicy Document\n\n1. Coverage\n1.1. This policy provides coverage for the insured vehicle as specified in the declarations page.\n1.2. Coverage types may include:\n\n1.2.1. Liability (Bodily Injury and Property Damage)\n1.2.2. Collision\n1.2.3. Comprehensive\n1.2.4. Personal Injury Protection (PIP)\n1.2.5. Uninsured / Underinsured Motorist\n\n1.3. Specific coverage limits and deductibles are listed on the declarations page.\n\n2. Policy Term\n2.1. The policy term is six months from the date of inception, unless otherwise specified.\n2.2. Renewal is not automatic and must be initiated by the policyholder.\n\n3. Exclusions\n3.1. Damage caused by intential acts or gross negligence is not covered.\n3.2. Normal wear and tear is excluded from coverage.\n3.3. Damage resulting from racing is excluded\n3.4. Damage resulting from using the vehicle for commercial purposes without proper endorsement\n\nis excluded.\n3.5. Driving under the influence of alcohol or drugs voids coverage for that incident.\n\n4. Claim Process\n4.1. Claims should be filed as soon as possible after an incident.\n4.2. Policyholders must provide all relevant information, including police reports when applicable.\n4.3. An adjuster may be assigned to inspect the vehicle and assess damages."
}, {
"role" : "assistant",
"content" : "{\n\"subject\": \"Claim Status Update for CLM195501\",\n\"body\": \"Dear Marty McFly,\\n\\nYour claim (CLM195501) has been reviewed, and its status has been changed to 'Denied'.\\n\\nPlease refer to our policy document for more information on the reasons behind this decision. If you have any questions or concerns regarding your coverage, we encourage you to reach out to us at 800-CAR-SAFE or [email protected].\\n\\nSincerely,\\nParasoft Insurance Claims Department\" }"
}, {
"role" : "user",
"content" : "The email body did not end properly. Please try again.\n\nThe email body should end with the following text, EXACTLY as it appears below:\n\nSincerely,\nParasoft Insurance Claims Department\n\n--------------------------------------------\nPlease note this is an unmonitored email box.\nShould you choose to reply, nobody (not even an AI bot) will see your message.\nCall a real human should you have any questions. 1-800-CAR-SAFE."
} ],
"options" : {
"temperature" : 0.3,
"top_k" : 40,
"top_p" : 0.9
},
"stream" : false
}
I think this is because AiServiceMethodImplementationSupport creates a ChatRequest and then calls chatModel.chat:
Currently when using
OutputGuardrail
s, if I issue areprompt
and my AI Service uses structured output, that output info isn't sent in the reprompts.For example, if my original request looks like this:
notice the
format
andrequired
elements.If an output guardrail issues a reprompt, the
format
andrequired
elements are missing:I think this is because
AiServiceMethodImplementationSupport
creates aChatRequest
and then callschatModel.chat
:quarkus-langchain4j/core/runtime/src/main/java/io/quarkiverse/langchain4j/runtime/aiservice/AiServiceMethodImplementationSupport.java
Lines 346 to 360 in 9314bcc
but in
GuardrailsSupport
it just callschatModel.generate(chatMemory.messages());
quarkus-langchain4j/core/runtime/src/main/java/io/quarkiverse/langchain4j/runtime/aiservice/GuardrailsSupport.java
Line 88 in 9314bcc
The text was updated successfully, but these errors were encountered: