What does this merge request do and why?
Fixes an issue where LLMs occasionally produce malformed tool call arguments (e.g. passing a string instead of a list), which causes tool execution to fail silently and results in blank or broken responses for the user.
This MR adds pre-approval validation of tool call arguments against their Pydantic schemas in ChatAgent. When a malformed tool call is detected, the agent automatically retries the LLM request (up to 3 times), appending the validation error to the conversation history so the model can self-correct. If all retries are exhausted, a user-friendly error message is returned.
Closes gitlab-org/gitlab#592447
How to set up and validate locally
- I was able to reproduce quite reliably with question "Add joke to readme" and the
use_generic_gitlab_api_toolsfeature flag enabled. - Other option is to apply this change to create a mock (which allows to change then number of malformed tool calls)
diff --git a/duo_workflow_service/agents/chat_agent.py b/duo_workflow_service/agents/chat_agent.py
index 702dccd5b..5aeb78444 100644
--- a/duo_workflow_service/agents/chat_agent.py
+++ b/duo_workflow_service/agents/chat_agent.py
@@ -173,10 +173,37 @@ class ChatAgent:
return messages
async def _get_agent_response(self, state: ChatWorkflowState) -> BaseMessage:
- return await self.prompt_adapter.get_response(
+ # --- DEV ONLY: mock corrupt tool calls to test retry exhaustion ---
+ import json as _json # noqa: F811,E402 # DEV ONLY
+
+ _MOCK_REMAINING = 4
+ if not hasattr(self, "_mock_corrupt_ctr"):
+ self._mock_corrupt_ctr = _MOCK_REMAINING
+
+ response = await self.prompt_adapter.get_response(
state, system_template_override=self.system_template_override
)
+ if (
+ self._mock_corrupt_ctr > 0
+ and isinstance(response, AIMessage)
+ and response.tool_calls
+ ):
+ for call in response.tool_calls:
+ if call["name"] == "create_commit" and isinstance(
+ call["args"].get("actions"), list
+ ):
+ log.warning(
+ "DEV MOCK: Corrupting create_commit tool call args",
+ remaining=self._mock_corrupt_ctr,
+ )
+ call["args"]["actions"] = _json.dumps(call["args"]["actions"])
+ self._mock_corrupt_ctr -= 1
+ break
+
+ return response
+ # --- END DEV ONLY ---
+
def _build_response(
self, agent_response: BaseMessage, state: ChatWorkflowState
) -> Dict[str, Any]:
@@ -317,3 +344,4 @@ class ChatAgent:
except Exception as error:
log_exception(error, extra={"context": "Error processing chat agent"})
return self._create_error_response(error)- You should see that Chat fixes the tool call without asking user to approve invalid call.
- If more than three times the tool fails, it ends with a message about the failure.
| Before | After - with one failure | After - with continuous failure |
|---|---|---|
 |
 |
 |
Merge request checklist
- Tests added for new functionality. If not, please raise an issue to follow up.
- Documentation added/updated, if needed.
- If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.