@manojmj Thanks for implementing this health check!

I'm reading this file. Is it the case that we just have one type of success message and then one type of failure message? If so, is it granular enough for users to debug?

cc: @nlee8

@fsieverding I separate out the editor extension checks to a separate issue from the Health Check issue which should be dedicated to Health Check on the self-hosted config page. I don't know if you thought through what / where this check should be?

@aelhusseiny @donaldcook - do you have any existing work related DAP-specific checks in editor extension diagnostics for self-hosted setup?

cc: @manojmj @poffey21 @jordanjanes

@bastirehm @wortschi What framework we need to allow smoke tests that can be run by our customers/support engineers/solution architectures? @bastirehm you mentioned in the AI Eng leadership call that there used to be evaluation runner based tests around 1 year ago?

cc: @cindy-halim

Per discussion with @manojmj since Health Check is not harder to do than script, we will prioritize Health Check - #593925

cc: @nlee8

Per discussion with @manojmj , I crossed out the command line script from the original proposal as Health Check is not harder to implement and better UX - gitlab#593948 (comment 3172319158)

@nlee8 For vLLM specific checks, I assume there's a way to see if the serving platform is vLLM?

I know you DM me the following:

Actually I found that  vLLM does expose some server info through an api. But it's hidden behind a flag.

Behind a flag meaning feature flag from vLLM?

cc: @nlee8 @cindy-halim @jordanjanes

Spoke with @manojmj and concluded it will be easier to implement health check directly.

Let's prioritize Health Check for now: #584698

cc: @fsieverding @jordanjanes @cindy-halim @nlee8

@eduardobonet What I mean is since we are charging per LLM call, per SOX (initially stringent requirement) we have to send a billing event per LLM call. I modified the description to say:

comply with SOX's atomic request approach (meaning since we are charging per request, we have to send a billable event per request)

@bastirehm @wortschi As discussed in the AI Eng Leadership call, starting this epic. I put the goals and draft out the short-term goals/deliverable in more details. Please review and let's collaborate async.

Adding in @AndrasHerczeg (as mentioned by @bastirehm in the call) and @cindy-halim from groupcustom models who's been looking into this topic.

@bcardoso- What questions would make sense to cover the self-hosted models?