Skip to content

fix: prevent server crash when unknown model is selected#3593

Merged
mldangelo merged 1 commit intopromptfoo:mainfrom
schipiga:hotfix/unknown-model-crashes-server
Apr 4, 2025
Merged

fix: prevent server crash when unknown model is selected#3593
mldangelo merged 1 commit intopromptfoo:mainfrom
schipiga:hotfix/unknown-model-crashes-server

Conversation

@schipiga
Copy link
Copy Markdown
Contributor

@schipiga schipiga commented Apr 4, 2025 •
edited
Loading

Hi! It's pretty important fix, which prevents server crash and all running tests, if unknown model is had chosen. For example, if to choose any azureopenai (azureopenai:gpt-35-turbo-0125) model from UI list and click "Run Evaluation", then server is crashed.

P.S. I think azureopenai-models list should be reformatted, because FE and BE parts have different options about azure format

@gru-agent
Copy link
Copy Markdown
Contributor

gru-agent bot commented Apr 4, 2025

TestGru Assignment

Summary

Link CommitId Status Reason
Detail a86605b 🚫 Skipped No files need to be tested {"src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx":"target file(src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx) not in work scope \n include: src/app/src//*.ts \n exclude: node_modules,/.test.ts,**/.test.tsx,/*.spec.ts,/.spec.tsx,**/.d.ts,/*.test.js,/.spec.js","src/server/routes/eval.ts":"target file(src/server/routes/eval.ts) not in work scope \n include: src/app/src/**/.ts \n exclude: node_modules,/*.test.ts,/.test.tsx,**/.spec.ts,/*.spec.tsx,/.d.ts,**/.test.js,**/*.spec.js"}

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

@gru-agent
Copy link
Copy Markdown
Contributor

gru-agent bot commented Apr 4, 2025

TestGru Assignment

Summary

Link CommitId Status Reason
Detail a86605b 🚫 Skipped No files need to be tested {"src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx":"target file(src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx) not in work scope \n include: src \n exclude: node_modules,/*.test.ts,/.test.tsx,**/.spec.ts,/*.spec.tsx,/.d.ts,**/.test.js,**/*.spec.js,src/app/src","src/server/routes/eval.ts":"Can not find valuable test target.\nevalRouter: Out of scope - This code consists of an Express router handling various HTTP endpoints, which involves interactions between multiple modules, external services, and database operations. It is more suitable for integration testing."}

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

@schipiga schipiga changed the title Prevent server crash when unknown model is selected fix: prevent server crash when unknown model is selected Apr 4, 2025
@schipiga
Copy link
Copy Markdown
Contributor Author

schipiga commented Apr 4, 2025

Hi @typpo ! Could you pls review this pretty important hotfix

ellipsis-dev[bot]
ellipsis-dev bot reviewed Apr 4, 2025
View reviewed changes
Comment thread src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx
Comment thread src/server/routes/eval.ts
Comment thread src/server/routes/eval.ts Outdated
schipiga
schipiga commented Apr 4, 2025
View reviewed changes
Comment thread src/app/src/pages/eval-creator/components/RunTestSuiteButton.tsx
@schipiga schipiga force-pushed the hotfix/unknown-model-crashes-server branch 3 times, most recently from b3c3d24 to 38c00ea Compare April 4, 2025 12:38
@schipiga schipiga force-pushed the hotfix/unknown-model-crashes-server branch from 38c00ea to 6132086 Compare April 4, 2025 14:51
mldangelo
mldangelo approved these changes Apr 4, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@mldangelo mldangelo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, thank you so much!

@mldangelo mldangelo merged commit 2c92fd0 into promptfoo:main Apr 4, 2025
26 checks passed
@schipiga schipiga deleted the hotfix/unknown-model-crashes-server branch April 4, 2025 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mldangelo mldangelo mldangelo approved these changes

+1 more reviewer

@ellipsis-dev ellipsis-dev[bot] ellipsis-dev[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@schipiga @mldangelo