When evals fail, exit with status code 1

[maraisr]

When running evals, I feel its a good idea to exit with status code of 1 that without parsing outputs we can know if evals passed or not.

Error: ❌ Some tests failed.
exit status 1

I did think maybe a CLI flag to invert this behaviour would be cool, but folk smarter than me say its something shell scripts can just do, see: jestjs/jest#10306 (comment)

[Copilot]

Pull Request Overview

This PR ensures eval commands exit with status code 1 when tests fail by introducing a sentinel error and updating CLI handling and tests.

• Adds FailedTests sentinel error and returns it when any test fails.
• Updates the Cobra command to silence usage on FailedTests and propagate a non-zero exit.
• Adjusts tests to expect FailedTests instead of no error.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
cmd/eval/eval.go	Introduced FailedTests error, updated runEvaluation to return it, and silenced usage in RunE when it occurs.
cmd/eval/eval_test.go	Changed assertions to expect FailedTests via require.ErrorIs instead of require.NoError.

Comments suppressed due to low confidence (2)

cmd/eval/eval.go:116

• Add a unit test for the Cobra command to verify that SilenceUsage is set to true when FailedTests is returned, ensuring the CLI usage output is correctly suppressed.

cmd.SilenceUsage = true

cmd/eval/eval.go:111

• Use a short variable declaration (err := handler.runEvaluation(...)) so err is defined in this scope rather than assigning to an undefined variable.

err = handler.runEvaluation(cmd.Context())