Bulba Code Eval Rating Chat Tasks 2-LLM Code Review and Rating

Example

Assessing if a response accurately follows the given instructions, distinguishing between explicit and implicit directives.

Scenario

In a scenario where a user requests optimization tips for a Python script, Bulba Code Eval would evaluate whether the provided response directly addresses the optimization aspect, considering both the explicit request for tips and the implicit expectation for applicable Python practices.

Example

Verifying the accuracy of provided information and the functionality of code.

Scenario

For a submission involving data sorting algorithms, this function would scrutinize the algorithms' correctness, efficiency, and reliability in sorting data as claimed.

Example

Evaluating the clarity, organization, and documentation of the response and any accompanying code.

Scenario

If a user submits a code documentation guide, the evaluation would focus on the documentation's comprehensibility, structure, and whether it effectively communicates the code's purpose and usage.

Example

Ensuring the response avoids unsafe or toxic content and code.

Scenario

When evaluating a code snippet that interacts with external systems, Bulba Code Eval checks for any potential security vulnerabilities or practices that could harm user data or privacy.