Skip to content

🐛 [TestRemoval] - HumanEval/0 , Asseration Oracle is Wrong #274

@Marvinmw

Description

@Marvinmw

EvalPlus version

[[1.0, 2.0, 3.0, 4.0, 5.0], 1.0], False

Output of running ls ~/.cache/evalplus

AssertionError

Task ID of the programming task

has_close_elements

Test input

https://github.com/evalplus/humanevalplus_release/releases
the 18th input,

0 ([[1.0, 2.0, 3.9, 4.0, 5.0, 2.2], 0.3], True)
1 ([[1.0, 2.0, 3.9, 4.0, 5.0, 2.2], 0.05], False)
2 ([[1.0, 2.0, 5.9, 4.0, 5.0], 0.95], True)
3 ([[1.0, 2.0, 5.9, 4.0, 5.0], 0.8], False)
4 ([[1.0, 2.0, 3.0, 4.0, 5.0, 2.0], 0.1], True)
5 ([[1.1, 2.2, 3.1, 4.1, 5.1], 1.0], True)
6 ([[1.1, 2.2, 3.1, 4.1, 5.1], 0.5], False)
7 ([[1.0, 2.5, 3.0, 4.5, 5.0, 6.5], 0.4], False)
8 ([[1.0, 2.0, 3.0, 2.9, 5.0, 6.0], 0.1], False)
9 ([[0.5, 0.6, 0.7, 0.8, 0.9], 0.05], False)
10 ([[10.5, 20.5, 30.5, 25.5, 40.5], 4.0], False)
11 ([[1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7], 0.2], False)
12 ([[0.1, 0.5, 1.0, 1.5, 2.0], 0.1], False)
13 ([[1.2, 2.4, 3.6, 4.8], 0.8], False)
14 ([[-1.0, -0.5, 0.0, 0.5, 1.0], 0.3], False)
15 ([[-2.0, 2.0, 3.0, 4.0], 1.5], True)
16 ([[1.0, 2.0, 3.0, 4.0, 5.0], 0.5], False)
17 ([[1.0, 2.0, 2.9, 5.0, 6.0], 4.4], True)
18 ([[1.0, 2.0, 3.0, 4.0, 5.0], 1.0], False)

check(has_close_elements)
assertion(candidate(*inp), exp, 0)
assert exact_match
AssertionErro

Description

Here is the wrong test input for has_close_elements, which should be removed.

Other context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions