Each file defines an evaluator for a task with a specific setting. Each evaluator takes predictions (a list of predictions) and golds (a list of the gold data item, and each gold data item is a dict) and returns a dict as the evaluation result.

You can add new evaluators here for the evaluation of a new task or an existing task with a new setting. If you use the ../third_party/ directory for the evaluator, please add them into the ../third_party directory, and specify their link in .gitsubmodule, which enables recursive cloning.