How to add a custom row-level text evaluator.
Contains
for word lists. See available descriptors.Toy data to run the example
CustomColumnDescriptor
that will:
num
) scores or categorical (cat
) labels.
CustomDescriptor
that:
target_answer
and answer
columns, and return a label:
CustomDescriptor
to run evals for multiple columns and return multiple scores.
As a fun example, let’s reverse all words in the question
and answer
columns: