How to use models from HuggingFace as evaluators.
Toy data to run the example
HuggingFace()
descriptor to call a specific named model. The model you use must return a numerical score or a category for each text in a column.
For example, to evaluate “curiousity” expressed in a text:
HuggingFace()
descriptor.
Model | Example use | Parameters |
---|---|---|
Emotion classification
| HuggingFace("response", model="SamLowe/roberta-base-go_emotions", params={"label": "disappointment"}, alias="disappointment") | Required:
|
Zero-shot classification
| HuggingFace("response", model="MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli", params={"labels": ["science", "physics"], "threshold":0.5}, alias="Topic") | Required:
|
GPT-2 text detection
| HuggingFace("response", model="openai-community/roberta-base-openai-detector", params={"score_threshold": 0.7}, alias="fake") | Optional:
|
label
or labels
fields.
predict
or predict_proba
for scoring.