Introduction

I wanted to use the metric logloss in pycaret, but there was no logloss by default, so I investigated how to add it. Previously, it was not possible to add metrics, but with the October 2020 update (version 2.2.0), it seems that metrics can be added arbitrarily.

Environment and version

PyCaret 2.2.0
Google Colaboratory

What is pycaret

pycaret is a machine learning library that automatically performs everything from machine learning pre-processing to modeling.

Evaluation index that can be used by default

About the add_metric method

The add_metric method allows you to work with any metric available in scikit_learn or your own metric using the make_scorer function.

from pycaret.classification import add_metric
from sklearn.metrics import log_loss

add_metric('logloss', 'Log Loss', log_loss, greater_is_better = False)

A description of the add_metric () arguments.

--id: first argument --str type --Evaluation index ID --name: second argument --str type --Name when displaying the evaluation index --score_func: Third argument --type type --Evaluation index you want to add --In this case, the log_loss pulled from sklearn is included. --greater_is_better: 4th argument --bool type --Default = True --Score_func indicates whether a higher number is a better index or a lower number is a better index. --True means higher and False means lower.

Now, let's check the evaluation index again.

I was able to successfully add logloss.

Reference site

https://pycaret.readthedocs.io/en/latest/api/clustering.html