[PYTHON] How to set up Random forest using Optuna

How to set up Random forest using Optuna

Since I wrote how to use ʻOptunalast time, I will describe the individual setting method from now on. .. There are various arguments that can be passed inRandomforest, but I set all the main ones in ʻOptuna. I was wondering whether to pass max_depth, n_estimators as integers or numbers as categories, but this time I passed them as integers.

def objective(trial):
    criterion = trial.suggest_categorical('criterion', ['mse', 'mae'])
    bootstrap = trial.suggest_categorical('bootstrap',['True','False'])
    max_depth = trial.suggest_int('max_depth', 1, 10000)
    max_features = trial.suggest_categorical('max_features', ['auto', 'sqrt','log2'])
    max_leaf_nodes = trial.suggest_int('max_leaf_nodes', 1, 10000)
    n_estimators =  trial.suggest_int('n_estimators', 30, 1000)
    
    regr = RandomForestRegressor(bootstrap = bootstrap, criterion = criterion,
                                 max_depth = max_depth, max_features = max_features,
                                 max_leaf_nodes = max_leaf_nodes,n_estimators = n_estimators,n_jobs=2)
    
    
    #regr.fit(X_train, y_train)
    #y_pred = regr.predict(X_val)
    #return r2_score(y_val, y_pred)
    
    score = cross_val_score(regr, X_train, y_train, cv=5, scoring="r2")
    r2_mean = score.mean()

    return r2_mean
#Execute optuna and set hyperparameters
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=1000)

#Create an instance with tuned hyperparameters
optimised_rf = RandomForestRegressor(bootstrap = study.best_params['bootstrap'], criterion = study.best_params['criterion'],
                                     max_depth = study.best_params['max_depth'], max_features = study.best_params['max_features'],
                                     max_leaf_nodes = study.best_params['max_leaf_nodes'],n_estimators = study.best_params['n_estimators'],
                                     n_jobs=2)
#learn
optimised_rf.fit(X_train ,y_train)

I used this to tune hyperparameters using a Boston dataset. It fits nicely.

Figure 2020-06-29 163110.png

Recommended Posts

How to set up Random forest using Optuna
How to set up Random forest using Optuna
How to set up SVM using Optuna
How to set xg boost using Optuna
How to set up a Python environment using pyenv
How to set optuna (how to write search space)
How to set up a local development server
How to set up public key authentication in ssh
How to set layer on Lambda using AWS SAM
How to set up and compile your Cython environment
I tried using Random Forest
How to set up Ubuntu for Windows Subsystem for Linux 2 (WSL2)
How to import Python library set up in EFS to Lambda
Day 67 [Introduction to Kaggle] Have you tried using Random Forest?
Survey log of how to optimize LightGBM hyperparameters using Optuna
How to install python using anaconda
How to speed up Python calculations
How to set up and use OMC Log Analytics --Linux version -
Set up a node to do MNIST on ROS using Tensorflow
How to set up the development environment of ev3dev [Windows version]
Random Forest (2)
[Blender] How to set shape_key with script
How to speed up instantiation of BeautifulSoup
Random Forest
[Python] How to display random numbers (random module)
How to draw a graph using Matplotlib
How to set the server time to Japanese time
How to install a package using a repository
Set up a mail server using Twisted
How to set up a jupyter notebook on ssh destination (AWS EC2)
Disease classification in Random Forest using Python
How to download youtube videos using pytube3
Summary of how to set up major Python Lint (pep8, pylint, flake8)
How to display Map using Google Map API (Android)
How to code a drone using image recognition
How to set browser location in Headless Chrome
How to speed up scikit-learn like conda Numpy
How to set Django DB to mongodb visual studio 2019
How to deal with SessionNotCreatedException when using Selenium
How to get article data using Qiita API
How to search HTML data using Beautiful Soup
How to upload to a shared drive using pydrive
How to uninstall a module installed using setup.py
How to set CPU affinity for process threads
How to create random numbers with NumPy's random module
How to set up a Google Colab environment with Coursera's advanced machine learning courses
How to set up a VPN gateway to establish a connection between Alibaba Cloud and AWS
I want to set up a mock server for python-flask in seconds using swagger-codegen.
[Rails] How to get location information using Geolocation API
How to read dynamically generated table definitions using SQLAlchemy
The easiest way to set up Last-Modified in Flask
How to write a GUI using the maya command
How to scrape horse racing data using pandas read_html
How to auto-submit Microsoft Forms using python (Mac version)
Set up a file server on Ubuntu 20.04 using Samba
How to hold a hands-on seminar using Jupyter using docker
How to right click using keyboard input in RPA?
How to make a Python package using VS Code
[Blender] How to dynamically set the selection of EnumProperty
Set PATH equivalent to "sudo su-" using Ansible environment
How to exit when using Python in Terminal (Mac)