Skip to content

Issues with Rust XGBoost Implementation - NaN F1 Score and Training Error with "objective" Hyper-parameter #1622

Closed
@cyccbxhl

Description

@cyccbxhl

Reproduction Steps:

  1. Start the official container using the following command:
    docker run \ -it \ -v postgresml_data:/var/lib/postgresql \ -p 5433:5432 \ -p 8000:8000 \ ghcr.io/postgresml/postgresml:2.9.3 \ sudo -u postgresml psql -d postgresml

Issue 1: F1 Score is NaN
After executing the following SQL commands:
SELECT pgml.load_dataset('digits');
SELECT * FROM pgml.train('Handwritten Digits', 'classification', 'pgml.digits', 'target', algorithm => 'xgboost', hyperparams => '{"n_estimators": 10}');
The output:
image
The issue arises where the recall becomes NaN. This is due to the model trained by XGBoost classifying all test instances into the category "9," leading to TP + FN equals 0, hence causing division by zero in F1 score calculation. Notably, this issue does not surface when using runtime => python.

Issue 2: Training Error When Assigning "objective" Hyperparameter
Attempting to specify the "objective" hyperparam during training leads to an error.
SELECT * FROM pgml.train('Handwritten Digits', 'classification', 'pgml.digits', 'target', algorithm => 'xgboost', hyperparams => '{"n_estimators": 10, "objective": "multi:softmax"}');
The error:
image
Similarly, there are no issues with the Python XGBoost.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions