XGBoost Reports Not Generated

Question

Hi!

I have been trying to create a model using XGBoost, and was able to successfully run/train the model. However, I have not been able to generate the training reports. I have included the rules parameter as follows: "rules=[Rule.sagemaker(rule_configs.create_xgboost_report())]".

I am following this tutorial, but I am using objective: "multi:softmax" instead of the "binary:logistic" used in the example.

When I run the model everything is fine but only the Profiler Report gets generated and I do not see the XGBoostReport under the rule-output folder. According to the tutorial it should be under the same file path.

Here is my code for the model if it helps any:

```
s3_output_location='s3://{}/{}/{}'.format(bucket, prefix, 'xgboost_model')
container = sagemaker.image_uris.retrieve("xgboost", boto3.Session().region_name, "latest")

train_input = TrainingInput(
    "s3://{}/{}/{}".format(bucket, prefix, "data/train.csv"), content_type="csv"
)
validation_input = TrainingInput(
    "s3://{}/{}/{}".format(bucket, prefix, "data/validation.csv"), content_type="csv"
)

rules=[
    Rule.sagemaker(rule_configs.create_xgboost_report())
]

xgb = sagemaker.estimator.Estimator(
    image_uri=container,
    role=sagemaker.get_execution_role(),
    instance_count=1,
    instance_type="ml.c5.2xlarge",
    volume_size=5,
    output_path=s3_output_location,
    sagemaker_session=sagemaker.Session(),
    rules=rules
)

xgb.set_hyperparameters(
    max_depth=6,
    objective='multi:softmax',
    num_class=num_classes,
    gamma=800,
    num_round=250
)
```

Any help is appreciated! Thanks!

Answer

I believe the issue here is the rules parameter is receiving a URL, not an array of RuleBase objects, as required by the Estimator documentation. Try re-writing your estimator accordingly, as suggested by the SageMaker Developer Guide:

xgb = sagemaker.estimator.Estimator(
   image_uri=container, 
   role=sagemaker.get_execution_role(), 
   instance_count=1, 
   instance_type="ml.c5.2xlarge", 
   volume_size=5, 
   output_path=s3_output_location, 
   sagemaker_session=sagemaker.Session(), 
   rules=[
      Rule.sagemaker(
         rule_configs.create_xgboost_report()
      )  
   ]
)

For more information on using Rules, check out the SageMaker Debugger documentation.

```python
xgb = sagemaker.estimator.Estimator(
   image_uri=container, 
   role=sagemaker.get_execution_role(), 
   instance_count=1, 
   instance_type="ml.c5.2xlarge", 
   volume_size=5, 
   output_path=s3_output_location, 
   sagemaker_session=sagemaker.Session(), 
   rules=[
      Rule.sagemaker(
         rule_configs.create_xgboost_report()
      )  
   ]
)
```

For more information on using Rules, check out the [SageMaker Debugger documentation](https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html#continuous-analyses-through-rules).

Answer

Hi,

I had the same problem. To be able to generate the xgboost report, make sure that you use xgboost version 1.2-1 in your image_uri and sklearn version 1.0-1 as your estimator_cls in FrameworkProcessor. Furthermore, set header to False in train, validation and test files in your preprocessing script so that you exclude them before you run estimator.fit(). This worked for me.

Answer

Hello jughead, Has your problem of your code been resolve? Let us know. Also, remember to click on the "Accept" button when an answer provided in the community helped you. This allows other community members to also benefit from it. Thank you for your participation.

XGBoost Reports Not Generated

Relevant content