Provide a Random State for DeepAR Build-In Algorithm

0

Hello,

I'm currently experimenting with the DeepAR algorithm for a sales forecast. The model is giving me decent results so far, which is great. However, I'm afraid it has one major downsight: There is no possibility to provide a "random state" or a state in any sense to produce comparable results. When I train two seperate models with exactly the same datasets and Hyperparameters, I end up with two different models that give me different predictions. Especially in the exploratory phase this is really inconvenient because I cannot be certain if a model improvement or deterioration comes from my changes or simply because of randomness.

Is there any way to avoid this issue and to set a state of some kind? If not, are there any plans in implementing this? It would be very helpful. The Implementation from the GluonTS library includes such a parameter.

Many Thanks Oliver

Oliver
asked 6 months ago206 views
2 Answers
0

Hello,

Unfortunately this is not fixing my problem. When I set the random seed in this way, I still end up with two different models, eventhough I have trained them with the exact same data and hyperparameters. I don't think that the locally set random seed will be transferred to the EC2 Instance where the DeepAR model is being trained. The random seed in the example provided is used for data preparation tasks, but not for the model training as it seems.

Are there any other ways to approach this problem? If not, are there any plans to implementing a random seed parameter for DeepAR?

Thanks!

Oliver
answered 6 months ago
-1

Hello,

I understand that you want to provide a "random state" or a state in any sense to produce comparable results with the in-built DeepAR algorithm.

Kindly note that numpy can be used for setting random seeds for reproducibility using numpy.random.seed -

# set random seeds for reproducibility
np.random.seed(42)
random.seed(42)

Also, there is an example of SageMaker/DeepAR demo on electricity dataset from AWS which uses the same technique of setting random seeds. Here is the link to it -> https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/deepar_electricity/DeepAR-Electricity.html

Thanks :)

AWS
SUPPORT ENGINEER
answered 6 months ago
  • Hello,

    Unfortunately this is not fixing my problem. When I set the random seed in this way, I still end up with two different models, eventhough I have trained them with the exact same data and hyperparameters. I don't think that the locally set random seed will be transferred to the EC2 Instance where the DeepAR model is being trained. The random seed in the example provided is used for data preparation tasks, but not for the model training as it seems.

    Are there any other ways to approach this problem? If not, are there any plans to implementing a random seed parameter for DeepAR?

    Thanks!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions