En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post

[Aws Glue DataBrew] - Does DataBrew has dataset limit size?

0

Hi! In the DataBrew, I'm trying to import a S3 dataset with aprox. 5 millions rows - I selected a job profile to run on the full dataset. The job run successfully but the number of rows are less than the full dataset (aprox. 2.5millions) Can you help me to understand if is there any size limitation? It's weird because there isn't any error during the process and in the Data profile overwiew is written: "Data profile was run on full dataset."

Thanks in advance.

  • Vikas
    il y a un an

    I think it depends on mode you selected. Did you provided value? https://docs.aws.amazon.com/databrew/latest/dg/API_JobSample.html

  • FJ
    il y a un an

    Hi! thanks for your comment.

    I selected the full dataset, so the job profile should run in the entire dataset. The problem is that the job run successfully but not on the entire dataset (only 2.5million rows were considered out of 5 millions).

Sujets
Analytique
Balises
AWS Glue DataBrew
Langue
English
FJ
demandé il y a un an74 vues
Aucune réponse

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents