By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

[Aws Glue DataBrew] - Does DataBrew has dataset limit size?

0

Hi! In the DataBrew, I'm trying to import a S3 dataset with aprox. 5 millions rows - I selected a job profile to run on the full dataset. The job run successfully but the number of rows are less than the full dataset (aprox. 2.5millions) Can you help me to understand if is there any size limitation? It's weird because there isn't any error during the process and in the Data profile overwiew is written: "Data profile was run on full dataset."

Thanks in advance.

  • Vikas
    a year ago

    I think it depends on mode you selected. Did you provided value? https://docs.aws.amazon.com/databrew/latest/dg/API_JobSample.html

  • FJ
    a year ago

    Hi! thanks for your comment.

    I selected the full dataset, so the job profile should run in the entire dataset. The problem is that the job run successfully but not on the entire dataset (only 2.5million rows were considered out of 5 millions).

Topics
Analytics
Tags
AWS Glue DataBrew
Language
English
FJ
asked a year ago70 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content