By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

How to set to False the [--fail-on-failure] option when using %load magic command

0

Hello, I am using %load command as follows: %load --source s3://path_to_my_csv -f opencypher -r my_region -l my_arn --run

I want to load a csv with a relationship node1_TO_node2.csv

But in this csv, some of the node1 elements do not exist in the database.

So far when I try to load the CSV, I get LOAD_FAILED because some vertex do not exist.

I would like the relationship to be loaded and created for the nodes than exist and ignoring the lines of the csv whose nodes do not exist.

I though to set the option [--fail-on-failure] to False like this:

%load --source s3://path_to_my_csv -f opencypher --fail-on-failure False -r my_region -l my_arn --run

, but I got an error: it seems --fail-on-failure option does not accept arguments??

How shall I do to set to False? Is there any way to load the relationship csv even when it contains nodes that do not exist in the DB and how to do it?

thanks

Topics
Database
Tags
Amazon Neptune
Language
English
Lorena
asked a month ago78 views
2 Answers
1

The default behavior of the %load magic is to not fail on error/failure. If you leave the --fail-on-failure flag off, it defaults to the False state.

Even in the event that the loader continues to execute and skips over the failures, it will end in a LOAD_FAILED state because of the errors encountered in the process. You can see the errors that are generated (and CSV lines skipped) by using the %load_status magic with the --details and --errors flags set.

profile pictureAWS
Taylor-AWS
answered a month ago
  • Lorena
    a month ago

    I already did that, and actually I see that some errors happen because some nodes of the relationship do not exist. Concerning the flag --fail-on-failure and its value False by default, I understand that in this case, even with errors, my relationship should be created for the nodes that exist. Is it True?? Because this is not the case today for me. I got a LOAD FAILED and when I check if the relationship has been created, I can see that it has not been created at all, even for the nodes that exist in the DB, is that normal?

  • Taylor-AWS
    a month ago

    If the flag is not set, it should load whatever edges it can that have both nodes of the edge already in the graph. Another possibility is that there could be a parsing error with those lines in the CSV, preventing it from loading those edges. You might also want to set the --mode flag to NEW to ensure the loader is passing over all of the CSV files that you are (re)attempting to load.

0

Seems this command doesn't have an option called --fail-on-failure

https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-load

profile picture
EXPERT
Oleksii Bebych
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content