Allowing Omics Access to Private AWS Elastic Container Registry

0

I am getting an error when attempting to run a simple Hello World script in a Private Amazon Omics Nextflow Workflow. I am attempting to use a private registry container, as I will need to do so for my more complicated workflow.

I am attempting to get this to function. In this regard, I am being overly flexible in permissions. I will be more restrictive once I have bypassed this issue. I have done the following:

  1. I have created a user role with the following policies:

    • AmazonOmicsFullAccess (AWS Managed)
    • ECR Full Access (Customer managed): Full access to ECR
    • AmazonEC2ContainerRegistryFullAccess (AWS Managed)
    • AWSAppRunnerServicePolicyForECRAccess (AWS Managed)
  2. For the private containers in question, I have added a Policy to grant Amazon Omics permission to access Amazon ECR (https://docs.aws.amazon.com/omics/latest/dev/permissions-resource.html). I have not provided Cross Account Access as we are operating in one account.

2.5. I have added the publishDir parameter as reequired by Amazon omics when using nextflow as described in the Nextflow private definition file example: https://docs.aws.amazon.com/omics/latest/dev/workflow-definition-examples.html

  1. My hello_world.nf file is:
workflow {
    writeHelloWorld()
    printHelloWorld(writeHelloWorld.out.file)
}

process writeHelloWorld {
    container "accountid.dkr.ecr.eu-west-2.amazonaws.com/specificcontainer"
    publishDir "/mnt/workflow/pubdir" //Required for AWS Omics
    output:
        path("hello.txt"), emit: file

    script:
    """
    echo "Hello, World!" > hello.txt
    """
}

process printHelloWorld {
    container "accountid.dkr.ecr.eu-west-2.amazonaws.com/specificcontainer"
    publishDir "/mnt/workflow/pubdir" //Required for AWS Omics
    input:
        path hello_file

    script:
    """
    cat ${hello_file}
    """
}

My nextflow.config file is:

nextflow.enable.dsl=2

docker {
    enabled = true
}
workDir = '/mnt/workflow/pubdir'

I have tried with and without the workDir = '/mnt/workflow/pubdir' line in the nextflow.config file.

  1. I have created and ran the workflow in the same region that my configuration is in. I have created the worklflow with the following code, after I zipped up the files appropriately:
parameters = {
    removed for privacy,
}

# Open the file in binary mode and read it
with open('../deploy/awsomics_test.zip', 'rb') as f:
    data = f.read()

response = client.create_workflow(
    name="TestNF",
    description="Test  workflow",
    definitionZip=data,  # Must be binary
    main="hello_world.nf",
    parameterTemplate=parameters
)

response
  1. I am running the workflow with the following code:
OMICS_JOB_ROLE_ARN = 'arn:aws:iam::accountnumber:role/service-role/role_i've configured'

response = client.start_run(
    workflowId=workflow_gatk['id'],
    name="Test nf workflow run hello world",
    roleArn=OMICS_JOB_ROLE_ARN,
    parameters = {
        removed for privacy
    },
        outputUri=f's3://awsomicsdata/output/',
)

run_greetings = response
response

Here, I would expect the process to work. Instead, I receive the following error when running the first task in the hello_world.nf file:

touch: cannot touch '/mnt/workflow/3c/d5ea6acdf25e46dd5b45dd8d987fa6/.command.begin': Permission denied

This error is a result of the following line:

echo "Hello, World!" > hello.txt

Is there a configuration issue I have?

Dan
gefragt vor einem Jahr320 Aufrufe
1 Antwort
0

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen