Allowing Omics Access to Private AWS Elastic Container Registry

I am getting an error when attempting to run a simple Hello World script in a Private Amazon Omics Nextflow Workflow. I am attempting to use a private registry container, as I will need to do so for my more complicated workflow.

I am attempting to get this to function. In this regard, I am being overly flexible in permissions. I will be more restrictive once I have bypassed this issue. I have done the following:

1. I have created a user role with the following policies:

   • AmazonOmicsFullAccess (AWS Managed)
   • ECR Full Access (Customer managed): Full access to ECR
   • AmazonEC2ContainerRegistryFullAccess (AWS Managed)
   • AWSAppRunnerServicePolicyForECRAccess (AWS Managed)

2. For the private containers in question, I have added a Policy to grant Amazon Omics permission to access Amazon ECR (https://docs.aws.amazon.com/omics/latest/dev/permissions-resource.html). I have not provided Cross Account Access as we are operating in one account.

2.5. I have added the publishDir parameter as reequired by Amazon omics when using nextflow as described in the Nextflow private definition file example: https://docs.aws.amazon.com/omics/latest/dev/workflow-definition-examples.html

3. My hello_world.nf file is:

workflow {
    writeHelloWorld()
    printHelloWorld(writeHelloWorld.out.file)
}

process writeHelloWorld {
    container "accountid.dkr.ecr.eu-west-2.amazonaws.com/specificcontainer"
    publishDir "/mnt/workflow/pubdir" //Required for AWS Omics
    output:
        path("hello.txt"), emit: file

    script:
    """
    echo "Hello, World!" > hello.txt
    """
}

process printHelloWorld {
    container "accountid.dkr.ecr.eu-west-2.amazonaws.com/specificcontainer"
    publishDir "/mnt/workflow/pubdir" //Required for AWS Omics
    input:
        path hello_file

    script:
    """
    cat ${hello_file}
    """
}

My nextflow.config file is:

nextflow.enable.dsl=2

docker {
    enabled = true
}
workDir = '/mnt/workflow/pubdir'

I have tried with and without the workDir = '/mnt/workflow/pubdir' line in the nextflow.config file.

4. I have created and ran the workflow in the same region that my configuration is in. I have created the worklflow with the following code, after I zipped up the files appropriately:

parameters = {
    removed for privacy,
}

# Open the file in binary mode and read it
with open('../deploy/awsomics_test.zip', 'rb') as f:
    data = f.read()

response = client.create_workflow(
    name="TestNF",
    description="Test  workflow",
    definitionZip=data,  # Must be binary
    main="hello_world.nf",
    parameterTemplate=parameters
)

response

5. I am running the workflow with the following code:

OMICS_JOB_ROLE_ARN = 'arn:aws:iam::accountnumber:role/service-role/role_i've configured'

response = client.start_run(
    workflowId=workflow_gatk['id'],
    name="Test nf workflow run hello world",
    roleArn=OMICS_JOB_ROLE_ARN,
    parameters = {
        removed for privacy
    },
        outputUri=f's3://awsomicsdata/output/',
)

run_greetings = response
response

Here, I would expect the process to work. Instead, I receive the following error when running the first task in the hello_world.nf file:

touch: cannot touch '/mnt/workflow/3c/d5ea6acdf25e46dd5b45dd8d987fa6/.command.begin': Permission denied

This error is a result of the following line:

echo "Hello, World!" > hello.txt

Is there a configuration issue I have?