AWS Glue ListCrawlers - StepFunctions

Question

Hi All,

I'm trying to get Glue crawlers names based on Tags (Key/Value) using "ListCrawlers" in Step Functions.  It is working with single tag, but how to give multiple tags in ListCrawlers step. Please find the code example below.  I was expecting two crawler names which have the tags -  **"test": "test_crawler",**. and     **"Name": "crawler_customer_csv"**

Can you please help me on this one?

**Step Functions Code**

```
{
  "Comment": "List Crawler names based on input tags.",
  "StartAt": "ListCrawlers",
  "States": {
    "ListCrawlers": {
      "Type": "Task",
      "End": true,
      "Parameters": {
        "MaxResults.$": "$.maxResults",
        "Tags.$": "$.tags"
      },
      "Resource": "arn:aws:states:::aws-sdk:glue:listCrawlers"
    }
  }
}
```

**Input Event:**

```
{
  "maxResults": 10,
  "tags": {
    "test": "test_crawler",
    "Name": "crawler_customer_csv"
  }
}
```

**Output Response**
```
{
  "CrawlerNames": []
}
```

Mikel Del Tio · Accepted Answer

Hi,

Both your code and the input parameters seem correct. Are you sure that when you filter by more than one tag the crawler contains all of them? Otherwise the `listCrawlers` method will not return it.

Additionally, remember that tags are case-sensitive, so they must be written exactly the same.

Ibrahim Shaik · Answer

Hi,

It looks like there is a bug in ListCrawler action/method.  It is not working with different Key Value pair in Tags.

I have modified my crawler tags to use same tags for two different crawlers.  At that time, ListCrawler actions returned two crawlers names which are matching to that  Tag value.

AWS Glue ListCrawlers - StepFunctions

Add your answer

Relevant content