Hello,
Im trying with EMR Serverless, switching from EMR getting below error
Traceback (most recent call last):
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/spark_main.py", line 7, in <module>
from app.parsers.core_signal_parser import CoreSignalDataParser
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/sources_5dbc92a.zip/app/parsers/core_signal_parser.py", line 29, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/sources_5dbc92a.zip/app/formatter/candidate_formatter.py", line 11, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/sources_5dbc92a.zip/app/parsers/general_parser.py", line 23, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/sources_5dbc92a.zip/app/core/config.py", line 120, in <module>
File "/tmp/spark-07b8331c-566b-43e9-bf1a-f0a77cb86420/sources_5dbc92a.zip/app/core/config.py", line 86, in __init__
FileNotFoundError: [Errno 2] No such file or directory: 'config.json'
I had tried to pass files with --files s3://{S3_BUCKET}/emr/pyspark/code/users/devops/processing_source_code/config.json
No luck, can anyone help with this