Error when trying to import package installed using lambda layer on nodejs14 ES module enabled lambda function

I have created a lambda function on the nodejs14 runtime that has the ES module enabled. I have installed a custom package using the lambda layer following the folder structure specified on documentation.

nodejs/node_modules/my-package-name/index.js

or

nodejs/node14/node_modules/my-package-name/index.js

When I try to import this package

import my-package-name from "my-package-name"

I get an error that says can't find the module. The weird part is it works perfectly fine when CommonJS is enabled (ES module disabled).