Lambda extensions overhead?

Assuming you have some requirements that demands your lambda to access secrets, the extension approach is much, much better then a direct aws api call to the secrets client.

Moreover the extensions has got environments variables that support caching: https://docs.aws.amazon.com/secretsmanager/latest/userguide/retrieving-secrets_lambda.html#retrieving-secrets_lambda_env-var

You probably should evaluate lambda with powertools to assess different approaches.