Do we need Lambda extensions for accessing AWS Secrets Manager ?

In order to access secrets manager you only need to define the security policy in your lambda and it should then be able to access it from your code.

More info here

You do not need a Lambda extension to retrieve a secret from AWS Secrets Manager in a serverless workload. You should retrieve the secret as a global variable, not in the handler, to maximize performance. To learn more, see https://aws.amazon.com/blogs/networking-and-content-delivery/securing-and-accessing-secrets-from-lambdaedge-using-aws-secrets-manager.

This link -https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/cache-secrets-using-aws-lambda-extensions.html provides a way to run local host to cache the secrets.

This setup is not necessary at all the times. Accessing Secrets Manager from Lambda can be done via the boto3 API, but the pattern above is considered best practice as it prevent cold start, increase speed of retrieval and keep the costs for Secrets Manager access low (in cases where you expect a lot of invocations).