Multitenacy in quicksight embedded dashboard without session capacity pricing

I want to restrict data access to some users where they can view data relevant to them only and not the complete dataset

If you need to restrict the data by dataset rows, you can use the Row Level Security with the help of user based rules. That does not need session capacity pricing.

If you need to restrict the dataset columns, then you can use Column Level Security.