How to use session token in AWS S3 Rest api

No, the session token is not enough. You must also include an Authorization header that contains Access Key ID and the calculated signature to provide a complete authentication info to the S3 API.

also - if the access key is still used to sign the headers and provide the public part of the access key to the aws server - what is the need of session token?

Session token is required for temporary credentials (i.e., the one time access key and the secret key received from AWS STS). AWS need it to validate your credentials.

How does AWS sdk browser client work?

If you use AWS SDK instead of plain HTTP client (e.g., older curl command), you don't need to sign the credential and headers. All of the signing process will be carried internally when you call each of the SDK API.

For example, if you want to call ListBuckets API, you simple create an S3Client with your credentials and session token, and send a ListBucketsCommand with the S3Client.

const idToken = await getIdToken();
const credentials = await getCredentials(idToken);
const accessKeyId = credentials?.AccessKeyId;
const secretAccessKey = credentials?.SecretKey;
const sessionToken = credentials?.SessionToken;
const expiration = credentials?.Expiration;
const client = new S3Client({
  credentials: { accessKeyId, secretAccessKey, sessionToken, expiration },
});
const listBucketsCommand = new ListBucketsCommand({});
const { Buckets } = await client.send(listBucketsCommand);
console.log(JSON.stringify(Buckets, null, 2));