Sagemaker Batch Transform for a tensorflow model

Question

I have trained a model on our own data to take two embedding vectors as input and provide a probability score as output. So far I have been using the model hosted as a real time endpoint and querying it periodically using Lambda functions.
However, the size of the data has increased exponentially (around 2.2 mil rows now) and I need to set the model up as a batch transform job. I can't find any good examples or details about how to do so for my particular case. The nature of my input data is as follows -> four columns: user_id, user_embedding, post_id, post_embedding in .parquet or .json format. The model takes the user_embedding and post_embedding as input and outputs the probability score. 
Can someone please point me in the right direction or tell me if there's a better solution?

The model is a tensorflow deep learning model whose artefacts are saved in an S3 bucket. The input data is also present in an S3 bucket.

AWS-User-8139205 · Accepted Answer

Hi Sarath, 
1. [Create the model](https://us-east-1.console.aws.amazon.com/sagemaker/home?region=us-east-1#/models/create) in SageMaker Console or using the [CreateModel API](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModel.html), specify the right [inference container image](https://docs.aws.amazon.com/sagemaker/latest/dg/neo-deployment-hosting-services-container-images.html) based on the model framework along with the s3 location that contains the model artefacts including the inference code. 
2. Create a[ BatchTransform job ](https://us-east-1.console.aws.amazon.com/sagemaker/home?region=us-east-1#/transform-jobs/create)in the SageMaker console or using the [CreateTransformJob API](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html), parallelise the prediction using multiple instances and MultiRecord Batch strategy to speed up the batch inference based on the dataset volume
3. [Start the transform job](https://sagemaker.readthedocs.io/en/stable/api/inference/transformer.html#sagemaker.transformer.Transformer.transform)

Check an example [here](https://sagemaker-examples.readthedocs.io/en/latest/sagemaker_batch_transform/introduction_to_batch_transform/batch_transform_pca_dbscan_movie_clusters.html).

Sagemaker Batch Transform for a tensorflow model

Relevant content