New NLP/CV Examples to Get Started on AWS Inferentia and AWS Trainium
Announcing the new AWS-Neuron-Samples repo to help users learn how to compile and deploy models for AWS Inferentia or Trainium for a wide range of different Computer Vision and Natural Language Processing models.
Authored by Samir Araujo
We are excited to announce new AWS Inferentia and AWS Trainium examples in the a new AWS Neuron samples repository with many samples & tutorials to help you to prepare and run Deep Learning models. In this repository you can find examples for Computer Vision & NLP models implemented in PyTorch and TensorFlow. Each model notebook has step by step instructions to help you prepare your models and deploy them to Inferentia.
|PyTorch Neuron (torch-neuronx)||Sample training scripts for training various PyTorch models on AWS Trainium||Trn1|
|PyTorch Neuron (torch-neuron)||Sample Jupyter notebooks demonstrating model compilation and inference for various PyTorch models on AWS Inferentia||Inf1|
|TensorFlow Neuron (tensorflow-neuron)||Sample Jupyter notebooks demonstrating model compilation and inference for various TensorFlow models on AWS Inferentia||Inf1|
Start by following the instructions for the first three steps in the Neuron Getting Started Guide, to start your Inf1 instance and install the basic Neuron and Framework packages:
Once you have your Inf1 instance running and configured, setup your Jupyter enviorment and clone the Neuron Samples repo:
sudo yum install jupyter git clone https://github.com/aws-neuron/aws-neuron-samples.git cd aws-neuron-samples/ jupyter notebook
Once Jupyter server is launched then in the console the local server IP shown. Copy this into a web browser.
Ready to get started!
Each set of examples will be organized into a table like the one below displaying the library dependencies, versions and original sources. Bookmark or star the repo, as more models will be added frequently.
All the inference notebooks start with a section called "Install Dependencies". This section is responsible for installing/ upgrading packages required for each specific model example (see Yolov5 example below). The models were tested with the exact combination of libraries/versions you see in the section.
Select any of the model examples notebooks to get started. Each one will start with individual dependencies and show to prepare the input data, compile the model and test inference with some sample data.
For inference with Inferentia, depending on the model the notebooks brings cells that customizes the model structure before and after the compilation. This is necessary to make the models compatible with the compiler. For instance, if you have a PyTorch model, it needs to be Jit Traceable.
At the end of each notebook you can see predictions and the output generated by the execution of the compiled model on an Inf1 instance. Once you are done testing you model, you can deploy it directly to EC2 or on SageMaker.
Now it is your time to play with these models! If you have any questions, please ask them below. You can request new examples to this repo by filling an issue or contribute your examples or updates with a pull request.
About the Author
Samir Araújo is an AI/ML Solutions Architect at AWS. He helps customers creating AI/ML solutions which solve their business challenges using AWS. He has been working on several AI/ML projects related to computer vision, natural language processing, forecasting, ML at the edge, and more. He likes playing with hardware and automation projects in his free time, and he has a particular interest for robotics.
Amazon Aurora PostgreSQL Auto Vacuum 이해하기 (2/2)lg...글전문가게시됨 20일 전lg...
AWS Aurora Database Parallel Query 활성화 방법 및 특징lg...글
AWS Aurora Global Database를 사용한 Cross-Region 장애조치lg...글
AWS Aurora DB 클러스터 엔드포인트 연결 종류 4가지lg...글
Gateway Load Balancer(GWLB)를 이용한 Architecture 사례와 장단점lg...글전문가hsjeonlg...게시됨 21일 전lg...
S3 Cross Account 사용 시 ACL 설정 관련 문제lg...수락된 답변teduri90lg...질문됨 5일 전lg...
AWS API 연동이 안됩니다.lg...Busanweblg...질문됨 5년 전lg...
AWS 크레딧 요금 관련 문의lg...kro551lg...질문됨 5년 전lg...
aws cloudsearch suggest processing timelg...lducodinglg...질문됨 21일 전lg...
AWS sagemaker real time endpoint는 request를 동시에 처리하지 않나요?lg...rePost-User-8411906lg...질문됨 4일 전lg...