Hi guys,
I need some help and was wondering if anyone have some guideline steps or a guide for me to implement the Mixtral-8x7B-Instruct-v0.1 model (the LLM for my AI chatbot) using this GPU optimized library called vLLM (https://github.com/vllm-project/vllm) in AWS using EKS (Elastic Kubernetes Service).
This is urgent. Would really really appreciate any help with this. Many thanks.
Ps: if you have some sort of rough guidelines without the vLLM part, that would work too.
