Is it possible to use Anthropic's Claude model without using Amazon Bedrock?

Question

It seems that the Amazon Bedrock service is not yet FedRAMP so I would like to use Claude without Bedrock. I also need the Claude model running in my AWS account versus using Anthropic's API.  Is this possible?

Accepted Answer

Hi @Ashley Miller,

As of today, it is not possible to access Claude FM without Bedrock. It's not available on SageMaker JumpStart as well. If it seems okay, you can try other models from AI21 labs/Meta/Cohere from SageMaker JumpStart.

On the FedRAMP certification, Amazon Bedrock is undergoing 3rd party assessment. PFB the latest update here - https://aws.amazon.com/compliance/services-in-scope/FedRAMP/

Thanks,

Answer

If you could overcome the following hurdles, then it's possible to run Claude outside of Bedrock. 
1. The license. You would need to work through Anthropic to gain the ability to deploy Claude to SageMaker. Claude is not available through the typical channels like Jumpstart or HuggingFace. 
2. Instance size. As of this writing it may be possible to run Claude on a P5 instance, but I'm not aware of a publicly reference-able successful deployment on Sagemaker. If you can get the container/wights/artifacts it's possible to work backwards to find the right instance. size.
Please let us know if you are successful.

Is it possible to use Anthropic's Claude model without using Amazon Bedrock?

Relevant content