Import customized fashions in Amazon Bedrock (preview)

Voiced by Polly

With Amazon Bedrock, you’ve got entry to a selection of high-performing basis fashions (FMs) from main synthetic intelligence (AI) corporations that make it simpler to construct and scale generative AI functions. A few of these fashions present publicly out there weights that may be fine-tuned and customised for particular use instances. Nonetheless, deploying custom-made FMs in a safe and scalable means just isn’t a straightforward job.

Beginning at the moment, Amazon Bedrock provides in preview the aptitude to import customized weights for supported mannequin architectures (reminiscent of Meta Llama 2, Llama 3, and Mistral) and serve the customized mannequin utilizing On-Demand mode. You possibly can import fashions with weights in Hugging Face safetensors format from Amazon SageMaker and Amazon Easy Storage Service (Amazon S3).

On this means, you should utilize Amazon Bedrock with present custom-made fashions reminiscent of Code Llama, a code-specialized model of Llama 2 that was created by additional coaching Llama 2 on code-specific datasets, or use your information to fine-tune fashions on your personal distinctive enterprise case and import the ensuing mannequin in Amazon Bedrock.

Let’s see how this works in observe.

Bringing a customized mannequin to Amazon Bedrock
Within the Amazon Bedrock console, I select Imported fashions from the Basis fashions part of the navigation pane. Now, I can create a customized mannequin by importing mannequin weights from an Amazon Easy Storage Service (Amazon S3) bucket or from an Amazon SageMaker mannequin.

I select to import mannequin weights from an S3 bucket. In one other browser tab, I obtain the MistralLite mannequin from the Hugging Face web site utilizing this pull request (PR) that gives weights in safetensors format. The pull request is presently Able to merge, so it may be a part of the principle department whenever you learn this. MistralLite is a fine-tuned Mistral-7B-v0.1 language mannequin with enhanced capabilities of processing lengthy context as much as 32K tokens.

When the obtain is full, I add the information to an S3 bucket in the identical AWS Area the place I’ll import the mannequin. Listed below are the MistralLite mannequin information within the Amazon S3 console:

Console screenshot.

Again on the Amazon Bedrock console, I enter a reputation for the mannequin and hold the proposed import job title.

Console screenshot.

I choose Mannequin weights within the Mannequin import settings and browse S3 to decide on the placement the place I uploaded the mannequin weights.

Console screenshot.

To authorize Amazon Bedrock to entry the information on the S3 bucket, I choose the choice to create and use a brand new AWS Identification and Entry Administration (IAM) service position. I take advantage of the View permissions particulars hyperlink to examine what might be within the position. Then, I submit the job.

About ten minutes later, the import job is accomplished.

Console screenshot.

Now, I see the imported mannequin within the console. The listing additionally exhibits the mannequin Amazon Useful resource Title (ARN) and the creation date.

Console screenshot.

I select the mannequin to get extra info, such because the S3 location of the mannequin information.

Console screenshot.

Within the mannequin element web page, I select Open in playground to check the mannequin within the console. Within the textual content playground, I sort a query utilizing the immediate template of the mannequin:

<|prompter|>What are the principle challenges to assist a protracted context for LLM?</s><|assistant|>

The MistralLite imported mannequin is fast to answer and describe a few of these challenges.

Console screenshot.

Within the playground, I can tune responses for my use case utilizing configurations reminiscent of temperature and most size or add cease sequences particular to the imported mannequin.

To see the syntax of the API request, I select the three small vertical dots on the prime proper of the playground.

Console screenshot.

I select View API syntax and run the command utilizing the AWS Command Line Interface (AWS CLI):

aws bedrock-runtime invoke-model 
--model-id arn:aws:bedrock:us-east-1:123412341234:imported-model/a82bkefgp20f 
--body "prompter" 
--cli-binary-format raw-in-base64-out 
--region us-east-1 

The output is just like what I bought within the playground. As you’ll be able to see, for imported fashions, the mannequin ID is the ARN of the imported mannequin. I can use the mannequin ID to invoke the imported mannequin with the AWS CLI and AWS SDKs.

Issues to know
You possibly can deliver your personal weights for supported mannequin architectures to Amazon Bedrock within the US East (N. Virginia) AWS Area. The mannequin import functionality is presently out there in preview.

When utilizing customized weights, Amazon Bedrock serves the mannequin with On-Demand mode, and also you solely pay for what you utilize with no time-based time period commitments. For detailed info, see Amazon Bedrock pricing.

The flexibility to import fashions is managed utilizing AWS Identification and Entry Administration (IAM), and you’ll enable this functionality solely to the roles in your group that have to have it.

With this launch, it’s now simpler to construct and scale generative AI functions utilizing customized fashions with safety and privateness inbuilt.

To be taught extra:


Recent Articles

Related Stories

Leave A Reply

Please enter your comment!
Please enter your name here