Integrating Foundation Models into Code with Amazon Bedrock

Table of Contents

• Importing Necessary Libraries and Creating the Bedrock Client
• Sending a Simple Prompt to a Foundation Model
• Using Different Foundation Models
• Getting Streaming Output
• Conclusion

Importing Necessary Libraries and Creating the Bedrock Client for AI Generation

To get started with Amazon Bedrock, we first need to import the Boto3 library to enable Python SDK access. We'll also import the JSON library since the responses from Bedrock models are in JSON format. After the imports, we create a Boto3 client for the bedrock-runtime service, specifying the AWS region we want to use.

The Bedrock client gives us an entry point to call the various Bedrock AI models. With just this client created, we are ready to start sending prompts and getting AI-generated responses.

Importing Boto3 and JSON

The code to import Boto3 and JSON is:

import boto3
import json

Boto3 enables access to Bedrock through the Python SDK. JSON will help parse responses later.

Creating the Bedrock Client

Next we create the Bedrock client:

bedrock_runtime = boto3.client('bedrock-runtime', region_name='<my-region>')

This gives us access to the Bedrock invoke_model API to call AI models.

Sending a Simple Prompt to a Foundation Model

To send a prompt, we first need to construct it properly for the chosen AI model. We can view model-specific API details in the Bedrock console which even provides code snippets. After setting up the keyword arguments for invoke_model based on what the console provides, we make the actual API call to generate the response.

The initial response is a streaming body object that needs to be processed to extract the generated text. We demonstrate grabbing the full response body and parsing out the completion text for our 'hello' prompt.

Constructing the Prompt

We start with a simple prompt like:

prompt = 'hello'

This will be passed to the model to generate a response.

Getting the API Request from the Console

In the Bedrock console, we select a model like Jurassic-2 Ultra. Then under the Playground we click View API Request to get code for invoking the model:

{
  'modelId': 'arn:aws:britn:us-west-2:<acct-id>:model/j2-ultra/prototype/1.0.0',
   ...
}

Invoking the Model and Processing the Response

We make the invoke_model call by plugging our prompt into the keyword arguments:

response = bedrock_runtime.invoke_model(**kwargs_from_console)

Then we parse the response body to extract just the generated text:

completion = response['body']['completions'][0]['data']['text']
print(completion)

This prints the model's response to our 'hello' prompt.

Using Different Foundation Models in Bedrock

Switching models in Bedrock is easy - we just update the invoke_model parameters and prompt structure expected by the new model. We show an example prompt for text summarization sent to the Claude model by configuring based on what the Bedrock console provides.

Sending a Summarization Prompt to Claude

For Claude we use a prompt structure like:

prompt = 'Human: <summarization-instructions>...

Then get updated invoke_model kwargs from the console configured for Claude. The final step of parsing the completion text stays the same across models.

Getting Streaming Output from Bedrock Models

Instead of invoke_model which returns the full generated text on completion, invoke_model_with_response_stream gives a event stream as text is being produced. This allows processing output as it is incrementally generated instead of waiting for the model to finish.

Invoking Claude with Streaming Response

We call the streaming version of the API:

response_stream = bedrock_runtime.invoke_model_with_response_stream(**kwargs)

This starts streaming back events as text is generated.

Processing the Streaming Output

We process the stream of text chunks like:

for event in response_stream['body']:
  chunk = event['chunk']
  print(chunk)

This prints each chunk of text as soon as it is ready rather than waiting for the full generated article.

FAQ

Q: How do I create a Bedrock client in Python?
A: Use the Boto3 client, specify the 'bedrock-runtime' service name, and choose the desired AWS region to create your Bedrock client.

Q: Where can I find the API parameters for different models?
A: You can find code snippets with the API parameters for different models in the Bedrock console playgrounds. Just select the model and view the API request.

Q: How do I process the streaming response from a model?
A: Use invoke_model_with_response_stream() to get the streaming response. Then iterate through the events to access the text chunks.

Q: What kind of prompts can I send to foundation models?
A: You can send a wide variety of prompts, including summarization, content generation, question answering, and more. Be creative!

Q: Do I need to format prompts differently for certain models?
A: Yes, some models like Claude have specific prompt formats. The console playground provides properly formatted snippets.

Q: Can I use Bedrock with languages other than Python?
A: Yes, Bedrock provides SDKs and integrations for various languages beyond Python like JavaScript and Java.

Q: Is Bedrock expensive to use?
A: No, Bedrock offers very cost-effective pricing for accessing state-of-the-art foundation models.

Q: Are there limits on model usage?
A: Yes, Bedrock enforces limits on concurrent model invocations and total monthly invocations depending on your account.

Q: Can I train my own models?
A: Currently Bedrock only allows invoking existing models, but custom model training support is on the roadmap.

Q: Where can I learn more about Bedrock?
A: Check the Bedrock product page and documentation for more details, tutorials, SDK references and more.