AWS Bedrock APIs
Below is how you can instantiate AWS Bedrock as a provider. Amazon Bedrock is a fully managed service that makes FMs from leading AI startups and Amazon available via an API, so you can choose from a wide range of FMs to find the model that is best suited for your use case
All feedback functions listed in the base LLMProvider class can be run with AWS Bedrock.
Bases: LLMProvider
Source code in trulens_eval/trulens_eval/feedback/provider/bedrock.py
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80 | class Bedrock(LLMProvider):
model_id: str
region_name: str
def __init__(
self,
*args,
model_id="amazon.titan-tg1-large",
region_name="us-east-1",
**kwargs
):
# NOTE(piotrm): pydantic adds endpoint to the signature of this
# constructor if we don't include it explicitly, even though we set it
# down below. Adding it as None here as a temporary hack.
"""
A set of AWS Feedback Functions.
Parameters:
- model_id (str, optional): The specific model id. Defaults to
"amazon.titan-tg1-large".
- region_name (str, optional): The specific AWS region name. Defaults to
"us-east-1"
- All other args/kwargs passed to the boto3 client constructor.
"""
# TODO: why was self_kwargs required here independently of kwargs?
self_kwargs = dict()
self_kwargs.update(**kwargs)
self_kwargs['model_id'] = model_id
self_kwargs['region_name'] = region_name
self_kwargs['endpoint'] = BedrockEndpoint(
region_name=region_name, *args, **kwargs
)
super().__init__(
**self_kwargs
) # need to include pydantic.BaseModel.__init__
# LLMProvider requirement
def _create_chat_completion(
self,
prompt: Optional[str] = None,
messages: Optional[Sequence[Dict]] = None,
**kwargs
) -> str:
# NOTE(joshr): only tested with sso auth
import json
import boto3
bedrock = boto3.client(service_name='bedrock-runtime')
assert prompt is not None, "Bedrock can only operate on `prompt`, not `messages`."
body = json.dumps({"inputText": prompt})
modelId = self.model_id
response = bedrock.invoke_model(body=body, modelId=modelId)
response_body = json.loads(response.get('body').read()
).get('results')[0]["outputText"]
# text
return response_body
|
__init__(*args, model_id='amazon.titan-tg1-large', region_name='us-east-1', **kwargs)
A set of AWS Feedback Functions.
- model_id (str, optional): The specific model id. Defaults to
"amazon.titan-tg1-large".
-
region_name (str, optional): The specific AWS region name. Defaults to
"us-east-1"
-
All other args/kwargs passed to the boto3 client constructor.
Source code in trulens_eval/trulens_eval/feedback/provider/bedrock.py
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53 | def __init__(
self,
*args,
model_id="amazon.titan-tg1-large",
region_name="us-east-1",
**kwargs
):
# NOTE(piotrm): pydantic adds endpoint to the signature of this
# constructor if we don't include it explicitly, even though we set it
# down below. Adding it as None here as a temporary hack.
"""
A set of AWS Feedback Functions.
Parameters:
- model_id (str, optional): The specific model id. Defaults to
"amazon.titan-tg1-large".
- region_name (str, optional): The specific AWS region name. Defaults to
"us-east-1"
- All other args/kwargs passed to the boto3 client constructor.
"""
# TODO: why was self_kwargs required here independently of kwargs?
self_kwargs = dict()
self_kwargs.update(**kwargs)
self_kwargs['model_id'] = model_id
self_kwargs['region_name'] = region_name
self_kwargs['endpoint'] = BedrockEndpoint(
region_name=region_name, *args, **kwargs
)
super().__init__(
**self_kwargs
) # need to include pydantic.BaseModel.__init__
|