New user sign up using AWS Builder ID
New user sign up using AWS Builder ID is currently unavailable on re:Post. To sign up, please use the AWS Management Console instead.
Amazon Q Operational Best Practices - Logging and Monitoring
Logging and monitoring best practices to operate Amazon Q services, including Amazon Q Business and Amazon Q Developer, at scale.
Amazon Q Introduction
Amazon Q is the most capable generative AI–powered assistant for accelerating software development and leveraging your companies' internal data. There are 2 Amazon Q products currently - Amazon Q Business and Amazon Q Developer. You can also find and use Amazon Q in different AWS services like Amazon QuickSight, Amazon Connect, and AWS Supply Chain. Together, Amazon Q helps all employees in your organization, like software developers, business intelligence analysts, contact center employees, supply chain analysts, and any AWS user, get insights on their data and accelerate their task, whether it's asking questions about the internal/internet data, getting guidance and troubleshooting AWS services, or developing software and more.
Amazon Q sits at the Applications layer, top layer of AWS’s three-layered generative AI stack. Since Amazon Q is a fully managed generative AI application, understanding native and integrated logging and monitoring features and operational best practices are important for your organization administrators to operate generative AI solutions at scale.
In this article, we will discuss logging and monitoring features and best practices in Amazon Q Business and Amazon Q Developer. Since Amazon Q is a new rapidly evolving service, I recommend you checking out the latest features in AWS what’s new page and the Amazon Q user guide.
Amazon Q Logging and Monitoring Overview
Monitoring and logging are center piece for operational excellence in the cloud. For Amazon Q, you can integrate with services like AWS CloudTrail, Amazon CloudWatch, AWS Health and leverage Amazon Q built-in features like analytics dashboards, subscription management, and prompt logging to monitor system health and performance, and discover trends and insights of your Amazon Q application environments.
We will look into these service integrations, features, and best practices below.
Amazon Q Logging and Monitoring Best Practices
1. Get Amazon Q usage trends and insights from the analytics dashboards
You can view Amazon Q Business and Amazon Q Apps key usage metrics like Average daily active users, Total queries and trends like Customer feedback trend in these customizable analytics dashboards. The analytics dashboards are located in the Insights section within your Amazon Q Business application admin console. Similarly, you can view the Amazon Q Developer dashboard in the admin console.
You can get started in here for Amazon Q Business, and here for Amazon Q Developer.
2. View and manage Amazon Q user subscriptions from the subscription management console
You can view and manage Amazon Q Business and Amazon Q Developer subscriptions from the central Amazon Q subscriptions page. For Amazon Q Business, user subscriptions are created and managed in each Amazon Q Business application, but you can view the aggregated application user subscriptions in the Amazon Q subscriptions page. You can turn on View subscriptions from member accounts in Settings in the Amazon Q service if you use AWS Organizations and want to view an aggregated list of subscriptions in your organization. After subscription in AWS Organizations management account with an organization instance of AWS IAM Identity Center deployed, users will be able to access Amazon Q in all member accounts they have access to and only billed once.
You can get started in here for Amazon Q Business, and here for Amazon Q Developer.
3. Enable Amazon Q conversation and prompt logging for auditing and analytics
If you have use cases to log user prompts in Amazon Q like auditing, debugging, analytics, and compliance, you can enable Amazon Q Business user conversation logging and prompt logging in Amazon Q Developer in the IDE. Since the prompt may contain sensitive or personally identifiable data, it’s recommended to filter this data in the service configuration or mask this data using Amazon CloudWatch Logs masking policies.
You can get started in here for Amazon Q Business, and here for Amazon Q Developer.
4. Log Amazon Q API calls using AWS CloudTrail
Amazon Q is integrated with AWS CloudTrail. Using the information collected by CloudTrail, you can determine the request that was made to Amazon Q, the IP address from which the request was made, who made the request, when it was made, and additional details.
You can get started in here for Amazon Q Business, and here for Amazon Q Developer.
5. Monitor Amazon Q metrics and analyze logs using Amazon CloudWatch
Amazon Q including Amazon Q Business, Amazon Q Apps, and Amazon Q Developer send telemetry data to Amazon CloudWatch in real time. You can use CloudWatch features like Amazon CloudWatch alarm that integrates with Amazon Simple Notification Service to send notifications to your operation team.
You can view the available metrics and get started in here for Amazon Q Business, and here for Amazon Q Developer.
You can also triage the Amazon Q logs like the conversation and prompt logs mentioned above in Amazon CloudWatch Logs. You can use other Amazon CloudWatch Logs features like Amazon CloudWatch Logs Insights to search and analyze your Amazon Q log data. Moreover, for Amazon Q Business, you can view and troubleshoot document-level sync reports like any data source indexing issue in Amazon CloudWatch Logs. You can get started in here.
For Amazon Q Developer, you can enable and deliver customization logs to Amazon CloudWatch Logs, along with Amazon Simple Storage Service (Amazon S3) and Amazon Data Firehose to troubleshoot any issue with Amazon Q Developer customizations. You can get started in here.
6. Monitor Amazon Q service health and account related events in AWS Health Dashboard
Just like other AWS services, you can view Amazon Q service status through AWS Health Dashboard, which displays all reported service events for services across all AWS Regions. You can view your account and organization specific events and notifications in the AWS Health Dashboard after you sign in to your AWS account. If you have question or need support during an event, you can contact AWS Support by opening a support case.
You can get started with AWS Health Dashboard here.
7. Analyze and visualize Amazon Q logs in log analytics tools
Once you enabled Amazon Q logs like prompt logs and CloudTrail logs, you can use Amazon Athena to query and analyze those logs by creating a table and using standard SQL. Additionally, you can build custom dashboard in Amazon QuickSight to visualize those logs by creating a dataset using Amazon Athena data.
You can also consider using Amazon OpenSearch Service, a fully managed OpenSearch service in AWS, to analyze and visualize Amazon Q and CloudTrail logs. For example you can ingest logs in S3 to OpenSearch via OpenSearch Ingestion and perform log analytics. To get started see this AWS blog post. Another example, you can leverage zero-ETL integration between Amazon CloudWatch and Amazon OpenSearch to analyze CloudWatch logs in OpenSearch, build indexes and dashboards, and also create automatic curated dashboards like CloudTrail logs dashboard in a few minutes. To get started see here.
Conclusion
Through this article you can understand the logging and monitoring features and best practices when you operate Amazon Q services, including Amazon Q Business and Amazon Q Developer, at scale. You can get started for each feature and best practice by following the get started links in each section above. You can share a question or comment in here.
This article is co-authored with Adam Hinman, Sr. Technical Account Manager at AWS.
Relevant content
- asked 7 months agolg...
- AWS OFFICIALUpdated 22 days ago
- AWS OFFICIALUpdated a month ago
- AWS OFFICIALUpdated 21 days ago
- AWS OFFICIALUpdated 15 days ago