Skip to content

Doubts regarding Innovation Sandbox solution by AWS

-3

This is the source of truth → https://docs.aws.amazon.com/solutions/latest/innovation-sandbox-on-aws/solution-overview.html

My team has deployed the ISB and we are currently testing it to understand how it works in real scenarios and to identify any loopholes or caveats before providing it to the client. Before going deeper, this is the reference solution link: https://docs.aws.amazon.com/solutions/latest/innovation-sandbox-on-aws/solution-overview.html

Summary of the ISB solution The solution is designed to provide temporary, governed access to AWS accounts so users can experiment without needing their own AWS accounts. It focuses on controlled usage, cost governance, and lifecycle management of sandbox environments. There are 3 levels of access: Admin, Manager, and User.

Administrator has full control over the solution, including managing users, managers, and overall configuration of accounts and governance controls. Manager is responsible for approving or rejecting account lease requests and assigning sandbox accounts to users. User is the end consumer who requests access and uses the sandbox account within defined limits.

Blueprints can be created, which act as templates to standardize sandbox environments. These can include predefined configurations and governance controls such as SCPs, and are applied when provisioning accounts for lease.

Leased accounts are provided for a fixed duration and are governed by budget constraints.

Thresholds can be configured based on both budget and duration. Budget thresholds can trigger alerts or restrict account activity when nearing limits. Time-based thresholds control the lifecycle of the lease and can restrict access or end the lease when the duration expires.

At the end of the lease duration, the solution initiates cleanup of provisioned resources and the account is returned to the pool for reuse (it is not permanently deleted). Cost data is stored (for example in S3), but it is important to note that AWS billing data is not real-time and is delayed.

Currently, the major issue observed during testing is:

  1. There is a delay in reflecting actual costs in AWS billing tools like Cost Explorer.
  2. The final cost report generated at the end of a lease may not include the most recent usage (especially from the last few hours or last day). This is due to AWS billing data lag, not because the cost is lost. The cost will eventually appear in AWS billing, but it may not be attributed correctly to that specific lease or user in the ISB reporting.

How to address or think about this issue: since usage on the last day may not be reflected immediately, it creates a gap in cost attribution within the sandbox system. This does not mean the usage is free, but from the ISB reporting perspective, it may appear missing at the time of lease closure.

Example: User leases an account for 1 week with a budget of $50. Thresholds are configured to trigger near $49 and at lease expiration. User consumes approximately $5 per day for 6 days (total ~$30). On the last day, the user creates expensive resources.

Result: The ISB-generated report at lease end may only reflect the ~$30 usage. The remaining cost from the last day will appear later in AWS billing systems but will not be linked back to that lease in the ISB report.

This creates a cost gap where the actual spend and reported spend per lease will not match due to billing delays.

3 Answers
2

This is expected due to AWS billing delay (eventual consistency), so the solution is to design around it rather than trying to eliminate it.

Approach:

answered 2 months ago
1

You've identified a legitimate limitation of the Innovation Sandbox on AWS solution related to AWS billing data latency. This is an inherent characteristic of how AWS billing systems work rather than a flaw in the solution itself.

The Innovation Sandbox solution monitors spend patterns and sends automated alerts at defined thresholds to help maintain cost control. It can also restrict access or clean up resources when budget thresholds are approached. However, because AWS billing data is not available in real-time, there will always be a delay between when resources are consumed and when those costs appear in billing systems like Cost Explorer.

To address this challenge, consider implementing these approaches:

Buffer your budget thresholds: Set your alert and restriction thresholds lower than the actual budget limit to account for potential lag. For example, if the budget is $50, configure alerts at $40 and restrictions at $45 to provide a cushion for delayed billing data.

Extend the cleanup window: Rather than immediately recycling accounts at lease expiration, consider adding a grace period (such as 24-48 hours) before final cost reporting and account cleanup. This allows more billing data to settle before generating final reports.

Communicate expectations: Make users and managers aware that final cost reports represent costs known at the time of lease closure, and that additional costs may appear in organizational billing later. This sets appropriate expectations about cost attribution.

Leverage Service Control Policies: Use the solution's ability to limit access to excessively expensive services through SCPs as a preventive measure, reducing the risk of surprise costs in the final hours of a lease.

Monitor at the organizational level: While individual lease reports may have gaps, all costs will eventually appear in your organization's overall AWS billing. Maintain separate tracking at the organizational level to reconcile any discrepancies.

The solution's automated cleanup mechanisms and standardized policies help minimize this issue, but the AWS billing delay is a constraint that requires operational processes to manage effectively.
Sources
Create temporary sandbox environments with configurable security and spend monitoring controls - Innovation Sandbox on AWS
Features and benefits - Innovation Sandbox on AWS

answered 2 months ago
EXPERT
reviewed 2 months ago
0

Currently, we can not have fresh cost data faster than 8-24 hours. The solution uses only the Cost Explorer API, with no other options. There is a GitHub issue where developers were informed about such an issue. I hope they will find a solution soon

EXPERT
answered 2 months ago
  • Oh Okay, I thought this was an untracked issue

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.