Questions tagged with Cost Optimization

Cloudformation create-stack generates an Athena access denied while writing error. The problem is generated when writing to the athena-results bucket. I'm logged in with a SSO role with AdministratorAccess access via CLI. I can create the specified object from the command line via "aws s3 cp" and I'm able to execute "aws athena start-query-execution" without trouble. It's only via cloudformation. Bellow the specific error: ResourceStatus: CREATE_FAILED ResourceStatusReason: 'Resource handler returned message: "[Simba][AthenaJDBC](100071) An error has been thrown from the AWS Athena client. Access denied when writing to location: s3://cost-athena-results-123456789012/8fefd451-2a3f-4bc9-881e-84061de8db91.csv [Execution ID: 8fefd451-2a3f-4bc9-881e-84061de8db91]" (RequestToken: b0d4b7d5-998b-6ca8-22c6-657fa2433fe8, HandlerErrorCode: null)' ResourceType: AWS::QuickSight::DataSource

Hi, We are using lambda in a docker (ECR), The code is in python with a lot of dependencies (Docker size 700mb) We noticed that our code execution is very small (+-650ms) but the billed time is very high in comparison (4500ms) with a init time of (+- 3800ms) So basically init time is 80% of our billed time for each execution. My previous pipeline : Every 45min we spin 200 instances of the lambda with a different argument at the same time. (I guess 200 cold start then?) My new pipeline : Every 45min i spin the same lambda with no argument (1 Cold start) and this lambda spin 200 times the same lambda but with an argument (200 warm start?) but with the new pipeline the problem is still the same and I dont see any improvement: REPORT RequestId: f9d67348-2deb-410e-b874-b29e8b3569b2 Duration: 653.58 ms Billed Duration: 4353 ms Memory Size: 300 MB Max Memory Used: 284 MB Init Duration: 3698.46 ms So here are my 3 questions then: - Is 3800ms a normal cold start time? - In my new pipeline why is it not improving the init duration? - What approach do you recommend to reduce/fix the cost? Thank you !

Hello Every one, I'm starting with AWS by trying to build a personal blog with downloadable content from S3 (PDF and images mostly for now) . I'm concerned at this point by data transfer costs when users download this content. Is my concern legitimate ? is it better to avoid downloadable content in this case if it will incur unpredictable costs ? Thanks in advance

Following the well architected labs 200: Cost and usage analysis I get the following error when adding partitions in Athena Query Editor: ``` MSCK REPAIR TABLE `cost_optimization_10XXXXXXXX321`; ``` and it returned the following error: > FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null This query ran against the "costfubar" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 856e146a-8b13-4175-8cd8-692eef6d3fa5 The table was created correctly in Glue with ``` Name cost_optimization_10XXXXXXXXX21 Description Database costfubar Classification parquet Location s3://cost-optimization-10XXXXXXX321// Connection Deprecated No Last updated Wed Apr 20 16:46:28 GMT-500 2022 Input format org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat Output format org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat Serde serialization lib org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe Serde parameters serialization.format 1 Table properties sizeKey 4223322objectCount 4UPDATED_BY_CRAWLER costfubarCrawlerSchemaSerializerVersion 1.0recordCount 335239averageRecordSize 27exclusions ["s3://cost-optimization-107457606321/**.json","s3://cost-optimization-1XXXXXXXX21/**.csv","s3://cost-optimization-107457606321/**.sql","s3://cost-optimization-1XXXXXXXX321/**.gz","s3://cost-optimization-107457606321/**.zip","s3://cost-optimization-107457606321/**/cost_and_usage_data_status/*","s3://cost-optimization-107457606321/**.yml"]CrawlerSchemaDeserializerVersion 1.0compressionType nonetypeOfData file ``` and has the following partitions shown in Glue: ``` partition_0 partition_1 year month detailed-cur-1XXXXXXXX57 detailed-cur-1XXXXXXXX57 2018 12 View files View properties detailed-cur-1XXXXXXXXX57 detailed-cur-1XXXXXXXXX57 2022 4 View files View properties detailed-cur-1XXXXXXXXX57 detailed-cur-1XXXXXXXXX57 2018 11 View files View properties detailed-cur-1XXXXXXXX57 detailed-cur-1XXXXXXXX57 2018 10 View files View properties ```

Hello, We build some app where our java job sent request to external app and in response we have some body as xml or json with data what we need. This is classified as Data Transfer ?

I have configured CloudWatch Agent to send metrics to Cloudwatch for ABC instance for every 60 Seconds but **I haven't enable detailed monitoring** in ABC instance does AWS charge in this case?

Hello guys! I was able to successfully recover my instances after the outage in the premise office, currently, all my machines are running in EC2 and source servers are shown as "Ready, lag 9d" "Stalled" and I keep being charged for replica EBS volumes, as well as AWS Replication Instances (t3.small). While I don`t have any info on when it will be possible to perform the failback, I`d like to cut some costs and remove replica EBS volumes. I was thinking about the action "Disconnect from AWS" but it does not work with the message: > Some Source servers could not be processed: > s-1ed6*****9: Cannot disconnect Source Server s-1ed******9 because it has a Recovery Instance. What would be the correct action in such a case? (remove source server\remove just the volumes\something else) (the goal to cut the costs while keeping the possibility of failback later). Thanks! Regards,

I have a T3 Unlimited EC2 instance tagged with some tag *project:xyz*. The cost of "regular" CPU credits is being correctly shown tagged as *project:xyz* in Cost Explorer, so I know for a fact that the tag is correctly is applied. However, when the instance uses the "unlimited" CPU credit portion this cost is shown in Cost Explorer as USE2-CPUCredits:t3a without any tags. What am I missing?

we had used Server Migration Service in past to migrate On Premise servers to AWS and this activity created many AMI with name *Created by SMS* can we delete that?

Hi Guys, does anyone know if it's possible to share RI/SavingPlans between different AWS Organizations? (someone told me that it's possible but I can't get documentation on that) - I am pretty sure that is not possible but I want to be sure. Thanks in advance! :)

If I have multi-measure records where most of the measures are empty, does that still affect my storage cost as if they were populated? Example: Will a multi-measure record of 250 empty values and 2 populated values take up storage cost for 252 values or 2 values? Also, if I have single-measure records with different data types, does the unused measure value have a cost to it? Example: "*measure_name='my_bigint', measure_value::double=NULL, measure_value::bigint=5000*", does the NULL take up 8 bytes of storage?

I observed that we are getting a big bill for AWS Config and to be honest, I have no idea what it is or why we have configured it? My question is: * Is it safe to turn off AWS Config? Are there any consequences to worry about? * How to turn it off? I found this post: https://forums.aws.amazon.com/thread.jspa?threadID=307384. Is it correct? Thanks

I trying to find the top contributors to traffic through the NAT gateway in my VPC. Therefore I've obtained a IP list through VPC flow logs. (https://aws.amazon.com/premiumsupport/knowledge-center/vpc-find-traffic-sources-nat-gateway/). However there are some IPs that I can't find either using the network interface console or describe-network-interfaces aws cli command. (https://aws.amazon.com/premiumsupport/knowledge-center/vpc-find-owner-unknown-ip-addresses/) Is there any other way to find them?

I'm following the steps provided from this [AWS Workshop](https://catalog.us-east-1.prod.workshops.aws/v2/workshops/fd889151-38aa-4fe2-a29d-d5fa557197bb/en-US/prerequisites) to create a **Cudos Dashboard** for cost reporting. When I get to step "**Prepare CUR & Athena Integration**" and attempt to create a new stack using CloudFormation I get the following error on the AWSCURInitializer event: `Resource handler returned message: "Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of [50]. (Service: Lambda, Status Code: 400, Request ID: a7a45878-caca-4e00-9df4-026caea7926a, Extended Request ID: null)" (RequestToken: 00c6a110-6f9b-2435-0682-7da69eaf96f2, HandlerErrorCode: InvalidRequest)` I think the key takeway from the erorr is this: *Specified ReservedConcurrentExecutions for function decreases account's UnreservedConcurrentExecution below its minimum value of 50* Anyone familiar with this error and know how to get around it? Thanks!

I see an increase of `Data Transfer $0.010 per GB - regional data transfer - in/out/between EC2 AZs or using elastic IPs or ELB` (approximately 119,626.889 GB) and I would like to determine the root cause. I am using EKS distributed in 3 AZs in one region, RDS, EC2s. The cost doubled from previous months with no apparent reason. How to determine what is causing this? and what are the best practices to reduce this cost?

I am attempting to use the AWS Pricing Calculator to estimate retrieving objects from S3, and it asks for the number of LIST requests to S3 and the number of GET, SELECT from S3. The calculator also asks for the estimated amount of data returned and scanned. I am unsure of how to estimate these numbers. If I want to retrieve 1,000 objects, would it incur 1,000 GET and SELECT requests but only one LIST request? I am assuming *data returned *is the amount of data transferred out, but what is *data scanned*? Any clarification would be greatly appreaciated.

My client wants to use application streaming to let employees access five or six corporate applications. Regarding the use of AppStream 2.0 I understand that all the applications can be installed in the same image. Does the number of applications have an impact on the price (apart from the size of the image)? Also, when using AppStream 2.0 how does the consistency of the data work? Where is the data stored? Thanks!

My client has asked me to estimate how much he would save if he moved his Ruby on Rails monolithic application to an environment where they only use microservices (EKS and Lambda). The calculation does not need to be exact although I want it to be realistic. I don’t know whether to start using the pricing calculator or maybe just use some example of other clients and give a benchmark. What are your thoughts on the best approach? Thanks

A customer wants to make a copy of 100TB of data: - From an S3 bucket in one account - To Deep Archive in a second account (but same region) The two accounts are not part of the same Organization. Is it possible to make a direct copy from an S3 bucket in one account directly to Deep Archive in a second account? Or do we have to copy first to an S3 bucket in the second account and configure a lifecycle policy to transition to Deep Archive in the second account? The goal is to minimize the pricing.

How can customers find out the S3 bucket with the highest cost or sort the cost for each S3 bucket? A customer is currently going through cost optimization and they want to find out the S3 bucket with the highest cost. They are also looking for a way to sort the cost for the S3 buckets. Thanks for any information!

Based on this document, https://aws.amazon.com/athena/pricing/ > Amazon Athena queries data directly from Amazon S3. There are no additional storage charges for querying your data with Athena. **You are charged standard S3 rates for storage, requests,** and data transfer. Would customer be charged for S3 GET Operation when using Athena to query data in S3 in addition to the total size of data being scanned?

Under [Amazon RDS Metrics][1] we have two metrics : - ReadIOPS - WriteIOPS My understanding is that the term "IOPS" refers to read and write operations collectively, so I'm a bit confused why there are two metrics here with the "IOPS" suffix. Are these just unfortunately named in a confusing way, or is there something more to the metric value that I'm missing ? Background : I'm trying to reason about IOPS utilization for an RDS DB with PIOPS storage. e.g. (Read + Write)/Provisioned as a utilization percentage. [1]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MonitoringOverview.html#metrics_dimensions