Questions tagged with Amazon Timestream

I have IoT things sending timestamps in GMT to a TimeStream table. This means that the data is corrected to GMT no matter where the data was collected. Are there time zone shift functions that I am missing? The only thing I can find (and works OK) is using `time - 5h`. Are there any functions that correct based on an actual timezone?

I have generated a timestream database and table, but am struggling to write data to the table. The data comes from an mqtt topic, and can be seen in the cloudwatch log, so I know my select statement is sound. I think i have given the IAM role the correct permissions. This answer [https://repost.aws/questions/QUmWAjLqG7RwGHCSL2tMrXvw/iot-core-rules-to-map-timestamp-to-timestream#COMoUviJP_TGCAgZvcltBQUw](by @Greg_b) describes how to enter the 'timestamp value' but does not say what is needed in the 'timestamp unit' field. I am a complete newcomer to AWS so apologies if my question is not worded correctly. I would really appreciate some help with this! One further question, why, after i have deactiveted a rule, does it still appear in the cloudwatch logs?

I have a Timestream table used for testing in a development environment. The table is set to 1 hour memory store retention-- the minimum-- and 30 days of magnetic store retention. I also have magnetic writes enabled. We have custom code to restore data from a production table archive, saved as hive-partitioned, parquet files in S3, to the development table. When I process 10 days of records, about 1.2M records, and write them to the development table it seems the eventual consistency is really, very "eventual". The writes occur without error over the course of a minute or two, but we don't see the data until several hours later or sometimes even the next day. What is the expected, consistent read latency for magnetic writes in Timestream?

Hi aws re:Post! I used the AWS Timestream Web App Query Editor while developing my CDK IaC to test certain queries - specifically, I run the following and receive 1000 rows in response: ``` SELECT measure_value::varchar FROM DATABASE.TABLE ORDER BY time LIMIT 1000 ``` However the same exact query sent in NodeJS via `aws-sdk/TimestreamQuery` returns 0 rows in response. If I remove `ORDER BY time` such that my query is: ``` SELECT measure_value::varchar FROM DATABASE.TABLE LIMIT 1000 ``` the NodeJS Query client receives the same data as in the AWS Timestream Web App Query Editor. This is very bizarre behavior- I have not found any mention of it whatsoever in the context of AWS Timestream, and the only contextual results I found on the internet was [MySQL Bug Report #70466](https://bugs.mysql.com/bug.php?id=70466) ([accompanying stack overflow post](https://stackoverflow.com/questions/19086553/query-returns-no-results-only-when-order-by-added)). This really does not seem like a software configuration issue - to reiterate, if I omit `ORDER BY time` my NodeJS client behaves the same as my Web App client. Any insight on how to resolve this would be greatly appreciated!

I am looking for an option to have aggregation over timeseries to be performed only for weekdays, like Monday, Tuesday and etc. I tried DAYNAME function but it doesn't seem to exist. Is there an alternative to that function or any other way to achieve what I am after? Would be also interesting to find out how to get aggregation limited from 06:00 to 16:00 every day for example, but this might be a separate question.

I have a cron job that exports a list of number of messages per user frequencies such as: * 10 * 1 * 5 * 2 * 8 * 3 (Which means: 8 users have posted 3 messages) I'd like to have a histogram chart which displays last snapshot's data (ie. bar, mean, avg). I was thinking of TimeStream, but it actually accumulate. What is the recommended storage for this usecase?

Is there a way to convert Interval type to total seconds, minutes, etc? I would like to be able to ouput the difference in seconds between two Timestamps For instance (note I am just denoting the type with the :: syntax): (2022-04-29 13:15:53::Timestamp) - (2022-04-29 12:58:30::Timestamp) = (0 00:17:23.000000000::Interval Day to Seconds) Is there way to convert these values to other types? I have tried cast(x as int), that gives "Cannot cast interval day to second to integer". I am seeing little documentation on the actions that can be taken on Interval types. Is there more significant documentation than the following? https://docs.aws.amazon.com/timestream/latest/developerguide/supported-data-types.html https://docs.aws.amazon.com/timestream/latest/developerguide/conversion-functions.html https://docs.aws.amazon.com/timestream/latest/developerguide/date-time-functions.html Having a complete SQL reference document for Timeseries syntax would be great!

I have an AWS TimeStream table defined as follows: * user_id (varchar) * measure_name (varchar) * time (timestamp) * measure_value::boolean (boolean) (`measure_name` being an event name) When I imported it in QuickSight however I only get the mesure in the fields list, not the dimension: * measure_name * time My goal is to have charts of the number of unique users per day for example. How can I import the dimensions from TimeStream?

Using Timestream with Grafana. I want a graph to display the count of records a machine has cycled (one record per cycle) on a daily basis. Displaying this data in last 24 hours is confusing because the bins keep changing based on current time. I need my binds to be calculated from midnight of that day. My query looks like this now: ``` SELECT BIN(time, 1d), count() AS cycles FROM "MfgRecords"."CycleCounts" where WorkCenter = 'MyMachine' and measure_name = 'Cycled' and measure_value::boolean = true group by BIN(time, 1d) order by BIN(time, 1d) desc ``` How would that be written? Also, same question but for weeks. I want to display past weeks and the current week SO FAR since SUnday at Midnight. How? Lastly, is there a time specification like "1d" or "15m" only for most recent dayname like "Sun"?

Hello, I wrote a python code that pushes cryptocurrency prices into AWS TimeStream. It was working fine until 20 seconds later where i am getting an error: 'dimension name and value exceeds the maximum supported length for dimension names and values? See quotas in timestream developer guide for additional information. I am storing an entire orderbook for each timestamped record. They look like this: symbol | ask_size ||. best_ask_timestamp || bid_size || asks. || best_bid_timestamp || time || bids || measure_value:varchar Those are my field names 'BTCUSD' || 0.603747 || 1649961447.49376 || 0.59564627 || [[164996, 398741, 1628566, ...] So literally the value for asks is a list of list that i am storing as a plain text. Is this now allowed? My goal is that when i query per timestamp, i will get a record showing the entire orderbook and best asks and bids etc. so in summary, my asks and bids are both a length of about 20 records. i.e asks = [[2, 3, 4, 5, 55667, ..., 20]] same with bids. Right now, when i am running my program, i am getting an error after 2 minutes: An error occurred (ValidationException) when calling the WriteRecords operation: The dimension name and value exceeds the maximum supported length for dimension names and values. See Quotas in the Timestream developer guide for additional information.

Hi, (Quite new to timestream, old hat with SQL) I've added Millions (80M+) rows of data to a timestream table around 3 days ago. The data is historical, and goes back to the late 1800s. When querying it with: ``` SELECT * FROM <database>.<table> ORDER BY time ASC LIMIT 1 ``` The earliest record it shows 'time' is in the year 2016 (this is actually around the date of the first record inserted into the table), there are however millions of records before this date (but these were added later to the table), I've confirmed all the records exist in the table by running: ``` SELECT count(id) FROM <database>.<table> ``` Which results in a count equal to all the rows I've added (80M+) The table is setup to hold data in memory for 6 months, and in magnetic store for 200 years. Magnetic writes are also turned on. Why does this behaviour happen? are query results or records cached/indexed by timestream in some way, if so is there a way to clear this cache or documentation about it? Is it the normal behaviour of the magnetic store, as its such a large dataset added recently, is magnetic store still organising it? Thanks, James

Hi everyone, **i got **a s3 bucket which continuously generates csv-data, every minute or so. I now want to visualize this big-data via my grafana instance. First **i tried **to do that with aws athena and a glue crawler. But this solution is not very suitable after every query inside my grafana instance generates high costs inside my s3 bucket for tier2 get requests. **Now the idea is** to store the data inside a timestream database and use the timestream-plugin inside grafana to visualize my data. **The question is, if anyone ever used timestream as primary database for this kind of scenario (or are Databases like InfluxDB more suitable). Also whats the easiest way to write the data automatically to my timestreamdb ? ** Greetings Wig

AWS Timestream says that : "To store late arrival data, you simply write the data into Amazon Timestream, and the service will automatically determine whether it gets written to the memory store or to the magnetic store, based on the timestamp of the data and on the configured data retention for the memory and magnetic stores." The maximum memory store time is 12 months but I'm trying to upload data from 2017 to my TimeStream and I'm getting the error message that the record timestamp is outside the time range of the memory store. I thought that it would automatically move it to the magnetic store. I have set : Memory store : 1 hour Magnetic store: 10 years

Hello, Since the clock change on 27th March it seems like Timestream is accepting only inserts that are at least 35 minutes old. Anyone has similar issue? Thanks, Adam

Hello! I was updating the data retention settings of a table within my Timeseries DB, and even though I can see settings have been updated in the console, I can see that the table still operates using the old settings. I know this because if I were to send a new telemetry entry that's older than 6hrs (which was the default `memoryStoreRetentionPeriodInHours`), I would get an error. So, that means the new `memoryStoreRetentionPeriodInHours` hasn't actually been applied - even though the console says so. Hope I've managed to explain my situation. Happy to share more info if needed.

I am testing some calculations in DraftSight that come from TimeStream. I an using an IOT thing to send a timestamp to AWS, a rule to insert the packet into TimeStream. When I send a timestamp in the past it seems to show up in TimeStream. When I send a timestamp from in the future it seems to not show up. I am not sure though if it is latency or if timestream is rejecting the entry because the timestamp is in the future. Does TimeStream reject timestamps from the future?

Hi, I have a TimeStream table containing power consumed by IoT devices. I would like to create a calculated field in QuickSight which computes the sum of last power values reported by all devices?

My scheme looks like this: **Dimensions:** * Department * Organization * SubDepartment * WorkCenter * measure_name **Time:** * time **Measures:** * measure_value::double * measure_value::bigint I keep getting the error: > 'measure_name' is reserved word for Timestream and cannot be used as dimension name. If I change measure_name to a measure then I get: >MultiMeasureAttributeMapping at index 0 with TargetMultiMeasureName 'measure_value::double' field of measure mapping at index 0 contains a reserved prefix for Timestream. If I leave it unassigned I get: > Invalid table schema > One or more columns in the destination table schema have undefined attributes. A Timestream attribute type must be set for each column. This looks like a bug in the saved wizard query to me.

First...is it possible? I am trying to add a column to a table called "MedFlaminator". I just created the table and it has the default layout. Two columns exist: > measure_name varchar MEASURE_NAME > time timestamp TIMESTAMP My query looks like this: > alter table MedFlaminator add column feedback_score int I have tried all kinds of combinations of this query. I am getting the following error: > 10:58:16 AM Failed The query syntax is invalid at line 1:1 alter table MedFlaminator add column feedback_score int What is wrong here?

We have a Kotlin (JVM) lambda that we use to insert records into a TimeStream DB. Both are in the same region. We are using version 2.17.100 or the aws sdk. When busy, it often fails with a EndpointDiscoveryFailedException (based on the file name in the stack trace) exception with the message "Failed when retrieving a required endpoint from AWS." and the message in the exception cause is "unable to create new native thread". The exception is thrown by software.amazon.awssdk.core.endpointdiscovery.EndpointDiscoveryFailedException$BuilderImpl I've searched and found some references to the exception being due to Lambda throttling, although that seems weird as there are instances of it occurring when the request rate is less than one request per second (much lower than I'd expect for lambda throttling). Also, it consistently occurs in Endpoint Discovery while attempting to call TimeStream which to me suggests a TimeStream issue instead of Lambda throttling. I also found reference to a timestream quota for CRUD APIS being one per second per region. That does seem to be a likely cause, the the exception message is weird AND I've seen the system handle more than one request per second. Has anybody seen this and do they have an explanation? Thanks

We have a Kotlin (JVM) lambda that inserts records into TimeStream using software.amazon.awssdk.services.timestreamwrite.TimestreamWriteClient from Version 2.17.100 of the aws sdk. It is hosted in us-east-1 as is the TimeStream DB that it accesses. We sometimes get a RejectedRecordsException exception, here's a sample: "The record timestamp is outside the time range [2022-02-24T18:21:52.301Z, 2022-02-24T23:01:52.301Z) of the memory store. " For the above record, the lambda attempted to insert the record at 2022-02-24T17:26:52.317-05:00 using that value as its timestamp, which is 2022-02-24T22:26:52:317Z and well within the accepted range. The record's timestamp is set to the current time of the lambda inserting the record and the lambda is in the same region as the TimeStream DB - so I don't think there are any potential issues with the time stamp being invalid due to improper time zone handling. The preceding and subsequent requests, i.e. for the above, a request submitted 3 seconds earlier and one submitted 9 seconds later, are all accepted. Can someone provide an explanation for the above error? [Update - Problem solve - it was due to a bug in our code, the timestamp was improper formatted when creating the timestream record]

I'm looking for an AWS solution as close as possible as PI AF by Osisoft. The solutions needs to: * Allow old data ingestion (could be of few years old); * Allow aggregation on data (could be summing two timeseries sensor values); * Allow aggregation on old data and on the upcoming one. I've already done some research and I came out with a couple of solutions, but they still don't satisfy me completly. AWS IoT SiteWise has assets concept and it could possibly be the best choice, but we have found that we couldn't ingest data that is older than 7 days, and it's unacceptable since we'd like to upload all the old data. Amazon Timestream with some SQL queries could do all the aggregation we needed but just to sum two series it's needed a pretty long query, and in case of dozens of series it would require a very long, not much manteinable and hard to write query. Thanks in advance.

Hi there, we have a timestream database in eu-west-1 region. I am trying to add this database to QuickSight to visualize it. All my attempts end up with "No tables found" during data source creation. Quicksight does not see my Timestream tables. Are there any permissions I must set to allow Quicksight listing my timestream tables? Can I find it anywhere in the documentation? All the tutorials "just work" or this part is skipped. Thanks for any advice. David

If I have multi-measure records where most of the measures are empty, does that still affect my storage cost as if they were populated? Example: Will a multi-measure record of 250 empty values and 2 populated values take up storage cost for 252 values or 2 values? Also, if I have single-measure records with different data types, does the unused measure value have a cost to it? Example: "*measure_name='my_bigint', measure_value::double=NULL, measure_value::bigint=5000*", does the NULL take up 8 bytes of storage?

Hi, I am trying to integrate IoT rule with timestream. I managed to handle a case when I have one incoming measurement per messages, but for network optimisation (or something) we receive packed measurements as JSON array (examples below). Is there a way to compose SQL code which will parse incoming array and put multiple measurements into timestream db? Single measurement example: >{ > "value": 335, > "dataId": "Wroclaw1" > "takenAt": "2022-02-01T11:40:46.770Z" >} my case: >[[{ > "value": 35, > "dataId": "Wroclaw1" > "takenAt": "2022-02-01T11:40:46.770Z" >}, >{ > "value": 37, > "dataId": "Wroclaw2" > "takenAt": "2022-02-01T11:41:49.260Z" >}]]

Hello: I'm having a weird behavior in Timestream where my data has a measure name "Latitude" which normally has decimal values like 121.451253 but it was stored as bigint truncating the decimal part. I also tried to send the measure as string in which case it was just ignored (not stored). I noticed that changing the name of the measure or just using lowercase like "latitude", everything is fine... Is there any issue using some specific words? I checked there is no reserved word with that name... Thanks

When I have a bigint value which is actually a duration in milliseconds, is there a better way than this to transform it? ``` parse_duration(cast(my_value as varchar) || 'ms') ```

Hi Team, i would like to know , what are the actual Data which gets feed into AWS TimeStream Database . 2) Is there any possibility that , if business wonot have any real time data to process into TimeStream database , then in that Case , can we Load Postgre SQL Write Ahead Logs Data to the Timestream Database ? my understanding that , write ahead logs are only used for postgre db backup / recovery / incorrect shutdown of DB , let me know and guide me. Regards, Mangesh

I tried to adapt timestream .net sample console application to a blazor wasm web application. Console application runs the query succesfully but blazor app cannot with the same settings. This is my query client config: ``` var queryClientConfig = new AmazonTimestreamQueryConfig { RegionEndpoint = RegionEndpoint.EUWest1, UseAlternateUserAgentHeader=true, }; var queryClient = new AmazonTimestreamQueryClient(<key>,<secret>,queryClientConfig); ``` when I call the query it throws exception "Failed to discover the endpoint for the request. Requests will not succeed until an endpoint can be retrieved or an endpoint is manually specified." ``` QueryResponse queryResponse = await queryClient.QueryAsync(queryRequest); ``` I tried with and without UseAlternateUserAgentHeader. What configuration am I missing?

Hi team, would like to know , Is there a possibility / use case, where Postgre SQL Write Ahead Logs (WAL) (which is mainly used for database backup recovery functionality) can be used / considered as a Event data or Source data and this WAL logs further can be processed , transform through Kinesis data streams and Load into destination db i.e. AWS Time stream database . just try to show in diagram ---- source --- ----Data processing----- Target DB Postgre SQL (WAL Logs) -----> Kinesis Data Streams (with Lambda) -------> Time Stream DB Is this architecture is feasible ? as per my understanding , Time Stream DB stores the device / sensor /web logs data and time stream can be considered as Primary DB always . please add your thoughts Regards, Mangesh

Hello: I want to retrieve my Timestream data from API Gateway, so I need to write query code in lambda. I tried the Timestream SDK in Nodejs, and it worked fine running from the CLI. However, trying to replicate same example in AWS Lambda, it runs without errors, but so far, I can't get any data... I was trying to get Query examples to run from Lambda but I couldn't find... Any help would be appreciated. Regards;

Hello: I'm trying to insert data into Timestream from AWS IoT, where I created a rule as: ``` SELECT * FROM 'dataTopic' ACTIONS: Write a message into a Timestream DB: test, table: sensors, dimension name: device, dimension value: ${device} Republish a message to an AWS IoT topic: test ERROR ACTION: Republish a message to an AWS IoT topic: error ``` And publishing data as: ``` { "device": "abc123", "temperature": "24.50", "humidity": "49" } ``` **works fine.** **NOW, my real data** is actually like this; ``` { "state": { "reported": { "device": "abc123", "temperature": "24.50", "humidity": "49" } } } ``` So I had to modify my Rule as: `SELECT state.reported.* FROM 'dataTopic'` but when I test it, I get an error from Timestream as it seems ``` "failures" : [ { "failedAction" : "TimestreamAction", "failedResource" : "test#sensors", "errorMessage" : "Failed to write records to Timestream. The error received was 'Errors at dimensions.0: [Dimension value can not be empty.]'. Message arrived on dataTopic, Action: timestream, Database: test, Table: sensors" ``` However, **checking the data received at topic test, I don't see differences** with the original data; ``` { "device" : "abc123", "temperature" : "24.50", "humidity" : "49" } ``` What could be the problem? So far, I see same data being ingested but for some reason, Timestream is seeing something different. I tried to use Cloudwatch to see what exactly is Timestream receiving, but I couldn't see the logs from this service. I would appreciate any help. Thanks

Hi, My message payload from my iiot device ( Advantech ECU-1251) will not ingest correctly into my timestream database. The payload looks like this: { "state": { "reported": { "FiveMins": 197, "device_id": 9654006.1, "ModStat": 0, "CJC00": 37.3, "Ch00": 37.3, } } } In my table the measure_name always = "state" and the measure_value::varchar has all the remaing data in it. measure_value::bigint = blank measure_value::double = blank I think its because the ECU-1251 curly bracket structure is wrong. I cannot change the ouput on the ECU-1251. From the docs I know it should look like this: { Measures "temperature": 24.62216675311284, "pressure": 31.381285890765184, "humidity": 1015.2094878981652, ”device_id": "sensor_02", "building": "Day 1", "room": "10.01" } Is there a way to change the "rule" to parse this format correctly so it can ingest correctly into time stream? Thanks! Evguy

Hi, We are a startup from Japan, currently creating our infrastructure architecture. Time series data will be central to all we do, and as we are planning to build as much as possible on serverless, AWS-native services, Timestream would be our preferred choice. MVP to be done by end of 2022, so I would like to know * **are there rollout plans for timestream in ap-northeast-1 at this point?** * **if yes, would it be possible to join a pilot program if you have?** Chatted with sales, but they didn't have an answer either, so here goes my request to the dev team/product owner. If we cannot use Timestream, we might settle on Azure instead, since Azure Time Series Insights is available in one of their Japanese AZs already.

Hi Team, I have an architecture based question, How Postgre SQL DB log processing can be captured through AWS lambda , aws Kinisis Data streams and finally Data should loads into Time Stream Database. Providing High level scenario: Draft Data flow : **Aurora Postgre DB** ----DB Logs Processing to ---->** Lambda** --->Ingestion to ----> **Kinesis Data Streams ** ---Process and Join context data and insert --- Insert to --------> **Time Stream Database** I believe , we can process / loads the AWS IoT (sensors , device data) to Time Stream Database through Lambda , Kinesis Streams , Kinesis Data analytics and finally Time series Database and we can do analytics on time series data . But i am not sure How the postgre SQL db logs (write ahead logs) process through Lambda and ingest through Kinesis streams and finally load into Time Stream Database . and above flow also required to Joins some tables like Event driven tables with associated Account , Customer tables and then it will load into Time Series Database . would like to know if above flow would be accurate , as we are not processing any sensors / devices data ( where sensors data captures all measures and dimensions data from device and loads into Time Stream DB ) so Time Series database always a primary database . if anyone can through some lights , how postgre sql db logs can be integrated with Time Stream database through Kinesis Data streams , Lambda . Need your help Thanks

How to set a S3 location to store timestream asynchonous logs ? I've enable Mggnetic Strorage Writes on Timestream, i need to store historical data. But, i can't set a location to write error log S3. Error logs location - optional Specify an S3 location for error logs because magnetic storage writes are asynchronous. Resource URI I can see my bucket name, choose it, but it's disappear when i came back on table configuration page. I think, i have to allow timestream to push on my S3 Bucket, but i didn't find the policiy or option to do that. Thanks for your help

I have 3 types of devices in the same site (I need them in the same table, cause join between tables is not supported) device A - have 3 measures: temperature, power, voltage device B - have 3 measures: temperature, power, soc device C -have 2 measures: temperature, power I have 2 option - which on is the best? 1) single measure table 2) multi measure table - however soc and/or voltage column will be null in more than half of the records. Will the answer be any different if there was 20 common measures (that exists in all record) and 2 specific measures (that exist in only some of the records)? What about 10 common and 1 specific? Is there any rule of thumb or best practice when to choose multi over single? Is there any cost to an empty column in a multi record table? Thank You!!

Hi, Is it possible to create a single db record with multiple measurements using IoT Greengrass Timestream rule action? I want to show 3 measurements from a device in a single row. Even though my select query has 3 measurement they are all inserted in to table as different rows. My Timestream rule in CF template: ``` TimestreamRule: Type: AWS::IoT::TopicRule Properties: TopicRulePayload: RuleDisabled: false Sql: !Join [ '', [ "SELECT cpu_utilization, memory_utilization, disc_utilization FROM 'device/+/telemetry'", ], ] Actions: - Timestream: DatabaseName: !Ref TelemetryTimestreamDatabase TableName: !GetAtt DeviceTelemetryTimestreamTable.Name Dimensions: - Name: device Value: ${deviceId} RoleArn: !GetAtt SomeRole.Arn Timestamp: Unit: SECONDS Value: ${time} ``` My message payload: ``` { "cpu_utilization": 8, "memory_utilization": 67.4, "disc_utilization": 1.1, "deviceId": "asdasdasd123123123", "time": "1639141461" } ``` Resulting records in Timestream: | device | measure_name | time | measure_value::bigint | measure_value::double| | --- | | 61705b3f6ac7696431ac6b12 | disc_utilization | 2021-12-10 13:03:47.000000000 | - | 1.1 | | 61705b3f6ac7696431ac6b12 | memory_utilization | 2021-12-10 13:03:47.000000000 | - | 67.1 | | 61705b3f6ac7696431ac6b12 | cpu_utilization | 2021-12-10 13:03:47.000000000 | - | 12.1 | This is not what I want. I want to have a single record including all three measurements, cpu, disc and memory. I know it is possible to do it somehow because provided sample db has multi measurement records, such as: | hostname | az | region | measure_name | time | memory_utilization | cpu_utilization | | --- | | host-n2Rxl |eu-north-1a | eu-north-1 | DevOpsMulti-stats | 2021-12-10 13:03:47.000000000 | 40.324917071566546 | 91.85944083569557 | | host-sEUc8 |us-west-2a | us-west-2 | DevOpsMulti-stats | 2021-12-10 13:03:47.000000000 | 59.224512780289224 | 18.09011541205904 | How can I achieve this? Please help! Bests,

I am trying to populate a new database/ new table using the newly announced multimeasure records. But I get error messages when doing so using MULTI mmeasure types, and don't know why. The error I am getting reads: "Error: Parameter validation failed Unknown parameter in Records\[0]: "MeasureValues", must be one of: Dimensions, MeasureName, MeasureValue, MeasureValueType, Time, TimeUnit, Version" so based on this error I am not allowed to pass "MeasureValues", only "MeasureValue"? This seems wrong for MULTI measures, has that not yet been implemented? I am using client.write_records with boto3 according to this documentation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/timestream-write.html#TimestreamWrite.Client.write_records MeasureValues is allowed for MULTI type objects. Using python, I am passing on: CommonAttributes = { "Dimensions": dimensions, "MeasureName": "data", "MeasureValueType": "MULTI", "TimeUnit": "SECONDS" } Records = \[ { "Time": "1628703068", "MeasureValues": \[ { "Name": "1", "Value": "5.57", "Type": "DOUBLE" }, { "Name": "2", "Value": "9.93", "Type": "DOUBLE" } ] } ] So I am using MULTI as MeasureValueType, but am getting a parameter validation error. What am I missing? Edited by: JH1234 on Dec 3, 2021 12:08 AM

We receive JSON-encoded data like **{"data":\\[{"n":1234,"s":"abcd","t":1630543365507,"x":0.5678}\],"type":"sample"}** where "t" is the (UNIX epoch milliseconds) time at which this sample was collected. How can we use that as the Timestream timestamp? Timestream's Product FAQ (https://aws.amazon.com/timestream/faq/?loc=5#Data_ingestion) says it "uses the timestamp of the time series event being written into the database." The Developer Guide (https://docs.aws.amazon.com/timestream/latest/developerguide/concepts.html) says "Timestamp - Indicates when a measure was collected for a given record." The Developer Guide (https://docs.aws.amazon.com/timestream/latest/developerguide/data-ingest.html) says "Data ordered by timestamps has better write performance" What do the ingested data's timestamps matter, if Timestream will apply timestamps corresponding to the time they're written into the database? A Timestream developer forum thread (https://forums.aws.amazon.com/thread.jspa?threadID=329388) refers to an IoT developer forum thread (https://forums.aws.amazon.com/thread.jspa?messageID=959502) in which the IoT feature Timestream Rule Action (https://docs.aws.amazon.com/iot/latest/developerguide/timestream-rule-action.html) was implemented (according to @manbeenaws) to specify an arbitrary value (i.e. a timestamp specified in the MQTT payload) as the Timestream timestamp. Is there a way to use a payload value as the Timestream timestamp, if the data don't arrive via IoT? Edited by: bobsut on Sep 1, 2021 11:44 AM Edited by: bobsut on Sep 1, 2021 12:35 PM

Hi, I have few problems with ingesting data into Timestream DB using IoT rule engine. My measurement data comes in few different data types from many sources with unique identifiers. Therefore, I can't just group the data under the same measure name. To resolve this I planned to use a unique measurement identifier as measure_name, so each series could have its respective data type. However, since those identifiers are dynamic (not just "cpu_temp" or something common like that), I can't just add them in the SELECT clause of SQL rule engine. Also, to add more complexity, the timestamp I want to use is the one that comes in the MQTT message, rather than the one at the time of rule invocation. I have tried, but I can't filter out metadata attributes in SQL rule engine, so all attributes get written to DB. So far I only see one solution, which is to have one measure_name for each possible data type and keep unique identifiers in dimensions. Is there any better solution to this scenario? Would it be possible to use the value from one attribute as the name for the outgoing result of the other attribute in SQL rule engine? From: { SignalName: "UniqueSensorIdentifier", Value: 12345.6789 } To: { UniqueSensorIdentifier: 12345.6789 } Thank you.

We are able to enter records into tables using any time unit. But when querying the table, I can only seem to be able to return it as what I can assume is the ISO format. IE: 2021-03-18 04:00:00.055000000 Is there any way to get a query result with a unix like timestamp instead?

Hi, I have a test table where I want to insert old data (less than 12 months old). Initially I had set the table to 2 hours retention period for in memory storage. I changed this configuration to 12 months in order to ingest the old data but I'm getting a RejectedRecordsException every time I try. In the exception I can see the following error message. {'RecordIndex': 17, 'Reason': 'The record timestamp is outside the time range \[2021-03-08T15:16:58.042Z, 2021-03-08T17:25:22.898Z) of the memory store.'}, which shows the retention period is still using the 2 hours configuration to accept new records. The console shows the new configuration and I've tried updating it several times but it didn't work.

Hello, We are developing an application, that will potentially end up in our production environment in the coming months. We would like to use recently released Timestream DB for managing timeseries data for the application. But we have a problem. Currently it is possible to set up an action via web UI console to push messages from IoT Core Rule to Timestream DB, but it is not possible to do so via Cloud Formation template according to these docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iot-topicrule-action.html Also I am confused by this: https://docs.aws.amazon.com/iot/latest/developerguide/timestream-rule-action.html As it would seem as if it is allowed but after trying it couple weeks ago - it did not work. Could we get this feature - defining IoT TopicRule action for timestream db in Cloud Formation template? If yes, is there any ETA. that you could provide?

I found in the Timestream documentation that "The maximum number of Create/Update/List/Describe/Delete database/table API requests allowed per second per account, in the current region" Does this mean that if I have a Lambda function that writes data points into Timestream. That function can only write 1 datapoint per second? If YES, What if I have hundreds or thousands of devices that need to write data into Timestream ? For sure there will be more than 1 data point write per second. thanks

Hello, I am planning on using Timestream instead of RDS for my website. I have historic data ranging back four months. Under normal operations I would have a memory store retention of maybe 24 hours as there won't be any data arriving later than that. To ingest the historic data I would increase it to a four months, ingest the data and then decrease it again. Is that the appropriate approach? What effect would decreasing the memory store retention have? kind regards Sebastian

Hi, Timestream is great in many ways, but seems to be hard/different to work with if you want to query several different measurements in the same query. This is because of the "flat" data model that Timestream uses. If we use the example dataset found in the documentation at https://docs.aws.amazon.com/timestream/latest/developerguide/sample-queries.devops-scenarios.html How can I write a query that returns the four columns time, hostname, (measure_value::double for) cpu_utilization and (measure_value::double for) memory_utilization? The CREATE_TIME_SERIES function only seems to work with one measure_name? Thanks.

Has anyone seen an error like: Problems with function : ANY. Either the function does not exist, or there is a problem with a dependent function SELECT * FROM "mytable"."temperature" WHERE device_name <> ANY('Foo', 'Bar') Edited by: dwery on Nov 22, 2020 9:50 AM

I wish to ingest data to Timestream with a custom timestamp from IoT. These timestamps will often be in the past -- but well within the retention period. Here are a couple sample messages: { "time": "2020-09-03T17:50:07.790000", "Battery Cranking Voltage": "12.035839999999999" } { "time": "2020-09-03T17:42:28.770000", "Electric Energy Out": "6798.6" } Note the time property is important and related to the measurement. Also note that the measurement changes with each message. AND there are a large and flexible number of measurements. Setting an IoT Rule/Action to send this to Timestream will succeed in creating measure_names and _values, but with the WRONG timestamp. The records are stamped with the INGEST time not the value from the time field. I can "fix" that by changing the NAME of the 'time' property to, say 'timestamp,' and parsing that field with `time_to_epoch`. BUT, that will create extraneous records in Timestream where the measure_name is 'timestamp' and the value is the same as the time. So I'm left with a choice between doubling the size of my database or not having useful timestamps. Is there another way? ---- EDIT ---- To be more detailed, here is another run: example messages published on topic `vt/cvra/teleTester` ` { "timestamp": "2020-10-01 19:50:36.050", "Road Speed": "2.0" } { "timestamp": "2020-10-01 19:50:34.147", "Gear Position": "3.0" } ` IoT Rule SQL: `SELECT * FROM 'vt/cvra/+/cardata'` TImestamp `value` field: `${time_to_epoch(timestamp, "yyyy-MM-dd HH:mm:ss.SSS")}` Units : `MILLISECONDS` Results in tImestream: device_id measure_value::varchar measure_name time ``` teleTester 2020-10-01 19:50:36.050 timestamp 2020-10-01 19:50:36.050000000 teleTester 2.0 Road Speed 2020-10-01 19:50:36.050000000 teleTester 2020-10-01 19:50:35.050 timestamp 2020-10-01 19:50:35.050000000 teleTester 0.0 Road Speed 2020-10-01 19:50:35.050000000 teleTester 3.0 Gear Position 2020-10-01 19:50:34.147000000 teleTester 2020-10-01 19:50:34.147 timestamp 2020-10-01 19:50:34.147000000 ``` See how there are extra rows with the measure of `timestamp`? I'm looking to suppress that.

Hello, We are planning to integrate our IoT data in lambda and then insert it into the timestream. But the problem is that Timestream has an in-memory database option and we are just afraid of it that what if something goes down, like a maintenance shutdown. Will we lose our data? or does AWS make multiple copies of the data stored in memory or something like that?

I'm using boto3/python to try and update a record via 'write_records(...)'. In response I get 'A record already exists with the same time, dimensions, and measure name. Updating measure value is not supported.' There is no boto3 api to delete or update a record. Is there anyway to either delete a record or mutate a record? I'm storing user data and would like to delete user data upon a GDPR request. I get the feeling that a record is immutable by design however the documentation doesn't say it is, at least that I could find. Edited by: Whale on Oct 8, 2020 10:36 PM

Hi, when writing messages into a Timestream table via IoT Core, the timestamp provided in the mqtt message payload is not is not taken over by Timestream. Instead the Timestream date is inserted into the database. Thats annoying since there is normally some delay between the creation time of some IoT data (which should be inserted into the database) and their arrival time at IoT Core / Timestream. Am I doing missing something or does this happen by Timestream design? msg.payload = { somename: somevalue, time = timestamp in ns } Edited by: asseldonk on Oct 8, 2020 1:05 AM

It seems to me that the pricing example for Timestream on <https://aws.amazon.com/timestream/pricing/> is incorrect. The example is for monthly costs but the magnetic storage piece appears to mix a GB/month rate of $0.03 with the number of days in a year 30 * 12? Magnetic store would cost: $108. This is computed as ( 10 ** 1GB ** **30 days** ** **12** **months** ) ** $0.03 GB-month = $108 Shouldn't this simply be 10 ** 1GB ** $0.03 GB-month = $0.30? Edited by: TedOC on Feb 25, 2020 6:05 AM