By using AWS re:Post, you agree to the Terms of Use

SiteWise avro file schema


I understand from this document all about how files in the cold tier are stored as S3 objects. What I think is missing is the actual avro schema file needed to load these files into something like a local Jupyter notebook. This schema file should look like this:

   "type" : "record",
   "namespace" : "Tutorialspoint",
   "name" : "Employee",
   "fields" : [
      { "name" : "Name" , "type" : "string" },
      { "name" : "Age" , "type" : "int" }

Is this schema the same for every SiteWise installation or does it depend on my model? Is there a way to export it from my model or asset? I just want to be able to peek through these files with my own software.

asked 7 months ago30 views
1 Answer

SiteWise exports the Avro Schema into the header of the exported .avro file, so no need to load a separate Avro schema when reading the file. Here a sample python script to read a SiteWise avro export and dumps the schema:

>>> from import DatumReader
>>> from avro.datafile import DataFileReader
>>> avro_reader = DataFileReader(open("/tmp/raw_a50553ae-184e-4cea-ba5a-847e6d76e4ac_c82738f4-d145-4b12-8b6a-
>>> avro_reader.meta
{'avro.schema': b'{"type":"record","name":"RawDatum","namespace":"","fields":[{"name":"seriesId","type":"string"},{"name":"timeInSeconds","type":"long"},{"name":"offsetInNanos","type":"long"},{"name":"quality","type":"string"},{"name":"doubleValue","type":["null","double"],"default":null},{"name":"stringValue","type":["null","string"],"default":null},{"name":"integerValue","type":["null","int"],"default":null},{"name":"booleanValue","type":["null","boolean"],"default":null},{"name":"jsonValue","type":["null","string"],"default":null},{"name":"recordVersion","type":["null","long"],"default":null}]}', 'avro.codec': b'snappy'}
profile picture
answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions