Unlocking XRP Ledger Data: Comprehensive Analysis with AWS Public Blockchain Datasets

The AWS Public Blockchain Datasets now include data from the XRP Ledger (XRPL), provided by SonarX. This addition enables access to XRP blockchain data through Amazon S3 and analysis with AWS Glue and Amazon Athena, opening up new possibilities for research and analytics in the XRP ecosystem.

Background

In 2022, AWS launched the Public Blockchain Datasets to provide researchers and developers with free access to blockchain data. The inclusion of XRP data expands the scope of blockchain analysis possibilities on AWS. SonarX, an AWS Partner specializing in blockchain data indexing, ensures that the XRP dataset is regularly updated and maintains high quality standards.

Dataset Access and Organization

The XRP blockchain dataset is publicly available in the s3://aws-public-blockchain bucket. The data is organized hierarchically by date, making it simple to query specific time ranges. The XRP data follows a consistent schema, with common fields like transaction hash, sender, receiver, and timestamp.

The public Amazon S3 URL for the XRP dataset is:

s3://aws-public-blockchain/v1.1/sonarx/xrp/

Use Cases

Customers can use the XRP dataset in various ways:

• Research and Analytics: Academic institutions analyze XRP network behavior patterns and conduct comparative studies with other blockchains.
• Business Intelligence: Companies monitor XRP network activity and user behavior to inform strategic decisions.
• Risk and Compliance: Financial institutions use XRP data for transaction monitoring, fraud detection, and auditing purposes.
• DeFi and Trading: Trading firms analyze market dynamics specific to XRP.

Getting Started

Let's walk through setting up and querying the XRP dataset using AWS services.

Create a Glue Crawler

You will create a Glue Crawler to catalog our XRP data. Use the following AWS CLI commands.

Note: You can run the following commands from your AWS CloudShell terminal.

Create the necessary IAM role:

aws iam create-role \
    --role-name AWSGlueServiceRole-XRP \
    --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Service": "glue.amazonaws.com"
                },
                "Action": "sts:AssumeRole"
            }
        ]
    }' > /dev/null

Create the policy document for S3 access:

aws iam put-role-policy \
    --role-name AWSGlueServiceRole-XRP \
    --policy-name S3Access \
    --policy-document '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject",
                    "s3:ListBucket"
                ],
                "Resource": [
                    "arn:aws:s3:::aws-public-blockchain",
                    "arn:aws:s3:::aws-public-blockchain/*"
                ]
            }
        ]
    }'

Attach the AWS managed policy for Glue Service:

aws iam attach-role-policy \
    --role-name AWSGlueServiceRole-XRP \
    --policy-arn arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole

Create the Glue Crawler for XRP, with a scheduled run:

aws glue create-crawler \
 --name xrp-blockchain-crawler \
 --role AWSGlueServiceRole-XRP \
 --database-name sonarx_xrp \
 --targets '{"S3Targets": [{"Path": "s3://aws-public-blockchain/v1.1/sonarx/xrp/", "SampleSize": 2}]}' \
--schedule "cron(0 5 * * ? *)" 

Run the Glue Crawler:

aws glue start-crawler --name xrp-blockchain-crawler

Monitor the crawler's progress:

The first time you start the crawler, it might take anywhere between five to ten minutes to fetch files from the datasets, and assemble a fitting data schema. You can monitor the first run of the AWS Glue Crawler from the Console, or by running the following command:

aws glue get-crawler --name xrp-blockchain-crawler --query '[Crawler][0].State'

Query the XRP Data with Amazon Athena

Once the crawler has run, you can query the XRP data using Amazon Athena. Here's a query to get daily transaction counts for XRP:

SELECT 
    date,
    COUNT(*) as tx_count
FROM "sonarx_xrp"."transactions"
WHERE date >= date_format(date_add('day', -30, current_date), '%Y-%m-%d')
GROUP BY date
ORDER BY date DESC;

Conclusion

The inclusion of XRP data in the AWS Public Blockchain Datasets empowers researchers, developers, and businesses to perform in-depth analysis of the XRP Ledger. With access to this valuable data and the power of AWS analytics tools, you can uncover insights, track network activity, and drive innovation in the XRP ecosystem.

We encourage you to explore the XRP dataset and share your findings or applications in the comments below.