Using S3 File Gateway as NFS to EC2 server with SAP BW

Amazon S3 File Gateway is a suitable solution for your SAP BW use case, as it allows your EC2 instance running SAP BW to write CSV files to an NFS mount point, which then stores the data in Amazon S3.

Network and Connection Best Practices: When configuring your S3 File Gateway for SAP BW on EC2, you should:

• Configure security groups on your SAP Content Server EC2 instance to allow only the source SAP application servers to communicate with the SAP Content Server via the required TCP/IP ports
• Deploy the S3 File Gateway as a virtual machine in your AWS environment (you can use an EC2 instance)
• Mount the S3 File Gateway as a filesystem mount point (e.g., "/content_server") on your SAP BW EC2 instance

S3 Synchronization and Performance: Files written to the S3 File Gateway are not immediately available in S3. The gateway uses local cache storage to provide low-latency access to recently accessed data and to buffer data pending upload to S3. The gateway manages data transfer to AWS, optimizes and streams data in parallel, and manages bandwidth consumption. The actual latency and throughput depend on factors like file size, network conditions, and cache configuration.

Configuring S3 File Gateway with S3 Standard: Setting up S3 File Gateway is straightforward using the launch wizard for Storage Gateway:

1. Deploy the gateway using an EC2 instance in your AWS environment
2. Configure Storage Gateway shares to be accessible through the NFS protocol
3. During setup, you'll specify which S3 bucket should store your files
4. Files and folders will be stored as objects in your designated S3 bucket

Moving Files Between S3 Buckets: After files arrive in your S3 bucket from the File Gateway, you can:

• Use S3 event notifications to trigger AWS Lambda functions to move the files
• Implement S3 lifecycle policies to automatically transition objects
• Use AWS Step Functions or other AWS services to orchestrate the movement of files based on your specific requirements

Cost Considerations: The costs for this solution include:

• Storage Gateway service fees
• S3 storage costs for your data
• Data transfer costs
• EC2 instance costs if you're running the gateway on EC2
• Cache storage costs

S3 File Gateway offers cost advantages compared to maintaining additional database instances or using other storage solutions, particularly for workloads like yours that involve storing and processing large numbers of files.
Sources
Sustainable and durable SAP document and data archiving with Amazon S3 | AWS for SAP
What is Amazon S3 File Gateway - AWS Storage Gateway
AWS Storage Gateway - AWS Prescriptive Guidance
Storage Gateway - AWS Prescriptive Guidance

Hi Felipe,

Great use case — and yes, Amazon S3 File Gateway is a good fit for integrating SAP BW output into your S3-based processing pipeline. I’ll walk through your questions step by step with best practices and considerations:

1. Use Case Feasibility & Best Practices Yes, this architecture is valid:

SAP BW EC2 ➜ writes CSVs to S3 File Gateway (NFS) ➜ data lands in S3 Standard ➜ triggers EKS/Lambda/EventBridge.

To optimize this:

Deploy the S3 File Gateway as an EC2 instance in the same VPC and subnet as your SAP BW server.

Use NFS v4 to mount the gateway’s share (e.g., /sap_data).

Ensure your EC2 SAP instance and S3 File Gateway have appropriate IAM roles and security groups (allow NFS TCP traffic).

2. Sync to S3 & Latency File upload is not instant — once SAP writes to the mount point, the File Gateway uploads in the background.

Latency: Typically a few seconds to minutes, depending on:

File size

Cache size and utilization

Network throughput to S3

For large files (like 30GB CSVs), consider using multipart uploads or parallelization in your SAP logic if possible.

3. How to Configure Gateway to Write to S3 Standard In Storage Gateway setup:

Choose File Gateway

Set bucket destination (it defaults to Standard class unless configured otherwise)

Choose NFS as the protocol

Assign an IAM role that grants s3:PutObject, GetObject, and ListBucket on the destination bucket

4. Moving Files to Another S3 Bucket Once the files land in your primary bucket, you have multiple options to move or copy them:

S3 Event Notifications → trigger a Lambda function

Use Amazon EventBridge with object-level events (if supported)

S3 Batch Operations or Step Functions for periodic processing

S3 Lifecycle Rules to transition/move objects

5. Cost Considerations Here's what to expect:

Component Cost Notes S3 File Gateway $0.01/GB/month of data written S3 Standard $0.023/GB/month (storage) EC2 (for Gateway host) Based on instance type, EBS usage Data Transfer No cost for upload to S3 in same Region Lambda/EventBridge/EKS Pay-per-use (minimal unless scaled heavily)

Tip: Consider S3 Intelligent-Tiering if files are infrequently accessed after processing.

Summary ✔️ Yes, your design is sound ✔️ Deploy File Gateway in same subnet as SAP EC2 ✔️ Mount NFS, configure correct IAM + security groups ✔️ Use S3 Events or lifecycle for downstream steps ✔️ Expect some delay due to File Gateway’s background uploads ✔️ Costs are predictable and manageable for your scale

Let me know if you want example mount commands or Lambda starter code!

— (Sharing this to help others connecting SAP BW to S3 using File Gateway for batch processing and analytics pipelines.)

Hi Felipe,

While the solution using S3 FGW sounds like it might work, I'm curious if you have considered using an FSx offering, which would provide additional features from a file system perspective such as snapshots, better performance for small file workloads, high availability, etc.

If your data needs to land in S3, for any specific reason, due to further processing downstream, or other apps that need to access via S3 directly, then S3 FGW could be a fit. But note there is no HA for EC2 based Storage Gateways, and it does have some limitations surrounding small file workloads (<5 MB files) especially when there are millions of small files, and it does write asynchronously.

My suggestion would be to work with your account team or through support case to take a deeper and more holistic look at the workload and the business requirements with a specialist, prior to moving forward. I want to make sure you are aware of all your options, potential drawbacks, and other limitations of the solution.

The blog you referenced, was particularly for a Database Backup use case, where S3 FGW is a target for backup data.

I would suggest taking a look at FSxZ INT as well, as that has HA, NFS, and a very comparable price point to S3. It also removes the complexity of needing to maintain an EC2 instance to run the S3 FGW.

More details on the workload requirements would be great to see, such as amount of data, performance requirements, data protection requirements, availability requirements, etc.

Regards, Ed