Questions tagged with Amazon Elastic File System

Content language: English

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

How to access and/or mount Amazon public datasets to EC2

I have an EC2 instance running in us-east-1 that needs to be able to access/manipulate data available in the [KITTI Vision Benchmark public dataset](https://registry.opendata.aws/kitti/). I'd like to make this data available to the instance, but would also like to be able to reuse it with other instances in the future (more like a mounted S3 approach). I understand that I can view the bucket and recursively download the data to a local folder using AWS cli from within the instance: `aws s3 ls --no-sign-request s3://avg-kitti/` `aws s3 sync s3://avg-kitti/` or `aws s3 cp s3://avg-kitti/ . --recursive` However, this feels like a brute force approach and would likely require me to increase my EBS volume size... and would limit my reuse of this data elsewhere (unless I was to snapshot and reuse). I did find some stackoverflow solutions that mentioned some of the open data sets being available as [a snapshot you could copy over and attach as a volume](https://opendata.stackexchange.com/questions/12699/how-can-i-download-aws-public-datasets). But the [KITTI Vision Benchmark public dataset](https://registry.opendata.aws/kitti/) appears to be on S3 so I don't think it would have a snapshot like it would on EBS datasets... That being said, is there an easier way to copy public data over to an existing S3 bucket? and then mount my instance to that? I have played around with S3FS and feel like that might be my best bet, but I am worried about 1) the cost of copying / downloading all data from public bucket to my own 2) best approach for reusing this data on other instances 3) simply not knowing if there's a better/cheaper way to make this data available without downloading or needing to download again in the future.
2
answers
0
votes
70
views
JLux
asked 10 months ago

EC2 - Could not set DHCPv4 address: Connection timed out (sa-east-1a)

Our c6i.2xlarge 3-year reserved instance, running for its first 5 days, generated this log entry **_Could not set DHCPv4 address: Connection timed out_** on Jan 28 02:59:51 UTC, followed by **_Failed_** and **_Configured_**. From there on, the machine became unresponsive and AWS finally raised a StatusCheckFailed_Instance at 06:59 UTC. At 09:06 UTC machine was stopped and restarted through the Console. I found these apparently related issues, but still clueless: [CoreOS goes offline on DHCP failure on Amazon VPC](https://github.com/coreos/bugs/issues/2020) [CoreOS on EC2 losing network connection once a day](https://github.com/coreos/bugs/issues/1551) The box is running MySQL 5.7.36 and Memcache 1.5.6 on top of Ubuntu 18.04. I would be thankful if someone could help me identify the **root cause** of this issue, and: 1. Could this be related to ntp-systemd-netif.service ? 2. This instance type has a separate channel for EBS, but with network down, and no customers making requests (no usage logs on the application machine, except the "MySQL connection timeouts"), what would explain a surge on EBS disk reads? CloudWatch graphs below. 3. We have an EFS disk attached to this instance, that started failing at 04:04 UTC _probably_ related to network failure. No errors reported at EFS sa-east São Paulo status page. Jan 28 02:17:01 ip-172-xxx-xxx-xxx CRON[18179]: pam_unix(cron:session): session opened for user root by (uid=0) Jan 28 02:17:01 ip-172-xxx-xxx-xxx CRON[18180]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jan 28 02:17:01 ip-172-xxx-xxx-xxx CRON[18179]: pam_unix(cron:session): session closed for user root Jan 28 02:29:11 ip-172-xxx-xxx-xxx systemd-networkd[728]: ens5: Configured Jan 28 02:29:11 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Network configuration changed, trying to establish connection. Jan 28 02:29:12 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Synchronized to time server 169.254.169.123:123 (169.254.169.123). Jan 28 02:29:12 ip-172-xxx-xxx-xxx systemd[1]: Started ntp-systemd-netif.service. Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Network configuration changed, trying to establish connection. Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-networkd[728]: **ens5: Could not set DHCPv4 address: Connection timed out** Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-networkd[728]: **ens5: Failed** Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-networkd[728]: **ens5: Configured** Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Synchronized to time server 169.254.169.123:123 (169.254.169.123). Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Network configuration changed, trying to establish connection. Jan 28 02:59:51 ip-172-xxx-xxx-xxx systemd-timesyncd[623]: Synchronized to time server 169.254.169.123:123 (169.254.169.123). Jan 28 03:00:01 ip-172-xxx-xxx-xxx systemd[1]: **Started ntp-systemd-netif.service.** Jan 28 03:01:21 ip-172-xxx-xxx-xxx systemd-udevd[503]: seq 16407 '/kernel/slab/proc_inode_cache/cgroup/proc_inode_cache(4935:ntp-systemd-netif.service)' is taking a long time Jan 28 03:01:28 ip-172-xxx-xxx-xxx systemd-udevd[503]: seq 16408 '/kernel/slab/:A-0000040/cgroup/pde_opener(4935:ntp-systemd-netif.service)' is taking a long time Jan 28 03:01:34 ip-172-xxx-xxx-xxx systemd-udevd[503]: seq 16409 '/kernel/slab/kmalloc-32/cgroup/kmalloc-32(4935:ntp-systemd-netif.service)' is taking a long time Jan 28 03:01:40 ip-172-xxx-xxx-xxx systemd-udevd[503]: seq 16410 '/kernel/slab/kmalloc-4k/cgroup/kmalloc-4k(4935:ntp-systemd-netif.service)' is taking a long time Jan 28 03:17:03 ip-172-xxx-xxx-xxx CRON[18284]: pam_unix(cron:session): session opened for user root by (uid=0) Jan 28 03:17:12 ip-172-xxx-xxx-xxx CRON[18285]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Jan 28 03:19:34 ip-172-xxx-xxx-xxx snapd[6419]: autorefresh.go:530: Cannot prepare auto-refresh change: Post https://api.snapcraft.io/v2/snaps/refresh: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Jan 28 03:19:34 ip-172-xxx-xxx-xxx CRON[18284]: pam_unix(cron:session): session closed for user root Jan 28 03:28:44 ip-172-xxx-xxx-xxx snapd[6419]: stateengine.go:149: state ensure error: Post https://api.snapcraft.io/v2/snaps/refresh: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) Jan 28 03:36:35 ip-172-xxx-xxx-xxx systemd[1]: Starting Ubuntu Advantage Timer for running repeated jobs... Jan 28 04:01:18 ip-172-xxx-xxx-xxx systemd[1]: **Started ntp-systemd-netif.service.** Jan 28 04:03:09 ip-172-xxx-xxx-xxx systemd-udevd[503]: seq 16496 '/radix_tree_node(4961:ntp-systemd-netif.service)' is taking a long time Jan 28 04:04:00 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:06:13 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:06:26 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:09:14 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:09:26 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:12:15 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:12:26 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:12:36 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:15:15 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:15:26 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:15:34 ip-172-xxx-xxx-xxx kernel: nfs: server fs-0ac698ea1xxxxxxxx.efs.sa-east-1.amazonaws.com not responding, timed out Jan 28 04:16:39 ip-172-xxx-xxx-xxx sshd[4657]: pam_unix(sshd:session): session closed for user ubuntu Jan 28 04:17:30 ip-172-xxx-xxx-xxx systemd-logind[974]: Failed to abandon session scope, ignoring: Connection timed out Jan 28 04:18:00 ip-172-xxx-xxx-xxx systemd-logind[974]: Removed session 27. [Cloud Watch Graphs](https://ibb.co/7tydsyQ) Thanks!
1
answers
0
votes
129
views
asked 10 months ago