Why do my FSx for ONTAP volume snapshots consume free space and exceed the snapshot reserve?

Short description

Snapshot reserve is a percentage of disk space that's allocated exclusively for snapshot storage. If you exceed the reserve, then snapshot spill occurs. Snapshot spill means that your backups exceed the allocated space and spill into the active file system.

Example

The following NetApp ONTAP CLI command output shows the snapshot reserve space of 5% is exceeded. The snapshot capacity is 292% and there is snapshot spill of twice the snapshot reserve (10%). In the following example, the volume name is spillvol.

FsxIdxxxxxxxxxxxxx::> df -h fullvol
Filesystem                total   used    avail capacity  Mounted on         Vserver
/vol/spillvol/            28MB    3740KB  24MB  12%       /spillvoll           fsx
/vol/spillvol/.snapshot   1536KB  4484KB  0B    292%      /spillvol/.snapshot  fsx
2 entries were displayed.


FsxId0523592630bb27ba9::> volume show-space spillvol
Vserver : fsx
      Volume  : spillvol

      Feature                                    Used      Used%
      --------------------------------     ----------     ------
      User Data                                  44KB         0%
      Filesystem Metadata                       528KB         2%
      Inodes                                     52KB         0%
      Snapshot Reserve                         1.50MB         5%
      Deduplication                              76KB         0%
      Snapshot Spill                           2.88MB        10%
      Performance Metadata                       92KB         0%

      Total Used                               5.15MB        17%

      Total Physical Used                      5.15MB        17%

The snapshot spill in the preceding example is caused by the following issues:

• The snapshot policy created snapshots too frequently. When snapshots create too frequently, the WAFL scan can't complete and reclaim the blocks.
• You deleted data from the active file system user data area. However, the old snapshot backups that contain that data still point to those blocks. So, the space from the deleted data isn't free.

Resolution

Method 1: Turn off the volume's snapshot policy

Amazon FSx console

1. Open the Amazon FSx console.
2. Choose File systems, and then select the file system that you want to update a volume for.
3. Choose the Volumes tab.
4. Select the volume that you want to update.
5. For Actions, choose Update volume. The Update volume dialog box displays with the volume's current settings.
6. Change the snapshot policy from default to none, and then choose Update.

NetApp ONTAP CLI

Run the volume modify NetApp CLI command to change the snapshot policy to none. When you run the command at the cluster level, you must specify the vserver (SVM) name that the volume belongs to:

FsxId0523592630bb27ba9::> volume modify -vserver fsx -volume spillvol -snapshot-policy none

Warning: You are changing the Snapshot policy on volume "spillvol" to "none". Snapshot copies on this volume that do not match any of the prefixes of the new Snapshot policy will not be deleted. However, when the new Snapshot policy takes effect, depending on the new retention
count, any existing Snapshot copies that continue to use the same prefixes might be deleted. See the 'volume modify' man page for more information.
Do you want to continue? {y|n}: y
Volume modify successful on volume fullvol of Vserver fsx.

To verify that the snapshot policy changed, run the following command:

FsxId0523592630bb27ba9::> vol show -vserver fsx -volume spillvol -fields snapshot-policy
vserver volume snapshot-policy
------- ------- ---------------
fsx     spillvol none

Method 2: Turn on the snapshot autodelete policy to delete older snapshots

When you use the snapshot autodelete option, make sure that you set the Delete Order and Target Free Space correctly so that you don't delete snapshots that you need.

If you turn on the snapshot autodelete option, then the snapshot autodelete policy deletes older snapshots (or new snapshots, depending on the delete order). The snapshot autodelete option affects the configured snapshot retention period on the data volume. Snapshot autodelete monitors capacity in the snapshot reserve space and overrides the set retention period if the reserved space crosses the capacity threshold.

Note: You can't set automatic deletion of snapshot copies on read-only volumes. For example, you can't use snapshot autodelete in SnapMirror destination volumes.

For more information about snapshot autodelete, see How to use snapshot autodelete on the NetApp Knowledge Base.

To turn on the snapshot autodelete policy, run the following command. You can set snapshot deletion based on volume capacity or on the snapshot reserve capacity percentage.

FsxId0523592630bb27ba9::> volume snapshot autodelete modify -vserver fsx -volume fullvol -enabled true -trigger snaps_reserve

To verify that the snapshot autodelete policy is turned on, run the following command:

FsxId0523592630bb27ba9::> volume snapshot autodelete show -vserver fsx -volume fullvol

         Vserver Name: fsx
          Volume Name: fullvol
              Enabled: true
           Commitment: try
         Defer Delete: user_created
         Delete Order: oldest_first
  Defer Delete Prefix: (not specified)
    Target Free Space: 20%
              Trigger: snap_reserve
         Destroy List: none
Is Constituent Volume: false