I do some work with customers transitioning from 7-mode to modern ONTAP (Clustered Data OnTap). When you transition with the excellent 7mtt tool it helpfully transitions snapshots made on the source system to the new system and optionally will duplicate the snapshot schedule on the old system using the new snapshot policies capability. However, the old snapshots from 7-mode are not automatically rolled off. Removal of those old snapshots is left to the administrator. Specifically, the ONTAP manual says:
Step
After the storage cutover operation is completed, delete the unwanted 7-Mode Snapshot copies:
snap delete -vserver <svm_name> -volume <volume_name> -snapshot <snapshot_name>
This is less helpful then it sounds as there may be hundreds of snapshots to
remove. 7mtt doesn't provide any post-transition facilities to help automate
this. So what to do with all of those hourly.1 , weekly.2 and other 7 mode
snapshots? The PowerShell Toolkit offers some useful capabilities to identify
and automate removal of legacy snapshots in a more intelligent way than banging
away in the GUI or console.
First of all, what is the task we'd like to express in code? We'd like to remove legacy 7-mode snapshots and we'd like to manage that by date relative to the current date. For example, we'd like to remove snapshots that transitioned over from the 7-mode source system that are older than 7 days. if they were still on the old system, the would have automatically rolled off based on they 7 mode snap schedule.
First order of business is to examine a snapshot that I know came over from 7-mode and see what sort of attributes we might use to filter and select snapshots matching my criteria.
PS C:\Users\Scott\Scripts> Get-NcSnapshot -Volume summation_d -SnapName hourly.1 |format-list
AccessTime : 1500073256
AccessTimeDT : 7/14/2017 6:00:56 PM
Busy : False
Comment :
CompressionType : none
ContainsLunClones :
CumulativePercentageOfTotalBlocks : 0
CumulativePercentageOfUsedBlocks : 0
CumulativeTotal : 1830633472
CumulativeTotalBlocks : 1787728
Dependency :
ExpiryTime :
ExpiryTimeDT :
InfiniteSnaplockExpiryTime :
InofileVersion : 3
Is7ModeSnapshot : True
IsConstituentSnapshot : False
Name : hourly.1
NcController : 172.27.0.20
PercentageOfTotalBlocks : 0
PercentageOfUsedBlocks : 0
SnaplockExpiryTime :
SnaplockExpiryTimeDT :
SnapmirrorLabel :
SnapshotInstanceUuid : 4adfc2ed-68e8-11e7-8340-123478563412
SnapshotOwnersList :
SnapshotVersionUuid : 4adfc2ed-68e8-11e7-8340-123478563412
State :
Total : 7139328
TotalBlocks : 6972
Volume : summation_d
VolumeProvenanceUuid : a5613fc1-fae7-4dff-af65-84c27712fb58
Vserver : BW_ISCSI_SVM
AccessTimeSpecified : True
BusySpecified : True
ContainsLunClonesSpecified : False
CumulativePercentageOfTotalBlocksSpecified : True
CumulativePercentageOfUsedBlocksSpecified : True
CumulativeTotalSpecified : True
ExpiryTimeSpecified : False
InfiniteSnaplockExpiryTimeSpecified : False
InofileVersionSpecified : True
Is7ModeSnapshotSpecified : True
IsConstituentSnapshotSpecified : True
PercentageOfTotalBlocksSpecified : True
PercentageOfUsedBlocksSpecified : True
SnaplockExpiryTimeSpecified : False
TotalSpecified : True
Cumulative : 1830633472
Created : 7/14/2017 6:00:56 PM
We have some interesting values here we can filter on. For the post-transition
use case, the boolean Is7ModeSnapshotSpecified looks promising. We should
also be able to key on the Created attribute's timestamp value.
Let's take a look at snapshots filtering first on 7-mode and then identifying snaps older than 7 days from volumes in a particular SVM.
PS C:\Users\Scott\Scripts> get-ncvol -vserver BW_ISCSI_SVM | get-ncsnapshot | ?{$_.Is7ModeSnapshot } | ?{ $_.Created -lt (Get-Date).AddDays(-7) }
Name Volume Vserver Created Total Cumulative Dependency
---- ------ ------- ------- ----- ---------- ----------
weekly.3 summation_archive BW_ISCSI_SVM 6/19/2017 12.9 MB 12.9 MB
weekly.2 summation_archive BW_ISCSI_SVM 6/26/2017 18.6 MB 31.5 MB
weekly.1 summation_archive BW_ISCSI_SVM 7/3/2017 11.4 MB 42.8 MB
nightly.6 summation_archive BW_ISCSI_SVM 7/7/2017 9.5 MB 52.3 MB
nightly.5 summation_archive BW_ISCSI_SVM 7/8/2017 9.5 MB 61.8 MB
weekly.3 summation_d BW_ISCSI_SVM 6/19/2017 76.3 MB 76.3 MB
weekly.2 summation_d BW_ISCSI_SVM 6/26/2017 76.6 MB 152.9 MB
weekly.1 summation_d BW_ISCSI_SVM 7/3/2017 74.0 MB 226.9 MB
nightly.6 summation_d BW_ISCSI_SVM 7/7/2017 21.1 MB 248.0 MB
nightly.5 summation_d BW_ISCSI_SVM 7/8/2017 26.9 MB 274.9 MB
weekly.3 summation_data BW_ISCSI_SVM 6/19/2017 16.3 MB 16.3 MB
weekly.2 summation_data BW_ISCSI_SVM 6/26/2017 16.3 MB 32.6 MB
weekly.1 summation_data BW_ISCSI_SVM 7/3/2017 15.2 MB 47.9 MB
nightly.6 summation_data BW_ISCSI_SVM 7/7/2017 12.7 MB 60.6 MB
nightly.5 summation_data BW_ISCSI_SVM 7/8/2017 13.2 MB 73.8 MB
(I snip the output for brevity). This works pretty well. By the way, the "?{" bit of syntax
for the filter is shortand for Where-Object -filterscript { .Deleting all the
snaps older than 7 days within the vserver would look like this
PS C:\Users\Scott\Scripts> get-ncvol -vserver BW_ISCSI_SVM | get-ncsnapshot | ?{$_.Is7ModeSnapshot } | ?{ $_.Created -lt (Get-Date).AddDays(-7) } | Remove-NcSnapshot -whatif
What if: Deleting snapshot weekly.3 from volume summation_archive.
What if: Deleting snapshot weekly.2 from volume summation_archive.
What if: Deleting snapshot weekly.1 from volume summation_archive.
What if: Deleting snapshot nightly.6 from volume summation_archive.
What if: Deleting snapshot nightly.5 from volume summation_archive.
What if: Deleting snapshot weekly.3 from volume summation_d.
What if: Deleting snapshot weekly.2 from volume summation_d.
What if: Deleting snapshot weekly.1 from volume summation_d.
What if: Deleting snapshot nightly.6 from volume summation_d.
What if: Deleting snapshot nightly.5 from volume summation_d.
What if: Deleting snapshot weekly.3 from volume summation_data.
What if: Deleting snapshot weekly.2 from volume summation_data.
What if: Deleting snapshot weekly.1 from volume summation_data.
What if: Deleting snapshot nightly.6 from volume summation_data.
What if: Deleting snapshot nightly.5 from volume summation_data.
I just remove the -WhatIf to proceed. That gets me all the snaps in one
vserver, but I'd like to get them all, I could just specify the above command a
few more times with the vserver names, or I could capture the relevant data
vservers. First I need to take a look at a vserver object to review it's
attributes.
PS C:\Users\Scott\Scripts> Get-NcVserver -Name BW_ISCSI_SVM | format-list
AggrList :
AllowedProtocols : {iscsi, ndmp}
AntivirusOnAccessPolicy : default
CachingPolicy :
Comment :
DisallowedProtocols : {nfs, cifs, fcp}
Ipspace : Default
IsConfigLockedForChanges : False
IsRepositoryVserver : False
IsVserverProtected :
Language : c.utf_8
LdapDomain :
MaxVolumes : unlimited
NameMappingSwitch : {file}
NameServerSwitch : {file}
NcController : 172.27.0.20
NisDomain :
OperationalState : running
OperationalStateStoppedReason :
QosPolicyGroup :
QuotaPolicy : default
RootVolume : BW_ISCSI_SVM_root
RootVolumeAggregate : bwnetapp3_data
RootVolumeSecurityStyle : unix
SnapshotPolicy : default
State : running
Uuid : beb7b977-423d-11e7-8aff-00a098b2985b
VolumeDeleteRetentionHours : 12
VserverAggrInfoList :
VserverName : BW_ISCSI_SVM
VserverSubtype : default
VserverType : data
IsConfigLockedForChangesSpecified : True
IsRepositoryVserverSpecified : True
IsVserverProtectedSpecified : False
MaxVolumesSpecified : True
VolumeDeleteRetentionHoursSpecified : True
Vserver : BW_ISCSI_SVMSo now I can capture all of my data vservers keying on the VserverType attribute.
PS C:\Users\Scott\Scripts> $vservers = Get-NcVserver | ?{$_.vservertype -eq 'data'}
PS C:\Users\Scott\Scripts> $vservers
Vserver State VserverType Comment
------- ----- ----------- -------
BW_CIFS_SVM running data
BW_ISCSI_SVM running data
BW_ISCSI_SVM2 running data
BW_NFS_SVM running dataAnd run my query again providing the cmdlet with my vservers list
PS C:\Users\Scott\Scripts> get-ncvol -vserver $vservers | get-ncsnapshot | ?{$_.Is7ModeSnapshot } | ?{ $_.Created -lt (Get-Date).AddDays(-7) } | Remove-NcSnapshot -whatif
What if: Deleting snapshot weekly.4 from volume admin_logs.
What if: Deleting snapshot weekly.3 from volume admin_logs.
What if: Deleting snapshot weekly.2 from volume admin_logs.
What if: Deleting snapshot weekly.1 from volume admin_logs.
What if: Deleting snapshot nightly.6 from volume admin_logs.
What if: Deleting snapshot nightly.5 from volume admin_logs.
What if: Deleting snapshot nightly.4 from volume admin_logs.
What if: Deleting snapshot weekly.0 from volume admin_logs.
What if: Deleting snapshot weekly.2 from volume software.
What if: Deleting snapshot weekly.1 from volume software.
What if: Deleting snapshot nightly.6 from volume software.
What if: Deleting snapshot nightly.5 from volume software.
What if: Deleting snapshot nightly.4 from volume software.
What if: Deleting snapshot weekly.0 from volume software.
This little bit of powershell will save tons of time. There are other use cases
for this as well. If I need to alter snapshot policies for several volumes, I
may need to go back and remove snapshots created by the previous policy. I can
filter using string matching on the snapshot names. I can exclude volumes
either by first putting together a list of relevant volumes and looping over
them or if it's a single volume by adding a -notlike 'volname` filter to skip
that volume.
After doing a 7mtt transition, snapshot policies and related schedules are
created with a transition prefix in the name. 7mtt does a good job
consolidating those policies but you might still want to simplify, rationalize
and remove those transition policies and eventually use the above methods to
expire their snapshots.
First we need to get a list of snapshot policies matching the wildcard
PS C:\Users\Scott\Scripts> $transitionpolicies = Get-NcSnapshotPolicy | ?{$_.policy-like "transition*"}
PS C:\Users\Scott\Scripts> $transitionpolicies
Policy Enabled SnapshotPolicySchedules
------ ------- -----------------------
transition_snapsh... True {transition_snapshot_schedule_0 (5), transition_snapshot_schedule_1 (7), transition_sna...
transition_snapsh... True {transition_snapshot_schedule_3 (104), transition_snapshot_schedule_4 (7), transition_s...
transition_snapsh... True {transition_snapshot_schedule_6 (5), transition_snapshot_schedule_7 (7), transition_sna...
transition_snapsh... True {transition_snapshot_schedule_9 (4), transition_snapshot_schedule_10 (7), transition_sn...
transition_snapsh... True {transition_snapshot_schedule_12 (4), transition_snapshot_schedule_13 (7), transition_s...
transition_snapsh... True {transition_snapshot_schedule_15 (6), transition_snapshot_schedule_16 (7), transition_s...
transition_snapsh... True {transition_snapshot_schedule_18 (4), transition_snapshot_schedule_19 (7), transition_s...
Then grab a list of volumes that have those policies. assigned. Get-NcVol
with a -query is a more advanced technique.
PS C:\Users\Scott\Scripts> $transitionsnappolvols = @()
PS C:\Users\Scott\Scripts> foreach ($policy in $transitionpolicies) {
>> $mytempvol = get-ncvol -query @{volumesnapshotattributes = @{snapshotpolicy = $policy.policy}}
>> $transitionsnappolvols += $mytempvol
>> }
>>
PS C:\Users\Scott\Scripts> $transitionsnappolvols
Name State TotalSize Used Available Dedupe Aggregate Vserver
---- ----- --------- ---- --------- ------ --------- -------
admin_logs online 200.0 GB 12% 175.8 GB True bwnetapp4_data BW_CIFS_SVM
wsdocs online 15.0 TB 81% 2.8 TB True bwnetapp4_data BW_CIFS_SVM
summation_d online 975.0 GB 72% 266.0 GB True bwnetapp4_data BW_ISCSI_SVM1
summation_data online 500.0 GB 49% 252.5 GB True bwnetapp4_data BW_ISCSI_SVM1
brewssql_db online 1.9 TB 73% 511.7 GB True bwnetapp4_data BW_ISCSI_SVM2
brewssql_logs online 2.2 TB 77% 510.2 GB True bwnetapp4_data BW_ISCSI_SVM2
summationsql_backup_lun online 4.4 TB 51% 2.1 TB True bwnetapp4_data BW_ISCSI_SVM1
summation_archive online 275.0 GB 72% 76.9 GB True bwnetapp4_data BW_ISCSI_SVM1
summation_pro_lun online 2.8 TB 75% 707.8 GB True bwnetapp4_data BW_ISCSI_SVM1
summationsql_d online 750.0 GB 70% 219.5 GB True bwnetapp4_data BW_ISCSI_SVM1
We go ahead and capture the list of matching volumes so we can proceed with
other actions. As an example we could alter their snapshot policies. Here we
need to use Update-NcVol with templates to populate the objects and attributes.
PS C:\Users\Scott\Scripts> foreach ($vol in $transitionsnappolvols) {
>> $VOLattrs = Get-NcVol -Template
>> $VOLquery = Get-NcVol -Template
>> Initialize-NcObjectProperty -Object $VOLattrs -Name VolumeSnapshotAttributes
>> Initialize-NcObjectProperty -Object $VOLquery -Name VolumeIdAttributes
>> $VOLquery.VolumeIdAttributes.Name = $vol.Name
>> $VOLattrs.VolumeSnapshotAttributes.SnapshotPolicy = 'arlaw_default'
>> update-ncvol -whatif -query $volquery -attributes $volattrs }
>>
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
What if: Modifying one or more volumes.
Later on we can go back and use that list of volumes we captured to clean up
transition* snapshots more than X days old or just use the above method to
first filter on a wildcard snapshot name
with "?{$_.Name -like "transition*"}" and then filter again
applying the date math to the Created attribute.
This post was inspired by this community forum response from Andrew Sullivan as well as the NetApp PowerShell 101 series on his blog.