wiki.getshifting.com

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools

Trace: • Remove MetroCluster SyncMirror
netappmcremove

Table of Contents

Remove MetroCluster SyncMirror

Summary: How to remove a syncmirror relationship for a NetApp metrocluster.
Date: Around 2015
Refactor: 7 March 2025: Checked links and formatting.

To free up disk space we had to remove a syncmirror relationship for a NetApp metrocluster. The metrocluster functionality was not used, only normal HA functionality. So these are the steps we took, and it was actually quite easy.

Summary Of Commands

These are the commands that were needed to get to our end goal:

  • aggr offline plex2
  • aggr destroy plex 2
  • disk zero spares
    • monitor: vol status -s
  • aggr options aggr0 raidtype raid_dp
    • monitor: aggr status aggr0 -r

Additionally we wanted to move the disks to one of the filers instead of keeping them on their originating filer:

  • disk remove_ownership
  • disk assign
  • aggr options aggr0 raidsize 20
  • aggr add aggr0 -d

And finally you'll have to remove the syncmirror license:

  • license delete cf_remote

Full Log of Commands on Destination Filer

The commands to destroy the plex and zero the disks: 

filer02> aggr destroy /aggr0/plex2
Are you sure you want to destroy this plex? y
Plex 'aggr0/plex2' destroyed.
filer02> aggr status -r aggr0
Aggregate aggr0 (online, raid4) (block checksums)
  Plex /aggr0/plex0 (online, normal, active, pool0)
    RAID group /aggr0/plex0/rg0 (normal, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      parity    0b.19   0b    1   3   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.18   0b    1   2   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.17   0b    1   1   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.16   0b    1   0   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.22   0b    1   6   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.21   0b    1   5   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.20   0b    1   4   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
filer02> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0d.16   0d    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.17   0d    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.18   0d    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.19   0d    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.20   0d    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.21   0d    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.22   0d    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (not zeroed)
spare           0d.23   0d    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0b.23   0b    1   7   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
spare           2d.26   2d    1   10  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
spare           2d.59   2d    3   11  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
filer02> disk zero spares
filer02> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0d.16   0d    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.17   0d    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.18   0d    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.19   0d    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.20   0d    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.21   0d    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.22   0d    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296 (zeroing, 1% done)
spare           0d.23   0d    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Wait for the zeroing to complete and change the raid type of aggr0: 

filer02> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0d.16   0d    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.17   0d    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.18   0d    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.19   0d    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.20   0d    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.21   0d    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.22   0d    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.23   0d    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0b.23   0b    1   7   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
spare           2d.26   2d    1   10  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
spare           2d.59   2d    3   11  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
filer02> aggr options aggr0 raidtype raid_dp
Aggregate aggr0: raidsize is adjusted from 8 to 16 after changing raidtype.
filer02> aggr status -r aggr0
Aggregate aggr0 (online, raid_dp, reconstruct) (block checksums)
  Plex /aggr0/plex0 (online, normal, active, pool0)
    RAID group /aggr0/plex0/rg0 (reconstruction 0% completed, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0b.23   0b    1   7   FC:A   0  FCAL 15000 272000/557056000  274845/562884296 (reconstruction 0% completed)
      parity    0b.19   0b    1   3   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.18   0b    1   2   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.17   0b    1   1   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.16   0b    1   0   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.22   0b    1   6   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.21   0b    1   5   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.20   0b    1   4   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
filer02>

Wait of the reconstruction te complete, and assign the non-needed spare disks from filer01 to filer02, keeping two spares on filer01: 

filer01*> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0b.16   0b    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.17   0b    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.18   0b    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.19   0b    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.20   0b    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.21   0b    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.22   0b    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.23   0b    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks (empty)
filer01*> disk show -n
disk show: No disks match option -n.
filer01*> disk remove_ownership 0b.16 0b.17 0b.18 0b.19 0b.20 0b.21
Disk 0b.16 will have its ownership removed
Disk 0b.17 will have its ownership removed
Disk 0b.18 will have its ownership removed
Disk 0b.19 will have its ownership removed
Disk 0b.20 will have its ownership removed
Disk 0b.21 will have its ownership removed
Volumes must be taken offline. Are all impacted volumes offline(y/n)?? y
filer01*> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0b.22   0b    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.23   0b    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks (empty)
filer01*> disk assign 0b.16 0b.17 0b.18 0b.19 0b.20 0b.21 -o filer02
filer01*> priv set
filer01> disk show -n
disk show: No disks match option -n.
filer01> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0b.22   0b    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0b.23   0b    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks (empty)

On filer02, check raidsize and spare disks before adding disks, keep NetApp - Designing your disks, raids and aggregates in mind: 

filer02> vol status -s

Pool1 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0d.16   0d    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.17   0d    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.18   0d    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.19   0d    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.20   0d    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.21   0d    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.22   0d    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
spare           0d.23   0d    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296

Pool0 spare disks

RAID Disk       Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
---------       ------  ------------- ---- ---- ---- ----- --------------    --------------
Spare disks for block checksum
spare           0c.16   0c    1   0   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           0c.17   0c    1   1   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           0c.18   0c    1   2   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           0c.19   0c    1   3   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           0c.20   0c    1   4   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           0c.21   0c    1   5   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
spare           2d.26   2d    1   10  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
spare           2d.59   2d    3   11  FC:B   0   ATA  7200 847555/1735794176 847827/1736350304
filer02> aggr status -r aggr0
Aggregate aggr0 (online, raid_dp) (block checksums)
  Plex /aggr0/plex0 (online, normal, active, pool0)
    RAID group /aggr0/plex0/rg0 (normal, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0b.23   0b    1   7   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      parity    0b.19   0b    1   3   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.18   0b    1   2   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.17   0b    1   1   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.16   0b    1   0   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.22   0b    1   6   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.21   0b    1   5   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.20   0b    1   4   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
filer02> aggr options aggr0
root, diskroot, nosnap=off, raidtype=raid_dp, raidsize=16,
ignore_inconsistent=off, snapmirrored=off, resyncsnaptime=60,
fs_size_fixed=off, snapshot_autodelete=on, lost_write_protect=on,
ha_policy=cfo, hybrid_enabled=off, percent_snapshot_space=5%,
free_space_realloc=off

So, current settings:

  • raidsize = 16
  • Current disks in raidgroup = 8
  • Available spare disks = 12 (keep 2 spares)
  • raidsize should be = 8+12

Set raidsize to 20 and assign disks to aggr0: 

filer02> aggr options aggr0 raidsize 20
filer02> aggr options aggr0
root, diskroot, nosnap=off, raidtype=raid_dp, raidsize=20,
ignore_inconsistent=off, snapmirrored=off, resyncsnaptime=60,
fs_size_fixed=off, snapshot_autodelete=on, lost_write_protect=on,
ha_policy=cfo, hybrid_enabled=off, percent_snapshot_space=5%,
free_space_realloc=off

filer02> aggr add aggr0 -d 0d.16 0d.17 0d.18 0d.19 0d.20 0d.21 0d.22 0d.23 0c.16 0c.17 0c.18 0c.19
aggr add: Disk '0d.16' is from the wrong pool. Use the -f option to override this check.
filer02> aggr add aggr0 -f -d 0d.16 0d.17 0d.18 0d.19 0d.20 0d.21 0d.22 0d.23 0c.16 0c.17 0c.18 0c.19
Addition of 12 disks to the aggregate has completed.

filer02> aggr status -r aggr0
Aggregate aggr0 (online, raid_dp) (block checksums)
  Plex /aggr0/plex0 (online, normal, active, pool0)
    RAID group /aggr0/plex0/rg0 (normal, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   0b.23   0b    1   7   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      parity    0b.19   0b    1   3   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.18   0b    1   2   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.17   0b    1   1   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.16   0b    1   0   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.22   0b    1   6   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.21   0b    1   5   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0b.20   0b    1   4   FC:A   0  FCAL 15000 272000/557056000  274845/562884296
      data      0d.16   0d    1   0   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.17   0d    1   1   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.18   0d    1   2   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.19   0d    1   3   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.20   0d    1   4   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.21   0d    1   5   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.22   0d    1   6   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0d.23   0d    1   7   FC:A   1  FCAL 15000 272000/557056000  274845/562884296
      data      0c.16   0c    1   0   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
      data      0c.17   0c    1   1   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
      data      0c.18   0c    1   2   FC:B   0  FCAL 15000 272000/557056000  274845/562884296
      data      0c.19   0c    1   3   FC:B   0  FCAL 15000 272000/557056000  274845/562884296

You're done now, in case you encounter performance issues take a look at NetApp Reallocate.

Now create a backup of your licenses and remove the cf_remote license to prevent this message in your logging: 

Message: Root volume is not mirrored. A takeover of this filer may not be possible in case of a disaster.

Delete the license: 

license delete cf_remote
netappmcremove.txt · Last modified: by 127.0.0.1