Avoiding Datastore Rebuilds


The A2 must rebuild its storage cache after certain types of events. The most common root cause is a storage intermittency. After a serious intermittency, the A2 takes the precaution of regenerating deduplication reference counts to avoid corruption.


This can also happen if your A2 loses power suddenly in the middle of certain backup operations. Lastly, if you reboot the A2 during an operation, it may not have enough time to correctly sync up its transactions before the reboot occurs.


Once a rebuild is started, there's no way to speed it up. Rebuild times impacted by

  • Block size. Larger A2 block sizes will vastly cut rebuild time.
  • Fast local A2 disk. Fast (SSD) disk can speed rebuild times x10 or more.
  • Data store size. Larger data stores will take longer to rebuild, all other factors being equal


To avoid rebuilds, do not reboot your A2 or cold-boot it. Instead, take the following steps if you want to be sure to avoid a rebuild evenIn terms of preventing rebuilds, this is also pretty simple. Follow these procedures before installing updates or powering off/rebooting.

  1. Drop to the shell (option 4)
  2. use tail /var/log/engine.log to check for recent engine.log entries. If there are entries newer than 10 minutes, wait and check again later.
  3. Safely stop services using /usr/local/sbin/stopAlike and waiting for this script to complete.
  4. At this point, verify clean shutdown in the engine.log. If there's an issue, please capture this log at this time.
  5. Restart the box or install updates or power off, as desired.


For more information on performance, please see:

https://docs.quadricsoftware.com/kb/troubleshooting/understanding-backup-performance/

https://docs.quadricsoftware.com/kb/guides/understanding-block-size/