Data Retention and Protection

Primary Storage (Tier 1)

MSI takes several precautions to prevent the loss of data stored on MSI's Primary Storage systems. These precautions protect against the majority of data storage system failures; however, they may not protect your data from a catastrophic failure of the file system or from damages sustained to the MSI data center. While catastrophic events are unlikely, users are nonetheless encouraged to take precautions to back up and archive data that are difficult or impossible to regenerate.  

Primary Storage Snapshots & Exclusions (Tier-1 Snapshots)

Snapshots allow both MSI staff and end users to recover lost, modified, or damaged files for up to one month from the given calendar day. Nightly copies called “snapshots” are made of both:

• User home directories (i.e., /users/*/*)
• Project directories (i.e., /projects/*/PROJECT_NAME) including subdirectories and SURFs folders.

Exclusions

• MSI's “scratch” filesystem does not have snapshotting or any other form of backup
• By extension, links to local or global scratch are not included in any snapshot
   

Primary Storage Tape Backups & Exclusions (Tier-1 Tape Backups)

Periodic “tape” backups, for use in disaster recovery, are made of the 'disaster_recovery' folders located in each group's home "shared" and "public" folders.  These backups are stored at a secondary UMN Twin Cities campus data center.  These are for MSI staff and administrators to recover data in the rare event that MSI’s Primary Storage (Tier 1) data suffers an event that renders snapshots non-viable.

Paths targeted:

• /projects/standard/GROUP/public/disaster_recovery
• /projects/standard/GROUP/shared/disaster_recovery

Tape backups by MSI are not scoped for individual users. Data that is recovered from tape may not be from a point in time that is desirable, or may not contain an ideal or fully complete copy of data. Therefore, users should take precautions to back up, archive, or in some other way maintain secondary copies of data that are difficult or impossible to regenerate.  

Notice for Regulated Projects:

As of December 02, 2025, tape backups are not available for data stored in /projects/regulated/GROUP/shared/disaster_recovery. This functionality is still pending some internal work, and future updates will be added to this page as they become available.

No Protections: Secondary Storage (Tier 2)

MSI’s Second Tier Storage system (Ceph) is not protected by snapshots or other backups. Users should therefore take precautions to make backups of any difficult-to-recover data that is stored in Tier 2 (Ceph), as MSI cannot recover this data if it is lost or deleted. It is the responsibility of the PI to ensure that their students and collaborators have transferred ownership of all relevant data to the PI or one of the group administrators before access to MSI is terminated. After a user’s access to MSI is terminated, the data is subject to deletion as soon as space is needed (no more than two years). No copies of this data are retained.

Data on Primary Storage are retained in a Principal Investigator (PI) group directory for each annual allocation period. The PI is considered the owner of all data within the storage allocated to the group. If a PI does not renew their affiliation with MSI before the end of an allocation period (December 31), MSI will lock the PI account, which will render the project data inaccessible.  

To understand how data is treated when a user leaves MSI, please consult Data Lifecycle After a PI Leaves MSI

Groups with Data Use Agreements

Groups that store data that are governed by a 3rd party Data Use Agreement or Data Use Certification (such as dbGaP, or data governed by NDA, etc.) are responsible for adhering to the terms of their agreement, and ensuring that they are choosing the appropriate MSI storage resource for their data. Please contact [email protected] for assistance with determining how your agreement impacts the availability of any or all of the data protection services listed here.