{"id":854,"date":"2012-02-03T22:38:48","date_gmt":"2012-02-03T20:38:48","guid":{"rendered":"http:\/\/www.data-protector.org\/wordpress\/?p=854"},"modified":"2015-12-09T22:13:29","modified_gmt":"2015-12-09T21:13:29","slug":"simple-media-vaulting-off-site-tape-vaulting","status":"publish","type":"post","link":"https:\/\/www.data-protector.org\/wordpress\/2012\/02\/simple-media-vaulting-off-site-tape-vaulting\/","title":{"rendered":"Simple Media Vaulting \/ Off-Site Tape Vaulting"},"content":{"rendered":"<p>The need,&nbsp;to backup the data from your servers to media and thus to maintain data security does not stop when backed up to tape,&nbsp;there is often a request to protect the tapes from further appending or overwriting or to move the media off the tape library and to store it on a safe place (safe,&nbsp;backup computer center,&nbsp;&#8230;).&nbsp;Each Data Protector admin knows this specific situation,&nbsp;finding out which media was used in the last full backup for vaulting.&nbsp;The experienced admin will configure Data Protector to send a report or use many different pools to keep the daily backup,&nbsp;weekly backup and monthly backup apart.&nbsp;Problem: The media sets often should then be moved to a own vaulting pool before they are taken off the library.<\/p>\n<p><b>Solution:<\/b>&nbsp;Just for this problem I wrote a small Perl script.&nbsp;As always,&nbsp;I write my scripts in Perl,&nbsp;as this language is very flexible for such simple tasks.&nbsp;The script does not need a full Perl installation,&nbsp;the stripped down version,&nbsp;which is included within the Data Protector installation on the Cell Server is sufficient.&nbsp;Functional way,&nbsp;in short: It searches for sessions in a specific period,&nbsp;analyzes the media and then the media will be moved to a vaulting pool.<\/p>\n<p><b>How it works:<\/b>&nbsp;The script searches for sessions of the time frames specified in the script.&nbsp;For the sessions the used media are detected.&nbsp;The list of media is now cleared of any existing duplicates (one media may have to be shared by multiple backup jobs) and for each media the pool is detected.&nbsp;If the the pool where the media belongs to should be excluded (configured inside the script),&nbsp;then the respective media is skipped,&nbsp;otherwise,&nbsp;the media is moved into the configured destination pool.&nbsp;If you need a bit more complex version it is possible more than one copy of the script with different values \u200b\u200bfor the variables set and run.&nbsp;This also applies to different types of media (LTO and what else is there on the market&#8230;).&nbsp;At the end of the script an email with the information which tapes were moved to the destination pool is sent.&nbsp;Who does not like this feature can comment out the call to the binary&#8230;&nbsp;If you like it you need to get the tool &#8220;Blat&#8221; and to configure it&nbsp;&#8211;&nbsp;Link: <a href=\"http:\/\/www.blat.net\" title=\"http:\/\/www.blat.net\" target=\"_blank\">http:\/\/www.blat.net<\/a>.&nbsp;For simplicity,&nbsp;I have everything (the script,&nbsp;the batch file to call the script and the Blat binaries) in the bin directory of Data Protector,&nbsp;but even that can be changed.&nbsp;It could also have an extension where the media are moved into the mail slots (5 more lines of code).&nbsp;This of course makes sense only if the library has configured mail slots and the number of media to be moved offsite is less than or equal to the amount of mail slots in place.<\/p>\n<p><b>Use case:<\/b>&nbsp;You have configured the pools FilesystemWeekend,&nbsp;FilesystemDaily (for the file backup),&nbsp;DatabasesWeekend,&nbsp; DatabasesDaily (for database backup),&nbsp;DPDatabase,&nbsp;Vaulting und FreePool.&nbsp;On each monday morning the media which were used from friday evening until monday mornning must be moved into pool &#8220;Vaulting&#8221;.&nbsp;The daily pools and the DPDatabase pool (The backup of the internal database I always do to a separate pool!) should not be included for vaulting in this case.&nbsp;The required settings for the script can be find below.<\/p>\n<p><b>Sett\u00edngs:<\/b>&nbsp;What do you need to change for our use case??&nbsp;Please note the with a backslash quoted @ for the email addresses and that yo have configure &#8220;blat&#8221; beforehand (only 1 command to execute).<br \/>\n<code><br \/>\n$exclude_pools=\"FilesystemDaily,DatabasesDaily,DPDatabase,Vaulting\";<br \/>\n$vault_pool=\"Vaulting\";<br \/>\n$timeframe_start=\"60\";<br \/>\n$timeframe_duration=\"60\";<br \/>\n$mailrecipient=\"servicedesk\\@data-protector.org\";<br \/>\n$mailserver=\"127.0.0.1\";<br \/>\n$mailsender=\"dataprotector\\@data-protector.org\";<br \/>\n$mailsubject=\"Media vaulting\";<br \/>\n<\/code><\/p>\n<p>[wpdm_file id=4]<\/p>\n<p><b>UPDATE Download:<\/b>&nbsp;There was a minor bug when counting the media &#8211; new download below.<\/p>\n<p>[wpdm_file id=5]<\/p>\n<p><b>Use it for&#8230;:<\/b>&nbsp;The script is best suited for small to mid-size companies.&nbsp;For larger environments I recommend to evaluate HP Media Operations.&nbsp;As an alternative I sell a script for large environments,&nbsp;it can be enhanced on your needs and on your request.&nbsp;To get more information and a offer, please send a mail with the subject &#8220;Off-Site Tape Vaulting&#8221; to da&#x6e;&#x69;&#x65;l-&#98;&#x72;&#x61;un&#64;&#x64;&#x61;ta&#45;&#x70;&#x72;&#x6f;te&#x63;&#x74;&#x6f;r.&#111;&#x72;&#x67;<\/p>\n<p><b>Donate:<\/b>&nbsp;If you like this article and if the script helps you in your daily Data Protector business,&nbsp;I would be glad to get a donation from you (5 \u20ac are sufficient) via PayPal.&nbsp;The link to donate you&#8217;ll find in the upper right corner of this site&#8230;<\/p>\n<p><b>Reference:<\/b>&nbsp;If you are behind a proxy and unable to download the file&#8230; below you will find the complete script. If you like it smaller, please feel free to modify, the script does not need to have the &#8220;prints&#8221; and can be cutted to the half size&#8230;<\/p>\n<p><code><br \/>\n#!perl -I..\/lib\/perl -X -CA<\/p>\n<p>### needed modules<br \/>\nuse Omniback;<\/p>\n<p># Get DP folders<br \/>\nmy $OMNIHOME = Omniback::CmnQuotePath($Omniback::PAN{'BASE'});<br \/>\nmy $OMNIHOME_STR = $OMNIHOME;<br \/>\n$OMNIHOME_STR =~ s\/\\\"\/\/g;<br \/>\nmy $OMNIBIN = \"\\\"${OMNIHOME_STR}\/bin\\\"\";<\/p>\n<p># define the exclude pools, pool for vaulting, ...<br \/>\n$exclude_pools=\"Pool1,Pool2,PoolX\";<\/p>\n<p># the pool mus exist in DP<br \/>\n$vault_pool=\"Vaulting\";<\/p>\n<p># the time frame to retriev the session information<br \/>\n# assuming the script should run monday 8am, the timeframe of<br \/>\n# 60 month covers backups starting friday evening<br \/>\n$timeframe_start=\"60\";<br \/>\n$timeframe_duration=\"60\";<\/p>\n<p># the stuff needed to send a mail at the end of the script<br \/>\n# you need to download blat (mail program) search for blat at google<\/p>\n<p>$mailrecipient=\"firstname.lastname\\@domain.com\";  \t\t# please remind the quoted (at)<br \/>\n$mailserver=\"192.168.2.10\"; \t\t\t\t\t# the ip address of the smtp server<br \/>\n$mailsender=\"dataprotector\\@domain.com\";\t\t\t# please remind the quoted (at)<br \/>\n$mailsubject=\"Media vaulting\";\t\t\t\t\t# Subject for the mail<\/p>\n<p>print \"Generating Report\\n\";<br \/>\nprint \"- skipping pools: $exclude_pools\\n\";<br \/>\nprint \"- target pool is: $vault_pool\\n\";<br \/>\nprint \"- start time: $timeframe_start\\n\";<br \/>\nprint \"- duration: $timeframe_duration\\n\";<br \/>\nprint \"\\n\\nNote: Ignore messages regarding \\\"No medium found\\\"\\n\\n\";<\/p>\n<p>@exec=`${OMNIBIN}\/omnirpt -report list_sessions -timeframe $timeframe_start $timeframe_duration -tab`;<br \/>\nforeach (@exec)<br \/>\n{<br \/>\n\tchomp $_;<br \/>\n\tnext if \/^#\/;<br \/>\n\t(@sessionid)=split(\/\\t\/,$_,23);<br \/>\n\tpush(@sessions,$sessionid[22]);<br \/>\n}<\/p>\n<p>foreach (@sessions)<br \/>\n{<br \/>\n\t@exec=`${OMNIBIN}\/omnidb.exe -session \\\"$_\\\" -media`;<br \/>\n\t$media = $exec[2];<br \/>\n\tchomp $media;<br \/>\n\t($media_short,$dummy)=split(\/  \/,$media,2);<br \/>\n\tif ($media_short ne \"\")<br \/>\n\t{<br \/>\n\t\tpush(@media,$media_short);<br \/>\n\t}<br \/>\n}<\/p>\n<p>@newmedia=&del_double(@media);<\/p>\n<p>print \"Summary\\n\";<br \/>\n@movedmedia=();<br \/>\nforeach (sort(@newmedia))<br \/>\n{<\/p>\n<p>\t($media1,$dummy)=split(\/ \/,$_,2);<br \/>\n\t$media1 =~ s\/\\[\/\/ig;<br \/>\n\t$media1 =~ s\/\\]\/\/ig;<\/p>\n<p>\t@exec=`${OMNIBIN}\/omnimm -media_info \\\"$media1\\\" -detail`;<br \/>\n\t$pool = $exec[2];<br \/>\n\tchomp $pool;<br \/>\n\tchop $pool;<br \/>\n\t$pool =~ s\/Pool name              : \/\/ig;<br \/>\n\tif ($exclude_pools =~ \/$pool\/)<br \/>\n\t{<br \/>\n\t\tprint \"skipping media: $media1 \\n\";<br \/>\n\t}<br \/>\n\telse<br \/>\n\t{<br \/>\n\t\tprint \"moving media: $media1 \\n\";<br \/>\n\t\t@exec=`${OMNIBIN}\/omnimm -move_medium \\\"$media1\\\" $vault_pool`;<br \/>\n\t\tpush(@moved_media,$media1);<\/p>\n<p>\t}<br \/>\n}<\/p>\n<p>$anzahl=@moved_media;<br \/>\nif ($anzahl gt 0)<br \/>\n{<br \/>\n\t$message=\"\";<br \/>\n\t$message=\"Vaulting media:\\n============================\\n\\n\";<br \/>\n\t$message=$message.\"Following media were moved to pool: $vault_pool for vaulting.\\n\";<br \/>\n\t$message=$message.\"Media:\\n\";<br \/>\n\tforeach (@moved_media)<br \/>\n\t{<br \/>\n\t\t$message=$message.$_.\"\\n\";<br \/>\n\t}<\/p>\n<p>\t$message=$message.\"\\n\\nMail generated by robot\\n\\n\";<br \/>\n\t`${OMNIBIN}\/blat -to $mailrecipient -f $mailsender -subject \\\"$mailsubject\\\" -body \\\"$message\\\"`;<br \/>\n}<\/p>\n<p>exit;<\/p>\n<p>sub del_double{<br \/>\nmy %all;<br \/>\ngrep {$all{$_}=0} @_;<br \/>\nreturn (keys %all);<br \/>\n}<br \/>\n<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The need,&nbsp;to backup the data from your servers to media and thus to maintain data security does not stop when backed up to tape,&nbsp;there is often a request to protect the tapes from further appending or overwriting or to move the media off the tape library and to store it on a safe place (safe,&nbsp;backup [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"spay_email":""},"categories":[11],"tags":[129],"translation":{"provider":"WPGlobus","version":"2.8.8","language":"en","enabled_languages":["en","de"],"languages":{"en":{"title":true,"content":true,"excerpt":false},"de":{"title":false,"content":true,"excerpt":false}}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p28cjj-dM","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1428,"url":"https:\/\/www.data-protector.org\/wordpress\/2013\/06\/basics-dp-catalog-protection-data-protection\/","url_meta":{"origin":854,"position":0},"title":"BASICS &#8211; DP Catalog Protection and Data Protection","date":"10. June 2013","format":false,"excerpt":"{:en}Within Data Protector the protection is set per backup job and not for the tape, which might be misunderstood by customers. In the GUI a distinction is made between \"Data Protection\" and \"Catalog Protection\". \"Data Protection\" describes how long the data is kept on media and \"Catalog Protection\" describes how\u2026","rel":"","context":"In &quot;BASICS&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":87,"url":"https:\/\/www.data-protector.org\/wordpress\/2010\/12\/was-ist-eigentlich-device-polling\/","url_meta":{"origin":854,"position":1},"title":"What is device polling","date":"31. December 2010","format":false,"excerpt":"{:de}Bestimmt jeder Data Protector Kunde mit einer Fibre Channel Tape Library hat dieses Problem schon mal wissentlich oder unwissentlich erlebt: ein SCSI Reset w\u00e4hrend einer Sicherung... und das ganze Band ist zerst\u00f6rt. Was war passiert? Das Bandlaufwerk wurde mit\u00a0SCSI Kommandos \u00fcbersch\u00fcttet - das nennt man auch Device Polling. Polling ist\u2026","rel":"","context":"In &quot;Common&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":694,"url":"https:\/\/www.data-protector.org\/wordpress\/2011\/07\/data-protector-migration-unix-to-other-os\/","url_meta":{"origin":854,"position":2},"title":"Data Protector Migration Unix to other OS","date":"29. July 2011","format":false,"excerpt":"{:de}Ich kenne nicht viele gute Gr\u00fcnde warum man eine Data Protector Installation von HP-UX auf ein anderes Betriebssystem wechseln sollte,\u00a0dennoch gibt es viele Kunden die diesen Schritt gehen wollen und zum Beispiel auf Windows oder Linux wechseln;\u00a0meist noch verbunden mit einem Upgrade auf eine aktuelle Version von Data Protector.\u00a0Und hier\u2026","rel":"","context":"In &quot;HowTo&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":93,"url":"https:\/\/www.data-protector.org\/wordpress\/2010\/07\/corrupt-media-header\/","url_meta":{"origin":854,"position":3},"title":"Corrupt Media Header","date":"5. July 2010","format":false,"excerpt":"{:en}No translation for this article available.{:} {:de} Woran erkennt man das ein Band korrupt ist und der Header nicht mehr lesbar ist? Folgende Meldungen k\u00f6nnen ein Indiz daf\u00fcr sein. In der Regel ist ein Restore von solchen B\u00e4ndern nicht mehr m\u00f6glich, einzig das HP lab k\u00f6nnte unter Umst\u00e4nden noch Daten\u2026","rel":"","context":"In &quot;Common&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1424,"url":"https:\/\/www.data-protector.org\/wordpress\/2013\/06\/storeonce-software-store-daily-maintenance\/","url_meta":{"origin":854,"position":4},"title":"StoreOnce Software Store &#8211; daily maintenance","date":"6. June 2013","format":false,"excerpt":"{:en}When in Data Protector media of a StoreOnce software store become expired, the used disk space is not released immediately. The disk space will be released once the media is overwritten or deleted. The housekeeping process running in the background to release the space will start only when the media\u2026","rel":"","context":"In &quot;Common&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1360,"url":"https:\/\/www.data-protector.org\/wordpress\/2013\/05\/data-management-business\/","url_meta":{"origin":854,"position":5},"title":"COM IN &#8211; data management for business","date":"8. May 2013","format":false,"excerpt":"{:de}Der HP Partner Comline l\u00e4dt am 18.06.2013 nach N\u00fcrnberg zu einer Veranstaltung rund um das Thema Datenmanagement ein. Business IT Alignment, unstrukturierte Daten, Sicherung von Au\u00dfenstellen (Deduplizierung), Archivierung, Enterprise Search, Backup to Cloud, Datensicherheit sind Punkte die w\u00e4hrend der Vortr\u00e4ge besprochen werden und diskutiert werden k\u00f6nnen. Mit der COM IN\u2026","rel":"","context":"In &quot;SCD&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854"}],"collection":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/comments?post=854"}],"version-history":[{"count":25,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854\/revisions"}],"predecessor-version":[{"id":2708,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854\/revisions\/2708"}],"wp:attachment":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/media?parent=854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/categories?post=854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/tags?post=854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}