{"id":854,"date":"2012-02-03T22:38:48","date_gmt":"2012-02-03T20:38:48","guid":{"rendered":"http:\/\/www.data-protector.org\/wordpress\/?p=854"},"modified":"2015-12-09T22:13:29","modified_gmt":"2015-12-09T21:13:29","slug":"simple-media-vaulting-off-site-tape-vaulting","status":"publish","type":"post","link":"https:\/\/www.data-protector.org\/wordpress\/2012\/02\/simple-media-vaulting-off-site-tape-vaulting\/","title":{"rendered":"Simple Media Vaulting \/ Off-Site Tape Vaulting"},"content":{"rendered":"<p>The need,&nbsp;to backup the data from your servers to media and thus to maintain data security does not stop when backed up to tape,&nbsp;there is often a request to protect the tapes from further appending or overwriting or to move the media off the tape library and to store it on a safe place (safe,&nbsp;backup computer center,&nbsp;&#8230;).&nbsp;Each Data Protector admin knows this specific situation,&nbsp;finding out which media was used in the last full backup for vaulting.&nbsp;The experienced admin will configure Data Protector to send a report or use many different pools to keep the daily backup,&nbsp;weekly backup and monthly backup apart.&nbsp;Problem: The media sets often should then be moved to a own vaulting pool before they are taken off the library.<\/p>\n<p><b>Solution:<\/b>&nbsp;Just for this problem I wrote a small Perl script.&nbsp;As always,&nbsp;I write my scripts in Perl,&nbsp;as this language is very flexible for such simple tasks.&nbsp;The script does not need a full Perl installation,&nbsp;the stripped down version,&nbsp;which is included within the Data Protector installation on the Cell Server is sufficient.&nbsp;Functional way,&nbsp;in short: It searches for sessions in a specific period,&nbsp;analyzes the media and then the media will be moved to a vaulting pool.<\/p>\n<p><b>How it works:<\/b>&nbsp;The script searches for sessions of the time frames specified in the script.&nbsp;For the sessions the used media are detected.&nbsp;The list of media is now cleared of any existing duplicates (one media may have to be shared by multiple backup jobs) and for each media the pool is detected.&nbsp;If the the pool where the media belongs to should be excluded (configured inside the script),&nbsp;then the respective media is skipped,&nbsp;otherwise,&nbsp;the media is moved into the configured destination pool.&nbsp;If you need a bit more complex version it is possible more than one copy of the script with different values \u200b\u200bfor the variables set and run.&nbsp;This also applies to different types of media (LTO and what else is there on the market&#8230;).&nbsp;At the end of the script an email with the information which tapes were moved to the destination pool is sent.&nbsp;Who does not like this feature can comment out the call to the binary&#8230;&nbsp;If you like it you need to get the tool &#8220;Blat&#8221; and to configure it&nbsp;&#8211;&nbsp;Link: <a href=\"http:\/\/www.blat.net\" title=\"http:\/\/www.blat.net\" target=\"_blank\">http:\/\/www.blat.net<\/a>.&nbsp;For simplicity,&nbsp;I have everything (the script,&nbsp;the batch file to call the script and the Blat binaries) in the bin directory of Data Protector,&nbsp;but even that can be changed.&nbsp;It could also have an extension where the media are moved into the mail slots (5 more lines of code).&nbsp;This of course makes sense only if the library has configured mail slots and the number of media to be moved offsite is less than or equal to the amount of mail slots in place.<\/p>\n<p><b>Use case:<\/b>&nbsp;You have configured the pools FilesystemWeekend,&nbsp;FilesystemDaily (for the file backup),&nbsp;DatabasesWeekend,&nbsp; DatabasesDaily (for database backup),&nbsp;DPDatabase,&nbsp;Vaulting und FreePool.&nbsp;On each monday morning the media which were used from friday evening until monday mornning must be moved into pool &#8220;Vaulting&#8221;.&nbsp;The daily pools and the DPDatabase pool (The backup of the internal database I always do to a separate pool!) should not be included for vaulting in this case.&nbsp;The required settings for the script can be find below.<\/p>\n<p><b>Sett\u00edngs:<\/b>&nbsp;What do you need to change for our use case??&nbsp;Please note the with a backslash quoted @ for the email addresses and that yo have configure &#8220;blat&#8221; beforehand (only 1 command to execute).<br \/>\n<code><br \/>\n$exclude_pools=\"FilesystemDaily,DatabasesDaily,DPDatabase,Vaulting\";<br \/>\n$vault_pool=\"Vaulting\";<br \/>\n$timeframe_start=\"60\";<br \/>\n$timeframe_duration=\"60\";<br \/>\n$mailrecipient=\"servicedesk\\@data-protector.org\";<br \/>\n$mailserver=\"127.0.0.1\";<br \/>\n$mailsender=\"dataprotector\\@data-protector.org\";<br \/>\n$mailsubject=\"Media vaulting\";<br \/>\n<\/code><\/p>\n<p>[wpdm_file id=4]<\/p>\n<p><b>UPDATE Download:<\/b>&nbsp;There was a minor bug when counting the media &#8211; new download below.<\/p>\n<p>[wpdm_file id=5]<\/p>\n<p><b>Use it for&#8230;:<\/b>&nbsp;The script is best suited for small to mid-size companies.&nbsp;For larger environments I recommend to evaluate HP Media Operations.&nbsp;As an alternative I sell a script for large environments,&nbsp;it can be enhanced on your needs and on your request.&nbsp;To get more information and a offer, please send a mail with the subject &#8220;Off-Site Tape Vaulting&#8221; to &#x64;&#x61;&#x6e;&#105;el-&#x62;&#x72;&#x61;&#x75;&#110;&#64;da&#x74;&#x61;&#x2d;&#112;&#114;ote&#x63;&#x74;&#x6f;&#114;.or&#x67;<\/p>\n<p><b>Donate:<\/b>&nbsp;If you like this article and if the script helps you in your daily Data Protector business,&nbsp;I would be glad to get a donation from you (5 \u20ac are sufficient) via PayPal.&nbsp;The link to donate you&#8217;ll find in the upper right corner of this site&#8230;<\/p>\n<p><b>Reference:<\/b>&nbsp;If you are behind a proxy and unable to download the file&#8230; below you will find the complete script. If you like it smaller, please feel free to modify, the script does not need to have the &#8220;prints&#8221; and can be cutted to the half size&#8230;<\/p>\n<p><code><br \/>\n#!perl -I..\/lib\/perl -X -CA<\/p>\n<p>### needed modules<br \/>\nuse Omniback;<\/p>\n<p># Get DP folders<br \/>\nmy $OMNIHOME = Omniback::CmnQuotePath($Omniback::PAN{'BASE'});<br \/>\nmy $OMNIHOME_STR = $OMNIHOME;<br \/>\n$OMNIHOME_STR =~ s\/\\\"\/\/g;<br \/>\nmy $OMNIBIN = \"\\\"${OMNIHOME_STR}\/bin\\\"\";<\/p>\n<p># define the exclude pools, pool for vaulting, ...<br \/>\n$exclude_pools=\"Pool1,Pool2,PoolX\";<\/p>\n<p># the pool mus exist in DP<br \/>\n$vault_pool=\"Vaulting\";<\/p>\n<p># the time frame to retriev the session information<br \/>\n# assuming the script should run monday 8am, the timeframe of<br \/>\n# 60 month covers backups starting friday evening<br \/>\n$timeframe_start=\"60\";<br \/>\n$timeframe_duration=\"60\";<\/p>\n<p># the stuff needed to send a mail at the end of the script<br \/>\n# you need to download blat (mail program) search for blat at google<\/p>\n<p>$mailrecipient=\"firstname.lastname\\@domain.com\";  \t\t# please remind the quoted (at)<br \/>\n$mailserver=\"192.168.2.10\"; \t\t\t\t\t# the ip address of the smtp server<br \/>\n$mailsender=\"dataprotector\\@domain.com\";\t\t\t# please remind the quoted (at)<br \/>\n$mailsubject=\"Media vaulting\";\t\t\t\t\t# Subject for the mail<\/p>\n<p>print \"Generating Report\\n\";<br \/>\nprint \"- skipping pools: $exclude_pools\\n\";<br \/>\nprint \"- target pool is: $vault_pool\\n\";<br \/>\nprint \"- start time: $timeframe_start\\n\";<br \/>\nprint \"- duration: $timeframe_duration\\n\";<br \/>\nprint \"\\n\\nNote: Ignore messages regarding \\\"No medium found\\\"\\n\\n\";<\/p>\n<p>@exec=`${OMNIBIN}\/omnirpt -report list_sessions -timeframe $timeframe_start $timeframe_duration -tab`;<br \/>\nforeach (@exec)<br \/>\n{<br \/>\n\tchomp $_;<br \/>\n\tnext if \/^#\/;<br \/>\n\t(@sessionid)=split(\/\\t\/,$_,23);<br \/>\n\tpush(@sessions,$sessionid[22]);<br \/>\n}<\/p>\n<p>foreach (@sessions)<br \/>\n{<br \/>\n\t@exec=`${OMNIBIN}\/omnidb.exe -session \\\"$_\\\" -media`;<br \/>\n\t$media = $exec[2];<br \/>\n\tchomp $media;<br \/>\n\t($media_short,$dummy)=split(\/  \/,$media,2);<br \/>\n\tif ($media_short ne \"\")<br \/>\n\t{<br \/>\n\t\tpush(@media,$media_short);<br \/>\n\t}<br \/>\n}<\/p>\n<p>@newmedia=&del_double(@media);<\/p>\n<p>print \"Summary\\n\";<br \/>\n@movedmedia=();<br \/>\nforeach (sort(@newmedia))<br \/>\n{<\/p>\n<p>\t($media1,$dummy)=split(\/ \/,$_,2);<br \/>\n\t$media1 =~ s\/\\[\/\/ig;<br \/>\n\t$media1 =~ s\/\\]\/\/ig;<\/p>\n<p>\t@exec=`${OMNIBIN}\/omnimm -media_info \\\"$media1\\\" -detail`;<br \/>\n\t$pool = $exec[2];<br \/>\n\tchomp $pool;<br \/>\n\tchop $pool;<br \/>\n\t$pool =~ s\/Pool name              : \/\/ig;<br \/>\n\tif ($exclude_pools =~ \/$pool\/)<br \/>\n\t{<br \/>\n\t\tprint \"skipping media: $media1 \\n\";<br \/>\n\t}<br \/>\n\telse<br \/>\n\t{<br \/>\n\t\tprint \"moving media: $media1 \\n\";<br \/>\n\t\t@exec=`${OMNIBIN}\/omnimm -move_medium \\\"$media1\\\" $vault_pool`;<br \/>\n\t\tpush(@moved_media,$media1);<\/p>\n<p>\t}<br \/>\n}<\/p>\n<p>$anzahl=@moved_media;<br \/>\nif ($anzahl gt 0)<br \/>\n{<br \/>\n\t$message=\"\";<br \/>\n\t$message=\"Vaulting media:\\n============================\\n\\n\";<br \/>\n\t$message=$message.\"Following media were moved to pool: $vault_pool for vaulting.\\n\";<br \/>\n\t$message=$message.\"Media:\\n\";<br \/>\n\tforeach (@moved_media)<br \/>\n\t{<br \/>\n\t\t$message=$message.$_.\"\\n\";<br \/>\n\t}<\/p>\n<p>\t$message=$message.\"\\n\\nMail generated by robot\\n\\n\";<br \/>\n\t`${OMNIBIN}\/blat -to $mailrecipient -f $mailsender -subject \\\"$mailsubject\\\" -body \\\"$message\\\"`;<br \/>\n}<\/p>\n<p>exit;<\/p>\n<p>sub del_double{<br \/>\nmy %all;<br \/>\ngrep {$all{$_}=0} @_;<br \/>\nreturn (keys %all);<br \/>\n}<br \/>\n<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The need,&nbsp;to backup the data from your servers to media and thus to maintain data security does not stop when backed up to tape,&nbsp;there is often a request to protect the tapes from further appending or overwriting or to move the media off the tape library and to store it on a safe place (safe,&nbsp;backup [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"spay_email":""},"categories":[11],"tags":[129],"translation":{"provider":"WPGlobus","version":"2.8.8","language":"en","enabled_languages":["en","de"],"languages":{"en":{"title":true,"content":true,"excerpt":false},"de":{"title":false,"content":true,"excerpt":false}}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p28cjj-dM","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":1819,"url":"https:\/\/www.data-protector.org\/wordpress\/2014\/01\/storeonce-success-story-australia\/","url_meta":{"origin":854,"position":0},"title":"StoreOnce success story from Australia","date":"27. January 2014","format":false,"excerpt":"{:en}Recently I asked you to share your StoreOnce success story. Here is a awesome story and a good example for federated deduplication from a company in Australia. Before using HP StoreOnce, the company used to protect its data in the two datacentres using HP Data Protector. The data (FileSystems, MS\u2026","rel":"","context":"In &quot;SCD&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":694,"url":"https:\/\/www.data-protector.org\/wordpress\/2011\/07\/data-protector-migration-unix-to-other-os\/","url_meta":{"origin":854,"position":1},"title":"Data Protector Migration Unix to other OS","date":"29. July 2011","format":false,"excerpt":"{:de}Ich kenne nicht viele gute Gr\u00fcnde warum man eine Data Protector Installation von HP-UX auf ein anderes Betriebssystem wechseln sollte,\u00a0dennoch gibt es viele Kunden die diesen Schritt gehen wollen und zum Beispiel auf Windows oder Linux wechseln;\u00a0meist noch verbunden mit einem Upgrade auf eine aktuelle Version von Data Protector.\u00a0Und hier\u2026","rel":"","context":"In &quot;HowTo&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2395,"url":"https:\/\/www.data-protector.org\/wordpress\/2015\/09\/future-tape-technology\/","url_meta":{"origin":854,"position":2},"title":"The future of the Tape Technology","date":"25. September 2015","format":false,"excerpt":"{:en} Many customers continue to rely on tape storage to protect their business critical applications and data. The benefit to continue performing classic backup is obvious: Tape storage is cheap compared with backup-to-disk technolgies. In a TCO consideration lower cost per GB and lower energy costs are speaking for an\u2026","rel":"","context":"In &quot;Common&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1478,"url":"https:\/\/www.data-protector.org\/wordpress\/2013\/07\/write-device-parameter-incorrect\/","url_meta":{"origin":854,"position":3},"title":"Cannot write to device &#8211; the parameter is incorrect","date":"5. July 2013","format":false,"excerpt":"{:en}When using the wrong block size for the tape devices, you may get the error message Cannot write to device ([87] The Parameter is incorrect.) during the backup or while formatting media. This error may occur if as SAS library, attached to the controller P212-ZM is used. When you try\u2026","rel":"","context":"In &quot;HowTo&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":211,"url":"https:\/\/www.data-protector.org\/wordpress\/2010\/10\/initialization-of-medium-failed-filelibrary-full\/","url_meta":{"origin":854,"position":4},"title":"Initialization of medium failed &#8211; filelibrary full","date":"22. October 2010","format":false,"excerpt":"{:en}The error Initialization of medium failed is shown when the Filelibrary is full or almost full (one or more mountpoints \/ folders in filelibrary). If you did not install the GR Patches from July 2010, you may also see a lot of media with blue question mark withhin the mountpoint\u2026","rel":"","context":"In &quot;Bugs&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":815,"url":"https:\/\/www.data-protector.org\/wordpress\/2011\/11\/data-protector-cases-and-hp-support\/","url_meta":{"origin":854,"position":5},"title":"Data Protector Cases and HP Support","date":"30. November 2011","format":false,"excerpt":"{:de}Nicht immer ist Software von Fehlern befreit und eine Funktion im Data Protector arbeitet nicht wie gew\u00fcnscht oder liefert nicht das notwendige Ergebnis. In diesen F\u00e4llen ben\u00f6tigt man die Unterst\u00fctzung von Support um einen m\u00f6glichen Fehler zu identifizieren, zu analysieren und zu beheben. Nachfolgend findet ihr ein Anleitung wie man\u2026","rel":"","context":"In &quot;Common&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854"}],"collection":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/comments?post=854"}],"version-history":[{"count":25,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854\/revisions"}],"predecessor-version":[{"id":2708,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/posts\/854\/revisions\/2708"}],"wp:attachment":[{"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/media?parent=854"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/categories?post=854"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.data-protector.org\/wordpress\/wp-json\/wp\/v2\/tags?post=854"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}