{"id":1161,"date":"2012-03-02T20:21:11","date_gmt":"2012-03-03T00:21:11","guid":{"rendered":"http:\/\/antipaucity.com\/?p=1161"},"modified":"2012-03-02T20:21:11","modified_gmt":"2012-03-03T00:21:11","slug":"storage-strategies-part-4","status":"publish","type":"post","link":"https:\/\/merikebi.warrenmyers.com\/?p=1161","title":{"rendered":"storage strategies &#8211; part 4"},"content":{"rendered":"<p>Last <a href=\"http:\/\/antipaucity.com\/2012\/03\/01\/storage-strategies-part-3\">time<\/a> I talked about storage robustifiers.<\/p>\n<p>In the context of a couple applications with which I am familiar, I want to discuss ways to approach balancing storage types and allocations.<\/p>\n<h2>Storage Overview<\/h2>\n<p>Core requirements of any modern server, from a storage standpoint, are the following:<\/p>\n<ul>\n<li>RAM<\/li>\n<li><a href=\"http:\/\/antipaucity.com\/2011\/08\/08\/why-technical-intricacies-matter\">swap<\/a><\/li>\n<li>Base OS storage<\/li>\n<li>OS\/application log storage<\/li>\n<li>Application storage<\/li>\n<\/ul>\n<p>Of course, many more elements <em>could<\/em>\u00a0be added &#8211; but these are the ones I am going to consider today. From a cost perspective, hardware is <em>almost <strong>always<\/strong><\/em>\u00a0the least expensive part of an application deployment &#8211; licensing, development, maintenance, and other non-&#8220;physical&#8221; costs will typically <em>far<\/em>\u00a0outweigh the expenses of hardware.<\/p>\n<h4>RAM<\/h4>\n<p>RAM is cheap, comparatively. Any modern OS will lap-up as much memory as is given to it, so I always try to err on the side of generous.<\/p>\n<h4>swap<\/h4>\n<p>After having found an instance where <em><a href=\"http:\/\/http:\/\/antipaucity.com\/2011\/08\/08\/why-technical-intricacies-matter\">Swap Really Mattered<\/a><\/em>\u2122, I <strong><em>always<\/em><\/strong>\u00a0follow the Red Hat guidelines which state that swap space should be equal to installed RAM plus 2 gigabytes. For example, if 16GB of RAM is installed, swap should at least equal 18GB. Whether swap should be on a physical disk or in a logical volume is up for <a href=\"http:\/\/serverfault.com\/q\/329928\/2321\">debate<\/a>, but do not chintz on swap! It is vital to the healthy operation of almost all modern systems!<\/p>\n<h4>Base OS<\/h4>\n<p>This will vary on a per-platform basis, but a common rule-of-thumb is that Linux needs about 10GB for itself. Windows 2008 R2 requests a minimum of 20GB, but 40GB is substantially better.<\/p>\n<h4>OS\/application logs<\/h4>\n<p>Here is another wild variable &#8211;\u00a0though\u00a0most applications have pretty predictable log storage requirements. For example, HP&#8217;s Server Automation (HPSA) tool will rarely exceed 10GB in total log file usage. Some things, like Apache, may have varying log files depending on how busy a website is.<\/p>\n<h4>Application<\/h4>\n<p>Lastly, and definitely most importantly, is the discussion surrounding the actual space needed for an application to be installed and run. On two ends of the spectrum, I will use Apache and HPSA for my examples.<\/p>\n<p>The Apache <em>application<\/em>\u00a0only requires a few dozen megabytes to install and run. The <em>content<\/em>\u00a0served-up by Apache can, of course, be a <em><strong>big<\/strong><\/em>\u00a0variable &#8211; a simple, static website might only use a few dozen megabytes. Whereas a big, complex website (like <a href=\"http:\/\/stackoverflow.com\">StackOverflow<\/a>) might be using a few hundred gigabytes (in total with any dynamically-generated content which might be in a database, or similar)<sup>*<\/sup>.<\/p>\n<p>The best way to address varying storage needs, in my opinion, is to use a robustifying tool like <a href=\"http:\/\/en.wikipedia.org\/wiki\/Logical_Volume_Manager_(Linux)\">LVM<\/a>\u00a0&#8211; take storage presented to the server, and conglomerate it into a single mount point (perhaps <code>\/var\/www\/html<\/code>) so that as content needs change, it can be grown transparently to the website.<\/p>\n<p>Likewise, with HPSA, there are several base storage needs &#8211; for Oracle, initial software content, the application itself, etc. Picking-up on a previous post on <a href=\"http:\/\/antipaucity.com\/2012\/02\/27\/binding-your-mounts\">bind mounts<\/a>, I think it a Very Good Thing\u2122 to present a mass of storage to a single initial mount point, like <code>\/apps<\/code>, and then put several subdirectories in place to hold the &#8220;actual&#8221; application. Storage usage for the &#8220;variables&#8221; of HPSA &#8211; Software Repository, OS Media, Model Repository (database) &#8211; is very hard to predict, but a base guideline is that you need 100GB in total to <em>start<\/em>.<\/p>\n<h2>Choosing Storage Types<\/h2>\n<h3>My recommendations<\/h3>\n<p>This is how <em><strong>I<\/strong><\/em>\u00a0like to approach storage allocation on a physical server (virtual machines are a whole other beast, which I&#8217;l address in a future post) for HP Server Automation:<\/p>\n<h4>Base OS<\/h4>\n<p>I firmly believe this should be put on local storage &#8211; ideally a pair of mirror RAIDed 73GB drives (these could be SSDs, to accelerate boot time, but otherwise the OS, <em>per se<\/em>, should not be being &#8220;used&#8221; much. You could easily get away with 36GB drives, but since drives <em>are<\/em>\u00a0cheap, using slightly\u00a0more\u00a0than you &#8220;need&#8221; is fine.<\/p>\n<h4>swap<\/h4>\n<p>Again, following the Red Hat guidelines (plus my patented <del>fudge<\/del> growth factor), ideally I want swap to be on either a pair of 36GB or a pair of 73GB drives &#8211; not RAIDed (neither striping, nor mirroring swap makes a great deal of sense). Yes, this means you should create a pair of swap partitions and present the whole shebang to the OS.<\/p>\n<h4>OS\/application logs<\/h4>\n<p>Maybe this is a little paranoid, but I like to have <em>at least<\/em>\u00a030GB for log space (<code>\/var\/log<\/code>). I view logs to be <a href=\"http:\/\/http:\/\/cnx.org\/content\/m40795\/latest\/?collection=col11350\/latest\">absolutely vital<\/a> in the monitoring and troubleshooting arenas, so don&#8217;t chintz here!<\/p>\n<h4>Application<\/h4>\n<p>HPSA has four main space hogs, so I&#8217;ll talk about them as subheadings.<\/p>\n<h5>Oracle<\/h5>\n<p>It is important that the database has <em>plenty<\/em>\u00a0of space &#8211; start at 200GB (if possible), and present it as a logically-managed volume group, preferably made up of one or more [growable] LUNs from a SAN.<\/p>\n<p style=\"padding-left: 30px;\">Note: Thin-provisioning is a perfectly-acceptable approach to use, by the way (thin provisioning present space as &#8220;available&#8221; but not yet &#8220;allocated&#8221; from the storage device to the server).<\/p>\n<h5>Core application<\/h5>\n<p>The application really doesn&#8217;t grow <em>that<\/em>\u00a0much over time (patches and upgrades <em><strong>do<\/strong><\/em>\u00a0cause growth, but they are pretty well-defined).<\/p>\n<p>Since this is the case, carve 50-60GB and present it as a [growable] LUN via LVM to the OS.<\/p>\n<h5>OS Media<\/h5>\n<p>Depending on data retention policies, number of distinct OS flavors you need to deploy, and a few other factors, it is a Good Idea\u2122 to allocate a minimum of 40GB for holding OS media (raw-copied content from vendor-supplied installation ISOs). RHEL takes about 3.5GB per copy, and Windows 2008 R2 takes about the same. Whether this space is presented as an NFS share from a NAS, or as a [growable] LUN under an LVM group from a SAN isn&#8217;t vitally-important, but having enough space most certainly is.<\/p>\n<h5>Software Library<\/h5>\n<p>This is truly the biggest wildcard of them all &#8211; how many distinct packages do you plan to deploy? How big are they? How many versions needs to be kept? How many target OSes will you be managing?<\/p>\n<p>I prefer to <em>start<\/em>\u00a0with 50GB available to the Library. But I also expect that usage to grow <em><strong>rapidly<\/strong><\/em>\u00a0once the system is in live use &#8211; Software Libraries exceeding 300GB are <em>not<\/em>\u00a0uncommon in my field. As with the OS Media discussion, it isn&#8217;t vitally-important whether this space is allocated from a NAS or a SAN, but it definitely needs to be growable!<\/p>\n<h4>Closing comments (on HPSA and storage)<\/h4>\n<p>If separate storage options are not available for the big hogs of SA, allocating one, big LVM volume (made up of LUNs and\/or DAS volumes), and then relying on bind mounts is a great solution (and avoids the issue of needing to worry about any given chunk of the tool exceeding its bounds too badly &#8211; especially if other\u00a0parts\u00a0aren&#8217;t being as heavily-used as might have been anticipated).<\/p>\n<hr \/>\n<p><sup>*<sub>Yes, yes, I know &#8211; once you hit a certain size, the presentation and content layers should be split into separate systems. For purposes of <em>this<\/em> example, I&#8217;m leaving it all together.<\/sub><\/sup><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last time I talked about storage robustifiers. In the context of a couple applications with which I am familiar, I want to discuss ways to approach balancing storage types and allocations. Storage Overview Core requirements of any modern server, from a storage standpoint, are the following: RAM swap Base OS storage OS\/application log storage Application &hellip;<br \/><a href=\"https:\/\/merikebi.warrenmyers.com\/?p=1161\" class=\"more-link pen_button pen_element_default pen_icon_arrow_double\">Continue reading <span class=\"screen-reader-text\">storage strategies &#8211; part 4<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[41],"tags":[394,450,600,641,725,727],"keyring_services":[],"class_list":["post-1161","post","type-post","status-publish","format-standard","hentry","category-technical","tag-hpsa","tag-linux","tag-proximal","tag-robustifier","tag-storage","tag-strategies"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts\/1161","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1161"}],"version-history":[{"count":0,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=\/wp\/v2\/posts\/1161\/revisions"}],"wp:attachment":[{"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1161"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1161"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1161"},{"taxonomy":"keyring_services","embeddable":true,"href":"https:\/\/merikebi.warrenmyers.com\/index.php?rest_route=%2Fwp%2Fv2%2Fkeyring_services&post=1161"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}