IT Focus Area: infrastructure operations
May 19, 2015
How to Navigate the Shifting Storage Landscape
Call it the burden of choice.
Many companies still capture and store data on traditional spinning disk drives in corporate data centers, but the data storage landscape is drastically changing. And while it is generally better to have options, the sheer volume of emerging solutions has left companies confused about how to handle their individual storage needs.
The process of choosing from options like flash arrays, software-defined storage, cloud and converged/hyper-converged can be overwhelming. There is misunderstanding permeating the industry with most IT organizations dissatisfied with the economics of their storage.
And with the digital universe doubling in size every two years, according to joint research by IDC and EMC, companies simply cannot afford to become mired in confusion. They must figure out the best methods for collecting and storing vast amounts of data, which IDC and EMC predict will reach 44 trillion gigabytes annually by 2020.
Making a decision about data storage is a complex process, if done correctly. Organizations need to carefully consider a number of factors including their own individual needs, industry, cost analyses and business goals. Some companies may find that a combination of different solutions best suits their needs.
But first, it is important to understand the intricacies of different solutions, and use them to weigh the pros and cons of each option against business needs.
Flash is poised to become one of 2015’s biggest trends, with its increasing popularity is due in part to a significant drop in price. Prices for flash storage, once hefty at more than $1,000 per GB, have fallen to less than $10 per GB in the past decade. Enterprise storage vendors and startups are largely responsible for this development.
The improvements they have made to the reliability and performance of flash-based arrays have lowered the effective cost per gigabyte. Dubbed “superefficient,” flash systems that combine high-performance flash and cutting-edge data-reduction technologies have also contributed to lower costs. The mainstream application of deduplication and compression, which is well suited for flash-based storage, will likely further reduce the cost once they deploy.
Given these developments, it is hardly surprising that the flash-based array market grew past $11 billion in 2014, according to new research from IDC. And price is not the only benefit flash offers. The agile solution excels in environments where performance is the driving factor. As performance demands increase, flash-based storage exhibits a very favorable total cost of ownership (TCO) pattern.
Companies are adopting flash left and right because of benefits such as the aforementioned characteristics. However, organizations must understand certain disadvantages when considering a move to an all-flash storage system. A major drawback is the rip-and-replace nature of all-flash. In many cases, it will not work in tandem with current data center technology without specialized software, meaning much of an organization’s current data center architecture may have to be reevaluated. This elevates cost and time investment right off the bat.
In addition, implementing flash may become even more costly for organizations that need synchronized replication with zero data loss. In order to prevent data loss, they should include a virtualization appliance with an all-flash array. Its inclusion may be necessary because enterprise data management features are still largely immature.
The cost of flash is certainly attractive compared to traditional storage, especially when paired with hyper-converged systems. Still, it is imperative for companies to evaluate their data compression abilities, as data compression is essential for flash.
Converged and Hyper-converged
Converged storage is a familiar industry concept. Although most storage continues to be deployed “non-converged,” adoption of converged infrastructure has been growing steadily over the past 5 years. The idea is simple: Users buy pre-integrated reference architecture, which the vendor ships to them. The architecture consolidates components into a highly redundant storage platform that users can scale out as needed. It is a solution built around capacity, performance and optimization.
Hyper-converged has recently emerged as the logical successor of converged. It takes the concept a step further, utilizing a pre-manufactured appliance that covers the capabilities of multiple functionalities. The appliance plugs into the network and covers the user’s needs, thereby simplifying the process. It is convenient for users because they get the resources they need with fewer moving parts. This user-friendly storage method also offers the efficiency of tight integration.
However, like any other solution, hyper-converged has drawbacks. The platform’s linear scaling model presents a potential issue because scaling to hundreds or thousands of appliances has not yet been explored outside of major cloud providers. Furthermore, the solution does not allow for customization according to individual needs. If a user needs additional storage but does not require additional compute functionalities, it still must pay for both. To put it simply, it’s akin to buying extra shoes, but really all you needed was additional shoelaces. Costs can quickly add up if one need continuously outpaces the other.
Despite the challenges associated with converged and hyper-converged, IDC predicts the integrated systems market will grow to $14.3 billion by 2017. Adoption rates are currently low, however, due in part to the buzz surrounding other solutions such as software defined storage.
Software-defined storage is a hot topic right now because of what it promises to deliver in the future – a platform for deploying applications based on automating the infrastructure underneath. This model implements a layer of virtual networks and policies on top of the physical network, which ultimately eliminates the need to configure hardware. The ease of use makes this method attractive to users; overlay solutions typically come prepackaged, with much of the required programming already configured in the controller.
Companies choose a software-defined approach when they wish to accomplish some or all of the following goals: building a storage solution at a low price point, establishing a solution specifically geared toward virtualized workloads and making their infrastructure programmable.
The last reason is a critical differentiator. Companies that are not prepared for a programmable infrastructure should take heed, as they are most likely not yet ready for a software-defined storage solution.
Software-defined networking has already demonstrated strong potential to become an essential component of future cloud-enabled, orchestrated data centers. Still, organizations are wise to take a wait-and-see approach, as software-defined networking standards, architectures and solutions continue to mature. It is best to track developments and products before taking the leap.
Cloud is perhaps the most well-known modern storage solution. In fact, public IT cloud spending is on track to exceed $127 billion in 2018, according to a 2014 forecast from IDC.
Users find the cloud appealing for two main reasons. First, the cloud helps them circumvent short-term costs. Secondly, and perhaps most important given the massive amounts of data companies collect in the digital era, it offers the ability to add data continuously without worrying about a cap, because space is unlimited.
Organizations typically use the cloud for one of three purposes, with backing up data to the cloud is perhaps the most common reason. While users should remain wary of migrating sensitive data, private clouds are easing some of the concern. It is also widely used for archiving data; this is particularly valuable in industries like healthcare where practices and hospitals must save certain information for legal and regulatory compliance. File sharing is the third popular use for the cloud. Google spreadsheets and programs like Dropbox have gained popularity in the past few years.
While companies are adopting the cloud at a rapid pace, the vast majority shy away from an all-cloud model. Instead, they opt for hybrid cloud – a blend of on-premises, private cloud and public cloud services with orchestration between the two platforms. Businesses like hybrid-cloud models because it offers flexibility by allowing workloads to move between private and public clouds as computing needs and costs change.
Finding the Right Fit
With all of the storage solutions available, it is wise for enterprises to make a decision based on their own needs. Leaders should focus on which model, or pairing of models, would best suit their organization and drive business – rather than simply reviewing their capabilities at face value. The process may prove lengthy and complex, but the investment will pay off in the long run.
The future of data storage will likely follow an approach that provides greater agility and enhanced flexibility, and allows for continuous growth to maximize long-term value.
Regardless of emerging and future solutions, challenges will always accompany transformational shifts in data storage. But with an accurate roadmap and the right gear, organizations can navigate even the rockiest terrain.