Navigating compliance while your data scalesBlog
By James Wesley, Dimension Data and David Schmitt, NetApp
Hybrid storage models including managed storage are essential in ensuring compliance and scale in an age of increasing regulation
Organisations today are looking towards innovative ways of managing the massive amounts of data being generated, data that has the potential to create massive business value. With systems collecting increasing amounts of customer data, and new technologies such as the Internet of Things adding to this, organisations need to take a hybrid approach to the management of this data.
At the same time as they’re facing an explosion in the amount of data needing to be managed and analysed, organisations have to ensure they comply with a number of different regulations which vary across continents and countries affecting how they store, manage, and process customer data. This in particular is a huge burden for organisations operating at global scale. Balancing these challenges drives many companies to consider managed storage as a key element in handling their data.
The implementation of the General Data Protection Regulation (GDPR) in Europe this year is just the most recent regulation to come into force, and its global implications have had companies across the world relooking at how they manage their data. There are, however, regulations from across the world that impact every company doing business in those territories. The risk posed by non-compliance is significant, both financially and to the business brand value, the increased focus on how organisations manage their data provides an opportunity for them to relook at how they cope with the data explosion.
The enormous amounts of unstructured data pouring into businesses - whether machine or user-generated – puts organisations at risk of becoming overwhelmed. They need the capacity to collect, store and analyse the data efficiently, and turn it into an asset that can be leveraged and used to deliver business value. This involves understanding what data has been collected, prioritising the most valuable data and ensuring that its stored in such a way that it’s immediately accessible when needed.
While ensuring that critical information is always available, this has to be done in a secure and compliant manner. The difficulty that many organisations face is that they simply don’t have the skills internally to deal with all these issues, and given that the complexity of managing data is only going to increase, the role of trusted partners will become increasingly vital. Having access to data best practises through partnerships with industry experts may prove to be the deciding factor in how successfully organisations are able to leverage their business data. Best practices need to focus more on the management processes to ensure that tasks are highly automated but always through enforcement of the rules associate with that data.
Organisations are looking to service providers to step up and assist them in managing not just how they deal with the data they are collecting, but also in ensuring that it all happens in line with the applicable regulations. A highly automated environment also provides audit ready records for all changes to the data records.
What has become clear is that organisations will have to adopt a hybrid strategy when dealing with the management of data. The old ways of storing data are simply not sustainable going forward. While some organisations are currently able to exclusively use on-premise storage infrastructure this will ultimately become impossible due to the scale and automation large data management will require. Ultimately organisations will adopt a hybrid model, with a consistent management approach.
It’s in this environment that managed storage comes into its own. In the hybrid environment organisations need to be able to seamlessly manage their data based on an automated set of rules that determine how data is managed throughout its lifecycle. While some of these rules will be determined by the need to comply with regulations, including GDPR and individual countries laws surrounding data sovereignty, where data is located is just as important both from a regulation and ability to extract value perspective.
With automation it’s possible to move information to the most appropriate location depending on where in the lifecycle the data is. Managed storage forms a critical part of this equation as it provides the flexibility for organisations to scale up their storage requirements in a seamless and cost-effective manner.
Manages storage services offer the ability to differentiate hot from cold data and provide a long-term archiving capability for rarely-used data, in addition this ensures compliance with regulations needed to keep records. This can be a combination of on-premise and off-premise, in both private and public clouds. Depending on organisation’s requirements it can use shared or dedicated platforms and scale to demand. This, with a flexible commercial approach to charge by usage, makes it easy for organisations to consume storage as a service rather than investing in infrastructure.
Finally, security is an important aspect of creating a hybrid data management environment, and organisations need a trusted advisor and capable service provider who can ensure that data is not just managed, but also highly secured in transport and storage environments.
Managed storage is not at odds with the issues of data sovereignty and privacy, as it’s possible to store confidential information in the cloud as long as the service provider is able to ensure that the relevant regulations are complied with and can produce audit ready records of compliance.
Information has become the lifeblood of the modern organisation and controlling all aspects of it is a critical task for any IT team. Adopting a data-centric approach allows organisations to effectively manage their data, automating the life-cycle of the data, and ensuring that they comply with all current and future regulations.
For any IT team struggling to manage the growth of data they should:
- Create models to map the lifecycle of data within the organisation.
- Define policies and rules throughout the lifecycle of that data.
- Find ways to automate the management of their data.
- Make use of managed services to leverage best practise around creating and managing storage requirements.
- Partner with experts where you are missing skills.
- Build out a hybrid model to ensure that the data is accessible.
- Design your storage architecture so it can scale up the cloud where appropriate.