The Future of Data Warehousing

In the late 1980’s the IBM team of Barry Devlin and Paul Murphy published the article An architecture for a business and information system where they introduce the term “business data warehouse”. This promoted the consolidation of cleansed data into one repository to remove redundancy and allow centralized reporting.

Shortly thereafter, Ralph Kimball and Bill Inmom caused great debate on whether it was better to have a normalized data model vs. a dimensional design approach.

Recently, the biggest topic in data warehousing has been centered around the best approach to handling unstructured data.  With different kinds of data structures, enterprises are adopting a variety of systems to manage the influx of data, resulting in complex and cumbersome infrastructures. In the past, there was a variance between structured and semi structured data. Historically it meant multiple on-premises storage systems to house both sets of data. But not anymore! Now structured and semi structured data can be managed in the same system, making it easier to store, analyze and consume.

The Future

Data Warehouses today are becoming more robust with simplified architecture and support. Several platforms help to make it easy for enterprises to migrate their data to the cloud as well as provide the ability to run analytics from anywhere. Combining structured and semi structured data into one warehouse that is cloud accessible makes this revolutionary. This system provides the end-user with a faster, easier to use and more flexible system than traditional warehouse offerings.

Enterprises have multiple options for adopting cloud based data warehousing systems. While most have the capability to store big data sets, there are distinct differences between the players. Below, we will provide a brief synopsis of the leading providers in the marketplace today.

Amazon Redshift

Providing considerable synergies with AWS, Redshift delivers large scale data warehouse service for use with business intelligence tools.  Redshift is a cost-effective solution that will allow your organization to deploy a new data warehouse in minutes. This DW service uses machines, allowing you to query your data across your data warehouse and data lake, together, with a single service.

Snowflake (2012)

Snowflake is focused almost exclusively on cloud data warehousing. It has been rated as the easiest platform to set-up and easiest to use. Business users can self service getting access to all of their data and insights.  With Snowflake, you can easily scale up, down or out automatically and without disruption.

Google BigQuery

With Google BigQuery, there is no infrastructure to manage, removing the headache of managing dataware warehouse capacity.  This DW option provides a real-time capability to deliver insights using “blazing-fast SQL queries”.  BigQuery also provides machine learning to allow business analysts to better forecast and predict outcomes.

If you are ready to modernize your data warehouse and create a scalable and affordable model, contact our experienced delivery team today