Christina Halliday

Nov 13, 2014

Big Data Meets the Internet Of Things – Evolution or Train Wreck?


As if IT departments weren’t busy enough dealing with security issues, users, BYOD, rapid requests for new applications and everything else, now they have two more worries: the Internet of Things and the rapid rise of big data. The two are connected. After all, the Internet of Things isn’t just expected to bring a whole wave of new and impressive gadgets. It’s bringing with it a veritable mountain of data, one large enough to make us wonder if we should consider renaming “big data” to “huge data.”

And if you think you have some time before the Internet of Things affects your business, I’d urge you to think again. The deluge of data is at our doorsteps, needing to be collected, mined, organized and reported. Take the recent announcements from Apple on wearables and Apple Pay, for example. Entering the realm of mobile payment systems is a major move for Apple. They’re making a statement, a statement saying “The devices that comprise the Internet of Things are important, but so are the apps that make that world tick.” Certainly, collecting all of this data, organizing it, retrieving it and presenting it will be a “huge data” project at Apple.

To paraphrase an old saying, “As Apple goes, so goes the rest of the industry.” We’ve been on this Internet of Things track for some time, and the surge is coming.  Even more devices and apps will continue to connect and build this network of ‘things’ to do even more tasks. The business case for doing so is clear and adoption is sure to follow, but there’s a problem: That’s a whole lot of data and a whole lot of processing; something needs to handle it, somehow.

Internet of Things: The process and challenges

The challenges of processing the data that will come from the Internet of Things are made up of a trio of steps. (Stop me if this sounds familiar.) The first is ingesting data or data harvesting. Often times the amount and variety of data collection points are far and wide. The next step is storing data, with all its security, volume and archiving implications. Lastly, all of this data needs to be processed and analyzed. This last step is where it all pays off for businesses, where the value is realized, but it’s all part of a larger ecosystem of data.

Once we look at the Internet of Things from this angle, we realize that the Internet of Things isn’t a coming phenomenon of big data, but a case study of it.

Unfortunately in far too many cases, organizations have fallen short just getting past the data collection phase. Quite often they lack the right service and infrastructure to manage it.

Enter Hybrid Cloud

hybrid-iconAs we’ve covered before, Hybrid cloud is the natural answer to big data. It’s the answer to the challenges of service and value that most people need. And, surprise, surprise, it’s also the answer to the Internet of Things.

A Hybrid cloud allows the matching of the right applications to the right underlying IT infrastructure with no bottlenecks. A horizontal scalable “bare metal” (AKA Dedicated Servers) infrastructure is needed for the storing and crunching of the deluge of data that is coming with the Internet of Things. However, the “presentation” layer of the data is best suited for a platform that can scale with demand (AKA Public Cloud).  Only an “On-Demand” Hybrid Cloud can offer that. Not a monolithic public Cloud ala AWS, monolithic Dedicated infrastructure or collocation.

Big Data: Pushing our thinking

The Internet of Things introduces challenges that companies will need to overcome in order to see the benefits of integrating it to their networks. The beginning of said challenges being the need to find the right architecture to run it on.

Today, the demand is great for data scientists and agents that can help strategize the usage of better data. That will only grow. And IT organizations don’t deserve to be derailed by the fallout from using an insufficient architecture to support big data or the Internet of Things. In short, both big data and the Internet of Things are pushing companies to upgrade their mindset and technology infrastructure, or risk getting left behind.


Source: 3Vs (volume, variety and velocity)

Looking again at the ‘three V’s’ of big data – volume, variety and velocity – I can’t help but notice its parallels to the Internet of Things. The demands for infrastructure, for flexibility, and for power are high. The demand for storage alone needs to be flexible, capable and possibly on-demand.  The demand for high-processing capable performance systems in big data/the Internet of Things is analogous in the hybrid stack to dedicated systems.  The demand for collections and flexibility on a high performance network, whether for big data, the Internet of Things or both, are best solved by a hybrid cloud solution because hybrid cloud is designed with this adaptability in mind.

The opportunity in the Internet of Things

Ultimately, whether it is wearables, NEST like technologies or Apple Pay that breaks the dam of the Internet of Things, or whether it evolves or perhaps it is something else, the signs are here and the data is coming. What the Internet of Things provides in value is immeasurable; it’s gathering a never-ending stream of accurate, valuable data from the real world from devices all around us. But that value can only come from properly collecting the data, storing it and transferring it to systems that can produce the analytics in question. This conversion of data to information to knowledge is the Internet of Things opportunity, and the best way to get there is to place it on a hybrid cloud solution.


Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedIn