From Big Data to Big Intelligence: Six Key Trends Shaping IT for 2018

Topics in this article

The turn of the calendar always brings the opportunity to both reflect and look forward. Technology continues to accelerate and the level of digital disruption continues to expand. Now is a good time to consider what’s happened in the last 365 days and contemplate what’s likely to happen next in this digital technology ecosystem that continues to be core to almost every business with new technical capabilities that bring us both unexpected opportunities and threats.

Dell Technologies recently issued its predictions for 2018 – and in looking back, many of Dell Technologies’ 2017 predictions are coming to fruition and evolving rapidly – from the New Media Evolution to Deep Learning and finally IoT sparking demand for a new level of accountability.  We at Dell Technologies even announced our own IoT division with VMware CTO, and now Dell Technologies’ IoT GM, Ray O’Farrell at the helm to drive a comprehensive and coordinated strategy – in close partnership with leading edge customers and solutions providers.

Some of the most exciting emerging technologies of 2017 were in the general use of artificial intelligence (AI) and machine learning – rapidly transforming data into actionable insights for many leading edge organizations.  Approximately 1,500 companies in North America alone have initiated projects related to AI – with an estimated 71% of enterprise applications expected to leverage AI by the year 2021 (IDC).  AI projects tend to be at the core of business systems and processes. All of that AI is powered by data, delivered via compute, and connected with networks that drive a new need for enhanced data protection and end-point security strategies.

There are a number of important trends that are being shaped by the convergence of data, advances in processing and economies of scale in the data center that we’ll see this year – below are my top six to watch.

Big Data Becomes Big Intelligence and Big Business

Extracting business value from large data sets is certainly not a new trend this year, but the sources of that data are both growing and diversifying significantly. Collecting, consolidating and analyzing data from the “edge” is critical, stemming from billions (and eventually trillions) of devices, apps and systems.  Figuring how to make use of the flood of this data will be a key focus for many enterprises this year.

Relatedly, we’ll see data unleash new possibilities in AI for business intelligence – and revenues.  IDC is predicting that by 2020, 90% of large enterprises will generate revenue from data as a service – where raw data, metrics, insights and recommendations emerge as a revenue stream and business opportunity – and subsequently fuel even greater predictability of business trends, enhance product cycles and innovation, and much more.

However, the major difference between these new AI driven systems and the past few years of big data and analytics is that more and more of the AI systems are being used in core real time business systems so they not only need to process and reason over huge data sets but they need to do that at a speed of outcome that can influence the real time flow of business. To achieve this will require the combination of storage and compute innovation.

The Data Center Is No Longer “The Center” for Data – We Need a Mega-Cloud

The explosion of data has created a highly-distributed ecosystem across increasingly decentralized data sources.  While most of the existing data processing is done in the core of the IT data centers today, as we look forward the ability to gain insights from data will spread throughout the enterprise IT topology – from the device, to the edge to the core to the public clouds. The key will be to develop architectures that allow the combined insight of the dispersed data to be used in the place where it can have the most impact and that may not always be the same area of the enterprise.

One emerging concept to better coordinate this multi-cloud world is the intelligent ‘mega cloud’ that we discussed in our 2018 prediction #1.  Many customers are finding – or will soon find – themselves struggling with silos of data that are a result of a multi-cloud environment. Data gravity – the larger the amount of data, the more applications, services and other data will be attracted to it and the more quickly those will be drawn to the data body – partitioning can be a significant impediment to gaining the in-depth insight and impact organizations seek as more and more information is captured. Additionally, issues like compliance and regulatory requirements, IP protection, and data leakage only compound the problem.

A mega cloud solves the issue of cloud silos by taking a holistic view to the collection of clouds that the enterprise interacts with to ensure that the right data is housed in the right place. It brings together multiple public and private clouds, as well as leverages data coming in from the edge and at the core. By evolving to have a single view into the multi cloud environment and creating consistency between clouds at the software, hardware, and services layers, enterprises are far more likely to be able to connect the right data sets to the best processing capability in the correct place to impact their business and make visions such as AI a reality.

The Maturation of New Media / Persistent Media

New forms of persistent media are emerging, blurring the distinction between large memory and fast storage, to tackle a new set of high-value workload requirements. These media will change the way that we view the intersection of application performance, storage density, and data placement. New media cost, capacity, and performance attributes will deliver new benefits for the most demanding workloads. New media technologies will be used as new tiers of data center storage, delivering more IOPs, lower latency and higher density. They will be used to deliver larger, more cost effective infrastructure for in memory workloads. As these technologies mature, they will enable an entire new class of workload-enhanced ‘collapsed servers’, containing fast and dense new media, SDN and virtual security tools, and new types of specialized processors, all tuned to host high performance applications. These “Decision Servers” will enable more rapid, relevant, and impactful business insights than ever before.

2017 brought with it the rapid adoption of a new host-storage interface: NVMe. Anticipating the capabilities and performance of new media, NVMe is architected to bring storage media directly onto the PCIe bus, closer to the processor, to reduce SW overhead and to exploit the advances of multi-core processors for server and storage system applications. While initially used with flash storage today, NVMe is expected to be the preferred SSD interface for low latency storage class memory (SCM) technologies. Finally, with the emergence of NVMe-OverFabrics, the benefits of NVMe will be extended to seamlessly operate across lossless, low-latency data center fabrics such as Ethernet, Fibre Channel, and InfiniBand.

IT Infrastructure Accelerates Its IQ for AI and IoT

With enterprise business requirements quickly evolving to deliver the benefits of data and AI, new requirements for IT infrastructure are emerging to ensure that this new distributed data environment is effectively monitored, managed, stored, secured and protected. It must be delivered cost effectively, within already tight IT budgets – and it must achieve demanding, sub-millisecond response times. This means that intelligence and efficiency must co-exist at the edge, core and cloud, driving server, storage and data protection design requirements that fully embrace new interconnect models like NVMe, diverse processing capabilities like FPGAs and GPUs and the growth of high speed in-memory data management workloads.

Embedded analytics and automation, enabled by machine learning, will become a default requirement for dynamic, consumption-based application and IT infrastructure provisioning. As I put it in our 2018 predictions, AI will do the “thinking tasks” at speed.  The “brains” of the IT infrastructure will evolve to quickly and efficiently recognize, analyze and label data, know what data goes where, identify how it needs to be stored and accessed in the future, and decide where it needs to live specifically. The payoff? Optimal costs, speed and efficiency for tactical IT functions, freeing up time, talent, and budget for more strategic projects that maximize an organization’s ability to take advantage of all that data.

Organizations Adopt a Security Culture

As a colleague recently stated, there are an average of 4,000 ransomware attacks per day with hackers gaining intelligence and skill with every attack. Protecting and securing our most valuable data is becoming increasingly complex as the way we capture, store and draw insight from that data becomes increasingly sophisticated. It’s not sufficient to just have multiple copies of data in different places. You must assure that the copies are uncompromised, protected and can endure even in the most catastrophic of events. This has led to new technologies such as the Dell Isolated Recovery Solution (IRS) that blends data protection and content security in a way that has now been shown to survive some of the most destructive attacks ever seen.

Additionally, the continued and expanding deperimeterization has changed the ways in which we need to view security. We expect to see the complexity of application placement become a foundational focus. Applications generate data that enterprises must be able to intelligently place and intelligently protect. Employing ‘security chains’ via Software Defined Networking technologies, such as NSX, associate application components as they are deployed with security capabilities logically and dynamically. This is rapidly becoming the primary way to lower risks while increasing visibility, creating a truly a new world of data security.

Organizations can’t simply make security a solution – they’ll need to deeply integrate it as part of their corporate culture where everyone, CEOs included, need to be invested in protecting the crown jewels. Further, consumers and customers are becoming increasingly vigilant, putting their trust and ultimately their business and loyalty with the organizations that prove they’re able to stay one step ahead of cyber attackers and malware. As a result, we’ll see progress in security transformation this year that shores up end-point security across a diverse and distributed mobile workforce and IT infrastructure.

IoT Takes Enterprise IT and Digital Strategy to the Mat: Transform or It’s a TKO

In 2015, the IoT market reached $900 million, and it’s predicted to grow to $3.7 billion by 2020 – only a little over two years away.  IoT forms the basis for new decision making and directional shifts within the business, and allows companies to remain competitive in nearly every market – healthcare, agriculture, automotive, banking, education, manufacturing, and many other industries. It will require new investments across the board but particularly in the evolution (or potentially revolution) of IT infrastructure.

The data generated by IoT will quickly exceed the capacity of traditional data centers.  To digitally transform, customers will rely on edge, core, and cloud components, from hyperscale data centers to optimized servers that can handle large and dynamic workloads at the edge. Ashley Gorakhpurwalla, my colleague and head of the Dell Server Division, recently discussed the notion that customers are looking to deploy accelerator-optimized server architectures, with GPU and FPGA processing at the edge in order to handle the demands of an IoT-centric environment.

IoT clearly changes the storage game as well. The 451Group recently conducted a survey where respondents said they expect to increase storage capacity by an average of 35.9 percent over the next year due to IoT activity, with security concerns being central to IoT strategies and deployment plans. As discussed in #4 above, innovation in the data center plays a critical role in developing a holistic, intelligent IT infrastructure that maximizes the power of the IoT information captured.

Last but never least, IoT is driving the need for mission-critical services at the network edge for emergency services, real-time communications for autonomous vehicles (Vehicle-to-Vehicle, and Vehicle-to-Infrastructure), and the tactile Internet (drones, robotics, etc.) – they all require a level of real-time, two-way communication.  A new battle for the network edge among cloud and service providers will commence with a focus on “fog” computing that exists between the edge and the cloud.

2018 is going to be yet another disruptive year in the world of emerging technologies. These six predictions tell a story driven by data – how it’s managed, stored, processed, protected and exploited – to reduce risk, derive value, and create new, meaningful opportunities. Data is the key.

To read more of what the innovative minds here at Dell Technologies are anticipating for 2018 – check out our 2018 Predictions at our Perspectives blog.

About the Author: John Roese

Topics in this article