The Age of Big Data: Five Business Analytics Technology Trends

The Age of Big Data: Five Business Analytics Technology Trends

Trend centers are now no less focused on how to address analytics challenges than they are on how to make the most of opportunities in new business perspectives. For example, as more and more companies begin to have to deal with massive amounts of data and consider how to utilize it, technologies to manage and analyze large and disparate data sets are beginning to emerge. Analyzing cost and performance trends ahead of time means that companies can ask more complex questions than ever before and provide more useful information to help them run their businesses.

In interviews, CIOs summarized five IT trends that are affecting the way they do analytics. They are: the growth of big data, fast processing technology, the declining cost of IT commodities, the proliferation of mobile devices, and the growth of social media.

1. Big Data

Big data refers to very large data sets, especially those that are not neatly organized to fit into traditional data warehouses. Web spider data, social media feeds, and server logs, as well as data from supply chains, industries, neighborhoods, and surveillance sensors all make a company's data more complex than ever.

While not every company needs the technology to handle large, unstructured data sets, Perry Rotella, CIO of Verisk Analytics, believes all CIOs should be looking at big data analytics tools. 2010 revenues exceeded $1 billion.

Rotella argues that technology leaders should take the attitude that more data is better, and that they should welcome large increases in data, and that Rotella's job is to preemptively look for connections and models.

Big data is an "explosive" growth trend, according to Cynthia Nustad, chief information officer at HMS, a company whose businesses include helping to control costs in the Medicare and Medicaid programs and private cloud services. HMS is in the business of helping to control costs for Medicare and Medicaid programs and private cloud services. Its clients include health and human services programs in more than 40 states and more than 130 Medicaid managed care programs. HMS helped its clients recover $1.8 billion in losses and save billions of dollars in 2010 by stopping erroneous payments.According to Nustad, "We're collecting and tracking a lot of material, both structured and unstructured data, because you don't always know what you're going to be looking for in it."

One of the most talked-about big data technologies is Hadoop, an open-source, distributed data-processing platform originally developed for tasks such as indexing web searches. Hadoop is one of several "non-relational (NoSQL)" technologies (including CouchDB and MongoDB) that organize network-level data in a special way.

Hadoop distributes subsets of data to hundreds or thousands of servers, each of which reports results that are collated by a master job scheduler, giving it the ability to process byte-sized data. Hadoop can be used both for data preparation prior to analysis and as an analytical tool. Companies without thousands of free servers can buy on-demand access to Hadoop instances from cloud vendors such as Amazon.

Nustad said that while not for its large database of Medicare and Medicaid claims, HMS is exploring the use of NoSQL technology. It includes structured data and can be handled by traditional data warehousing techniques. She says it's not wise to start with traditional relational database management when answering the question of what relational technology is the proven best solution. However, Nustad believes that Hadoop is playing an important role in fraud prevention and waste analysis, and has the potential to analyze patient visit records reported in a variety of formats.

The CIOs interviewed who have experienced Hadoop, including Jody Mulkey, CIO of Rotella and Shopzilla, work for companies that make data services a part of their business.

Mulkey said, "We're doing things with Hadoop that we used to do with data warehouses. What's more, we're gaining access to practical and useful analytics that we've never used before." For example, as a comparison shopping site, Shopzilla accumulates terabytes of data every day. Previously, we had to sample and categorize data," he states. That's a lot of work when you're dealing with massive amounts of data." Since adopting Hadoop, Shopzilla has been able to analyze the raw data and skip much of the middle ground.

GoodSamaritan Hospital, a community hospital in southwest Indiana, is in a different category. According to Chuck Christian, the hospital's chief information officer, "We don't have what I would consider big data." Nonetheless, regulatory requirements have prompted it to store whole new types of data such as huge electronic medical records. This will certainly require them to be able to gather healthcare quality information from the data, he says. However, this will probably be accomplished at regional or national healthcare associations, not at individual hospitals like theirs. As a result, Christian may not invest in this new technology.

IslandOneResorts CIO John Ternent says its analytics challenge depends on whether it's "big" or "data" in big data. However, he is cautiously considering using Hadoop instances on the cloud as an economical way to analyze complex mortgage portfolios. The company currently manages eight timeshare resorts in Florida. According to him, "This solution has the potential to solve real problems that we are currently experiencing."

2. Accelerating the Speed of Business Analytics

Vince Kellen, chief information officer at the University of Kentucky, sees big data technology as just one element in the megatrend of rapid analytics. According to him, "What we're looking forward to is a more advanced approach to analyzing massive amounts of data." The size of the data is less important than analyzing it more quickly, "because you want the process to be done quickly."

Because current computing can process more data in memory, it can compute results faster than searching through data on a hard disk. This is still the case with this even if you are only processing a few gigabytes of data.

Despite decades of development, database performance has improved a lot by caching frequently accessed data. This technique becomes even more practical when loading entire large data sets into the memory of a server or cluster of servers, where the hard disk serves only as a backup. Because retrieving data from spinning disks is a mechanical process, it is much slower than working with data in memory.

Rotella says the analyses he performs in seconds now would have taken him an evening five years ago, and Rotella's company focuses on prospective analysis of large data sets, which often involves querying, searching for models, and tweaking before the next query. When it comes to speed of analysis, query completion time is very important. According to him, "Before, the run time was longer than the modeling time, but now the modeling time is longer than the run time."

Columnar database servers change the traditional row-and-column structure of relational databases to address another performance need. Queries access only useful columns, rather than reading entire records and selecting optional columns, which greatly improves performance for applications that organize or measure key columns.

Ternent warns that the performance benefits of columnar databases need to be coupled with proper application and query design. "In order to make the distinction, you have to ask it the right questions in the right way," he claims. At the same time, he points out that columnar databases really only make sense for applications that handle more than 500 gigabytes of data. According to him, "You have to collect one size of data before you can make a columnar database work, because it relies on a certain level of duplication to improve efficiency."

Allan Hackney, chief information officer at insurance and financial services giant John Hancock, said that to improve analytics performance, the hardware would also need to be upgraded, such as by adding GPU chips, which are the same graphics processors used in gaming systems. The calculations needed for visualization are very similar to those used in statistical analysis," he says. Graphics processors are hundreds of times faster than regular PC and server processors. Our analysts love this equipment."

3. Declining Technology Costs

As computing power grows, analytics are beginning to benefit from declining memory and storage prices. At the same time, competitive pressures are leading to further price reductions in commercial products as open source software becomes an alternative to commercial products.

Ternent is a proponent of open source software. Before joining IslandOne, Ternent was vice president of engineering at Pentaho, an open source business intelligence company. According to him, "For me, open source determines the field of involvement. Because mid-sized companies like IslandOne are able to replace SAS with the open source application R for statistical analysis."

Previously, open source tools had only basic reporting capabilities, but now they are able to provide the most advanced predictive analytics. According to him, "Open source participants are now able to span the entire continuum, which means anyone can use them."

HMS's Nustad argues that changes in the cost of computing are altering some of the infrastructural choices. For example, a traditional element of creating data warehouses has been to get the data together into servers with lots of computing power to process them. Separating analytics workloads from the operating system prevents performance degradation of day-to-day workloads when compute power is insufficient, which Nustad says is no longer an appropriate choice.

She says, "As hardware and storage get cheaper, you're able to have these operating systems handle a business intelligence layer." By reformatting data and loading it into a warehouse, analytics built directly on operational applications can provide answers more quickly.

Hackney observes that while the price/performance trend favors managing costs, these potential savings will be offset by the growing demand for capacity. Although JohnHancock's storage cost per device has dropped 2 to 3 percent this year, consumption has grown 20 percent.

4. Popularity of Mobile Devices

Like all applications, business intelligence is becoming increasingly mobile. For Nustad, mobile BI has priority because everyone wants Nustad to have in-person access to reports on whether her company is meeting service-level agreements anytime, anywhere. She also wanted to provide her company's clients with mobile access to data to help them monitor and manage healthcare overhead. It's a feature that customers really like," she says. Five years ago, clients didn't need this feature, but now they do."

For CIOs, catering to this trend is more about creating user interfaces for smartphones, tablets, and touch-screen devices than it is about more sophisticated analytics capabilities. Perhaps for that reason, Kellen finds it relatively easy. According to him, "For me, it's just small stuff."

Rotella doesn't see it as easy. He claims, "Mobile computing affects everyone. Many people are using iPads for work, while other mobile devices are exploding. This trend is accelerating and changing the way we interact with computing resources within our companies." Verisk, for example, has developed products that enable adjusters to do quick analysis in the field so they can perform replacement cost assessments. He claims, "This approach has impacted our analytics while making it readily available to everyone who needs it."

Rotella states, "What triggers this challenge is the speed at which technology is updated. Two years ago, we didn't have an iPad, and now many people are using iPads. with the emergence of multiple operating systems, we're trying to figure out how that affects our R&D so that we don't have to write these apps over and over again."

IslandOne's Ternent points out that, on the other hand, the need to create native apps for each of the mobile platforms may be fading because of the more powerful browsers now available on phones and tablets.According to Ternent, "If I can use a web-based app that's specifically for a mobile device, then I'm not sure that I'm going to be able to use it. app, then I'm not sure I'd invest in a customized mobile device app."

5. The Addition of Social Media

With the rise of social media such as Facebook and Twitter, more and more companies are looking to analyze the data generated by these sites. Newly launched analytics apps support statistical techniques such as human language processing, sentiment analysis and web analytics, which aren't part of the typical suite of business intelligence tools.

Because they are new, many social media analytics tools are available as a service. A prime example is Radian6, a software-as-a-service (SaaS) product that was recently acquired by Salesforce.com. Radian6 is a social media dashboard that displays positive and negative numbers for specific terms mentioned in TwITter messages, Facebook posts, blog and discussion board posts and comments, and, in particular, provides a vivid visual representation of brand names. Provides vivid visual inferences for trademarked names. When purchased by marketing and customer service departments, such tools are no longer heavily reliant on the IT department. For now, Kellen of the University of Kentucky remains convinced that he needs to pay close attention to them. According to him, "My job is to identify these technologies, evaluate which algorithms are right for the company based on competitiveness, and then start training the right people."

Like other companies, universities are very interested in monitoring their universities' reputations. At the same time, Kellen said he may also look for opportunities to develop applications specifically designed to address issues of concern to schools, such as Monitoring student enrollment, among other issues. For example, monitoring student posts on social media could help schools and administrators get an early indication of the trouble students are having at their universities, Kellen said, adding that Dell already does this with its product that allows the company to detect tweets about faulty laptops. He said IT developers should also look for ways to push alerts from social media analytics into apps so that companies can react quickly to incidents.

Hackney said, "We don't have the know-how or the tools to process and mine the value of massive social media posts. However, once you've collected the data, you need the ability to get enough information about company events to correlate them." While Hackney says John Hancock's efforts in this area are in their "infancy," he believes that IT departments will play an important role in correlating data provided by social analytics services on company data. For example, if social media data shows that a company's social commentary in the Midwest is getting more negative, he'll want to see if a company's pricing or strategy adjustments in that region will reverse that negative trend.

Hackney said the point of discovering such correlations is to convince company leaders that investing in social media has a high return. He claimed, "In the industry I work in, everyone is an actuary, everyone does the math, and they don't base anything on taking anything for granted."

The above is what I shared with you about the era of big data:five business analytics technology trends, more information can be concerned about the Global Green Ivy share more dry goods