Big data, or giant data, massive data, big data, refers to the amount of data involved in the scale is so huge that it can not be manually, in a reasonable period of time to achieve the interception, management, "big data" in the Internet industry refers to a phenomenon: the Internet company In the Internet industry, "Big Data" refers to a phenomenon in which Internet companies generate and accumulate data on users' network behavior in their daily operations. The scale of this data is so large that it can't be measured in G or T.
Big Data in China
Characteristics of Big Data:
Specifically, Big Data has four basic characteristics:
First, the volume of data is huge. Baidu information shows that its new home page navigation needs to provide more than
1,5PB (1PB = 1024TB) per day, these data if printed out will be more than 500 billion pieces of A4 paper. There is information to confirm that, so far, the human production of all printed materials, the amount of data is only 200PB.
Deduction of history in just 133 days
Secondly, the data type is diverse. The type of data is now not only in text form,
but also in multiple types of data such as images, video, audio, geolocation information, etc., with personalized data making up the absolute majority.
Third, the processing speed is fast.
Data processing follows the "1-second rule", which allows us to quickly obtain high-value information from various types of data. Fourth, low value density. In the case of video, for example, an hour's worth of video, in the course of uninterrupted monitoring, may be useful for only one or two seconds of data.
Useful data is only one in 3600 facing the arrival of the big data era, are you ready?
The era of big data
The earliest proposal of the era of "big data" is the world-renowned consulting firm McKinsey and Company, McKinsey said: "Data, which has permeated every industry and business function today, has become an important factor in production. The mining and utilization of massive amounts of data heralds a new wave of productivity growth and consumer surplus."
As we enter 20XX, the term bigdata is increasingly being mentioned to describe and define the massive amounts of data generated in the age of the information explosion, and to name the technological developments and innovations associated with it. It has been on the New York Times, Wall Street Journal column cover, into the U.S. White House news, appearing in some domestic Internet-themed lectures salon, and even by the sensitive sense of smell of the National Golden Securities, Guotai Junan, Galaxy Securities, etc. written into the investment recommendation report.
2, the data is rapidly expanding and getting bigger, it determines the future development of the enterprise, although many companies may not realize the pitfalls of the explosive growth of data to bring problems, but with the passage of time, people will be more and more aware of the importance of data to the enterprise. As a February 20XX column in The New York Times put it, the era of "big data" is upon us, and decisions in business, the economy, and beyond will increasingly be based on data and analysis rather than experience and intuition. Gary King, a professor of sociology at Harvard University, said, "It's a revolution, and the sheer volume of data resources is making it possible to begin the process of quantification in all fields, whether it's in academia, business, or government, all of them are going to begin this process."
Examples of big data applications:
1, the medical industry
In a hospital in Toronto, Canada, for premature babies, there are more than 3,000 data readings per second. By analyzing this data, the hospital is able to know in advance which preterm babies have problems and take targeted measures to avoid premature babies from dying.
2, the energy industry, the smart grid has now been done in Europe to the terminal, the so-called smart meter. In Germany, in order to encourage the use of solar energy, will install solar energy in the home, in addition to selling electricity to you, when your solar energy has excess electricity can also be bought back. Data is collected through the grid every five or ten minutes, and this data collected can be used to predict the customer's electricity habits, etc., so that it can be deduced how much electricity the entire grid will probably need in the next two to three months time. With this prediction, a certain amount of electricity can be purchased from the generating or supplying company. Because electricity is a bit like futures, if you buy in advance will be cheaper, buy spot is more expensive. With this forecast, the purchase cost can be reduced.
3, the communications industry
Telecommunications industry through tens of millions of customer data, can analyze a variety of user behavior and trends, sold to the need of the enterprise, which is a new information economy. China Mobile through the big data analysis, the enterprise operation of the whole business of targeted monitoring, early warning, tracking. The system automatically captures market changes in the first time, and then pushed to the designated person in charge in the quickest way, so that he was informed of the market situation in the shortest possible time, combined with their own experience, the mobile launch of the night traffic package.
4. Retail
Retail companies also monitor customers' in-store walks and interactions with merchandise. They combine this data with transaction records to analyze and advise on what to sell, how to display the goods, and when to adjust the selling price. This approach has helped one retailer reduce inventory by 17%, while increasing the proportion of high-margin private label goods while maintaining market share.
Sample three-minute speech on big data (2)
Since we're talking about big data, it's important to understand the definition of big data. What is big data. Imagine what will happen to the world if in the future one of us owns more computer equipment than the sum of the world's current computing power, one person generates more data than the sum of the world's current data volume, and even your pet puppy generates more information than the sum of the world's current data volume, what will happen to the world? So let's start by looking at the definition of big data as given by some official websites.
To derive data that is useful for a cause. There is a purposeful definition for big data business: "Using big data to solve economic and social problems. Increase the value added to business. It is not only the size of big data, but also what kind of data it consists of, and how it can be utilized. It is different from the current system. It is a term to say that it is difficult to deal with a complex collection of data in a huge way with the data base management tools that have been sold or the applications that have been used to process the data up to now.
Look at the two Chinese ones again.
So what exactly is big data. It's really just a lot of data. It is a lot of information stored in the way of data, and then continue to accumulate, until these data is so large that there is no way to use simple to carry out the utilization. Big data is not something magical. It's like the question posed by the movie Neverwhere: humans typically use only 20% of their brains, what would the world be like if the remaining 80% of their brain potential was unlocked? Less than 20% of data (or even less) is typically used effectively in the management of businesses, industries, and nations; what would the world be like if the value of the remaining 80% of data were stimulated? Especially with the new Moore's Law of massive amounts of data, what would the world be like if data exploded and then data was used more effectively? Use your imagination to think about it.
So how did Big Data come about?
Previously, we are concerned about the transaction system and business systems to generate data, through the data warehouse to analyze the display, in fact, the terminal, especially individuals a variety of water operations, such as the list of purchases, Internet browsing history, photos, microblogging and so on, but do not pay attention to the era of big data, more attention to these large amounts of data, the expectation that analyze these data to find value, so big data is actually in the system, the terminal, and the individual to find value. Big data is actually generated in systems, terminals, individuals and so on, just not utilized, and now we need to analyze these data to find value from them. This is the reason and significance of the formation of big data. More data, but if it is blocked or not used, it is also worthless. China's flights are very late, compared to the United States flight punctuality is much better. In this regard, a good practice of the U.S. air traffic control agency has played a positive role, which is very simple to say, that is, the U.S. will publish the past year's late rate and average late time of each airline and each flight, so that customers will naturally choose flights with high punctuality when purchasing air tickets, and thus the airlines will be pulled by market means to strive to improve their punctuality rate. This simple approach is more direct and effective than any management tool (e.g. the Chinese government's macro-control tools).
Let's start with this printed textbook, which gives two examples. For the Fukushima nuclear power plant thing, as well as for the convenience store advantage, are the use of big data. Supermarkets, for example, have to buy a lot of goods, and then some of them may be piled up with fewer people buying them, and even if they reduce the price to deal with them, they may not be able to sell them, thus causing losses. Convenience stores, then, will be based on the usual life around the people to selectively buy goods. If it is near an apartment for office workers, it will stock more fast food, and if it is in a place where there are a lot of elderly people or middle-aged people, it will stock household goods. In the case of the Fukushima nuclear power plant, if the information from the monitoring of the plant is collected and analyzed in time, it is possible to find out that there is a problem and that it may be leaking, and then to solve the problem to avoid damages. This is to predict possible future accidents, which can then be effectively avoided. Another example is that astronomers study the movement of celestial bodies and discover more planets. Collecting these data together is big data, analyzing and organizing these data, it is possible to calculate that a certain planet may hit the earth in the future. Then take some necessary measures to remove this devastating blow. How to recognize the present? In medicine, for example, a cell is observed through medical equipment and based on other tests it is then hypothesized whether the cell is cancerous or not. If the basis of judgment is turned into data and input into a computer, and then give her a certain learning ability, for example, give him a cancer cell, he will remember the characteristics of these cells, and over time, he will become more and more accurate. It will also become more and more obvious the faster it is useful in medicine. This was a simple understanding of his application. After that, Zhao Fei will talk about this aspect more y and more carefully.
So, what Big Data brings to us is good. No, it's not. Take, for example, the medical example. Machines are being developed that will make the staff who diagnose cancer cells pass away from their jobs. It's like the gradual automation of factories in the 1980s, and worker unemployment will
be bad. If you're a horse, then you sure don't like the industrial revolution.
Moving on to the challenge that big data faces now, it's how to utilize it for the benefit of people, for our benefit. As an example take LSST. He is a wide-area astronomical telescope that was developed by a number of countries, and is now on a mountain in Chile. 20XX began construction, and in 20XX opened Charles Simonyi and Bill Gates donated 20XX million dollars and 10 million dollars to the LSST program. the LSST program still seeks to the U.S. National Science Foundation can be allocated to nearly 400 million dollars in sponsorship fees. It can shoot the entire sky once every three days. By 20XX, it will carry a super digital camera that will take pictures with 3.2 billion pixels. It would take 1,500 high-definition television screens to display such a picture. lSST's camera takes more than 200,000 images per year than a human can analyze. This data would be a great asset if utilized properly, but there is no effective solution to this problem. There isn't that much manpower available to analyze it, and there isn't that kind of equipment to analyze it directly. So this pile of data is just useless data with no value. So big data is a very potential science. Let's watch a video of a TED talk with you next.
The speaker is Kenneth Cookye The Economist data me, formerly of the Wall Street Journal (Asia edition) and the International Herald Tribune. He is a member of the Council on Foreign Relations and one of the regular business and technology commentators on CNN, BBC and NPR. With a deep knowledge of big data, hear what he has to say about it. After watching this presentation, in have Fei Zhao bring a deeper understanding of big data.