According to the forecast of the United States, the amount of data in the world increased by 62% in 2010 compared with 2009, according to this inference, the international Internet traffic will increase by 1,000 times in 10 years, and the Internet traffic in the United States itself will also increase by 1,000 times in 10 years. This curve is the traffic of the world's backbone network, regardless of the Asian financial crisis or other crises of the Internet, the traffic is not affected, and it still maintains rapid growth. ”

"Optical fiber transmission capacity has expanded 1,000 times in ten years, and there is still a lot of room for development, and now the cost of optical fiber and cable is very low, China produces half of the world's optical fiber and cable, and also consumes half of the world's optical fiber and cable. It can be seen that in 95, the total data capacity was relatively small, only 2.5G to 1G. In 2010, it can be seen that the single wavelength of the channel has reached 100G, and in 2020, the single wavelength will reach 1T, and the total capacity will increase. ”

Everyone nodded, and the representatives of China Telecom Unicom were present, and they also nodded in approval of President Wu's words.

And Hang Yu recognized it more than the representative of China Telecom Unicom, and he also had to understand President Wu's words. Perhaps Dean Wu could not have imagined that in the near future, we will make quantum computers, quantum satellites, quantum communications, and enter the 5G era.

"In the past, no one was talking about cloud computing, in the 80s they talked about databases, in the 90s they talked about IBC, and now they talk about cloud computing. In fact, cloud computing should be more accurate cloud services, of course, what will it develop in the future? There is an infrastructure at the bottom of cloud computing, like many of our enterprises host their databases to operators, which uses cloud computing, which has data centers, storage, and servers, if this is not enough, for operators, it is nothing more than "digital real estate". ”

Dean Wu continued: "Operators hope to further add development tools called PaaS, which can provide JAVA, eb2.0 development tools, middleware, etc., and enterprises can rent these development tools to develop some software required by enterprises, such as data mining, etc. For some small businesses, there is no development capacity at all, so simply rent your software directly, which is SaaS. ”

"For example, when we talk about big data analysis now, every company wants big data analysis, but it is not cost-effective for every enterprise to buy these data analysis software, so renting third-party analysis software may be a direction. Of course, there is Business further, and capable enterprises can develop more on it. Therefore, cloud computing was not born for big data, but cloud computing just adapts to the needs of big data. ”

"Big data technology involves data collection, data storage, data computing, data mining, data presentation, data security, etc., involving many links. Mining for example, requires cleaning, merging, compressing, and formatting, followed by statistical analysis, knowledge discovery, and visualization. Then find out its association rules, classification, clustering, arrangement, and optimization paths. There is a lot of data mining software involved here. ”

"To put it simply, first of all, MapReducers, there are a lot of data on the left diagram, different colors represent different types, first of all, these data are classified through Map, and the data of different business types are divided into different storage servers, so that in order to simplify the operation, the data should be labeled in the classification process, and the duplicate should be removed, which is some operations before the pre-analysis of big data. In addition, big data requires a lot of servers. ”

"Some people think it's reliable to buy high-end servers, but as far as I know, Jiangyan uses low-end servers. Mr. Hang, is your company's Weibo cloud reliable and how is its performance?" Dean Wu looked at Hang Yu again.

Hang Yu took the microphone, stood up and said: "I can answer you with certainty, our company's Weibo cloud is absolutely reliable, both in terms of performance and security, and it is no different from those who use high-end servers." ”

Dean Wu then asked, "The performance of the low-end server is not good, how do you turn decay into magic?"

Regarding the question of whether to use high-end or low-end servers, when the data center was built, Jiangyan also set off relevant technical discussions within the company, and finally Hang Yu decided to use low-end servers.

First, at that time, the company's capital was not strong enough, and it used low-end servers to save money. Second, Hang Yu has experienced the era of big data and knows that many large enterprises in the future will use low-end servers.

Hang Yu didn't know how they did it, but this did not prevent him from making a decision, he said, the technical department clarified the goal, began to study related technologies, and the result was of course successful.

"This question is too professional, and I want Professor Guan from our company to answer it. Professor Kwan, the chief engineer who developed the Basnake system, knows the specific technical issues better than me. Hang Yu gave the microphone to Guan Yonglin.

"When it comes to the choice of low-end servers and high-end servers, in fact, we were also forced to be helpless at that time, because the chairman said that the funds were difficult and refused to approve the money, so we could only retreat to the second. Guan Yonglin stood up and said.

Everyone smiled when they heard this, and felt that their development story was quite interesting.

"To solve this problem, we need to use distributed storage and redundant configuration technology. As we all know, redundant configuration is to copy one data into three servers, and the price of three low-end servers is still cheaper than one high-end server, which improves reliability and reduces costs. Guan Yonglin briefly introduced.

"Thank you, Professor Kwan, for your answer, let me add. Dean Wu said: "Big data is different from the analysis of the past, which is stored in a static database and then analyzed. And big data is there all the time, for example, a few milliseconds to send a piece of data, aircraft engines are also constantly sending data, data does not stop at all.

"We can't wait for the data to stop and then analyze it, we have to analyze it as we go, what should we do? In the past, the analysis was static, called "bringing data into the program", but now the analysis is active, that is, "bringing the program into the data". Therefore, big data analysis will also bring great challenges. ”

"Another difficult challenge is unstructured data. The so-called structuring means that it can be expressed in a text table, etc., even if the text table expression is still difficult to understand from the semantic meaning. For example, during the earthquake, in order to monitor public opinion on the Internet to see whether there were more positive or negative comments, there was a message that said, "When he found out that his son was still alive, he hugged his head and cried." According to the analysis, "crying" is definitely negative. But in reality it's positive. And that's why? It's so hard for computers to understand human feelings. If the analysis of the text is so difficult, the analysis of the photo is even more difficult, and the text must be scanned through OCR and added to the photo as a label. In January last year, Zhou Kehua killed someone in Nanjing, and when the camera took him down, the city of Nanjing called up hundreds of thousands of camera videos, and you have to watch as long as you want, and if you don't have a way to analyze it, you can rely on people to see it, so it's very slow. Therefore, big data exchange intelligent processing and intelligent analysis.

"In addition, big data needs to be virtualized and visualized. Dean Wu said: "For example, on Jiangsu Road in Shanghai, there are many cameras on the road, and there is a TV screen behind each camera, and many screens are placed on one wall of the traffic management center. Of course, no matter how big the wall is, it can't fit so many traffic cameras in Shanghai, so it can only show the cameras of a road for 10 seconds, which are all separated, and it is difficult to see the problem one by one. ”

"We hope to use the software to synthesize the cameras on this road into a video, and just watch this video to know the condition of the cameras on the whole road. Of course, it is not enough to have only one road, we also have to combine it into a map of the whole of Shanghai, just like the leaders of Shanghai are looking down on Shanghai in a helicopter, and see the entire city of Shanghai, at a certain latitude north of Tokyo, at a certain time period, which section of the road is congested. Big data, no matter how big the data is, whether it is petabytes or terabytes, the most important results should be a very intuitive picture. ”

Dean Wu's speech was relatively long, but it was not difficult to understand, nor was it boring, because he gave many examples to let everyone know some details and development trends more intuitively.