"So, I think it's an optimization problem that doesn't get better results for deeper networks, not a model design problem, or a model capability problem. The model itself has more potential, it's just that the way it is optimized needs to change. ”

"And that's what depth residuals are all about."

"For any of the mappings H(x) that we need to learn, I expect the network to learn F(x) + x = F(x) in H(x), rather than learning H(x) directly."

"This can be done simply by adding an addition to the distance H(x) where the difference is F(x) of x itself, which we call the residual mapping about identity."

"If this identity is ideal, then it's easy to set the weight to a very small value. The form of residuals solves the gradient problem that has been a problem for a long time, and it can be seen that it enables hundreds of layers of networks to achieve better performance consistently. ”

"The results achieved in the recognition and classification of this competition are only the most basic embodiment of the idea of residuals, in fact, it is a better feature extractor, which can better extract the features of the image for application to various image tasks."

"Not only the testing track of this competition, but also at the technical conference of whiteness two days ago, everyone must have seen a significant performance improvement."

Speaking of this, Meng Fanqi paused, because the audience had begun to talk uncontrollably.

Although Baidu announced that the real-time detection method was mainly contributed by Meng Fanqi, a special researcher, he was tight-lipped about the details of the specific algorithm and did not mention anything.

It's just a promise that it will be announced in 6-12 months.

Although everyone has probably guessed it, listening to Meng Fanqi himself, everyone is sure of at least one thing, that is, the real-time detection algorithm that whiteness is now leading the world by a large margin, and it uses DreamNet.

"Of course, there are a lot of visual tasks, in addition to the generation, detection, segmentation, and recognition that I have already done, there are also pose estimation, depth estimation, super-resolution, and so on. There are so many variety. ”

"My DreamNet paper has been published, the code has been open sourced, and there are still many directions that everyone needs to explore."

Meng Fanqi ate the cakes of the largest tracks, so naturally he had to leave some soup for others.

In a more specific direction, it is enough to have a few very recognized masterpieces.

Meng Fanqi has so many technologies that he can't finish it, so there is no need to do it one by one in all the subdivisions.

To put it bluntly, it is to change the things that make the machine run slightly, change the data, and adjust the individual structures and parameters slightly.

It is more cost-effective to open source the code and let more and more work be done based on your own technology and algorithms.

Meng Fanqi said this, in fact, it only took ten minutes in total.

According to the original plan, he could talk for about 25-30 minutes.

It's a pity that the waist bag is bulging, the temper is hard, and the mentality has changed. Meng Fanqi no longer has the kind of psychological needs that need to be recognized and recognized by the academic community before meeting Li Yanhong.

In retrospect, before Li Yanhong's advance was credited to the account, Meng Fanqi had always been a little worried, and always felt a little uneasy in his heart, hoping to be recognized.

Some are skeptical about how many resources they can leverage.

Now, these are all in the past, and there is no need to be recognized, and Meng Fanqi's display has become much more concise.

Ten minutes is actually not a short time on this occasion, especially since this exhibition has two parts, and there is also a theoretical explanation of Han Ci after Meng Fanqi is finished.

Therefore, everyone present didn't feel abnormal, only Han Ci was dumbfounded, what's the situation, shouldn't there be ten more minutes? Why is this coming to me?

"When I was doing these studies, I received a lot of help from a professor of mathematics at our school, Fu Deqing, who was a co-author of the paper, but not a scholar in our field, so he did not want to participate in this conference.

Regarding why the idea of residuals is effective, and what is its actual significance, we ask Professor Fu's sister, Han Ci, to bring us her views and explain from the perspective of the dynamical system. ”

After Meng Fanqi finished speaking, he was about to go off the stage, took two steps and turned back, and added to the microphone.

"By the way, since I signed a contract with Google, considering that I am still studying in an undergraduate degree, I urgently need a university near Silicon Valley to take me in."

"I hope Stanford can be a little more self-conscious."

After speaking, the audience burst into laughter.

The participating teams are Microsoft, UC Berkeley, St. Petersburg, IBM, Tokyo, National Singapore, Oxford, and Toronto.

The level of participants is not low, most of them are holders of master's and doctoral degrees from world-class top universities and technology giants.

The results of these people were completely blown up a few streets, and people gathered here to listen to Meng Fanqi's introduction of his algorithm, which made many people present who did not know the inside story completely ignore Meng Fanqi's identity as an undergraduate student today.

In terms of the quality and standard of the papers he has published, the doctoral graduation criteria are sufficient.

Network structure, generation, segmentation, optimizers, and normalization methods are visible to anyone with a discerning eye, and these ideas will become the foundational paradigm of the new AI era.

There are even two or three new directions on the foundation laying, digging pit type of mountain opening work.

No one thought at all that he still had such a problem to solve.

After Meng Fanqi said this, Li Feifei, the data organizer of the event and one of the instructors of the Stanford AI Lab, directly opened the on-site enrollment.

WHEN SHE COLLECTED IMAGENET DATA A FEW YEARS AGO, BECAUSE THERE WAS TOO MUCH GAP BETWEEN THE ALGORITHM'S ABILITY AND THE HUMAN LEVEL, LI FEIFEI ALWAYS HOPED THAT ONE DAY AI ALGORITHMS COULD SURPASS HUMANS IN THE LARGE-SCALE DATA SHE COLLECTED.

She thought it would take a decade or two, but she didn't expect it to take less than five or six years.

Especially in the past two years, the accuracy has been blown by a total of 20%, which directly fulfilled her wish.

Now that he has got what he wants, and the person who has done it happens to be seeking the opportunity to study at Stanford, Li Feifei will naturally not let it go.

Moreover, in 2014, Stanford was preparing to start offering courses in the direction of deep learning, and the addition of Meng Fanqi was also greatly beneficial to this matter.

Li Feifei thought so, and didn't feel that his idea seemed a little strange, recruiting an undergraduate, but thinking of asking him to assist the university in providing the quality of courses.

I really don't know if it's an undergraduate or a lecturer, which seems a little absurd.

There were many professors and scholars from Oxford, Cambridge, MIT, Harvard and other famous universities, and when they heard that he wanted to study in the United States, they all wanted to solicit him.

But when I heard Meng Fanqi say that he signed a contract with Google, and named Stanford to be a little conscious, several old professors are still some burdened people.

Seeing Li Feifei and Meng Fanqi talking and laughing, he was embarrassed to step forward and interject.