Generally speaking, there are two types of more mature competition lists, one is the public list, and the other is the final private list.

The corresponding data is also different, and it is divided into a validation set and a test set.

The true answers to these two parts of the data will not be released, but after the contestants submit the results, the public list will only publish the results of the validation set for everyone's reference, and will not publish the results and content of the final test set.

This is because the real-time ranking of the list is only to help you understand what level your algorithm is and how much it is worse than the strongest algorithm. Although no direct answer is provided, if the contestant adjusts the settings and submits the results repeatedly, the content and distribution of this part of the data can still be analyzed to a certain extent.

Therefore, this part of the data is only used for reference purposes, and only the part of the test set determines the final ranking. Therefore, the public list of the event can only reflect the problem to a certain extent, and cannot indicate 100% of the final position.

"Although there are tournament lists that are not used at all for the final rankings, it is easy to be attracted by the magic of such lists." Meng Fanqi remembered that he had participated in two small competitions with Don Juan in later life, and he didn't know why he wanted to stare at that list all day long.

A higher or lower score in your submission will cause a huge fluctuation in your mood.

"Any industry is like this, what kind of novel industry do you watch, film and television drama industry, star industry. Create anxiety. Don Juan disagreed, "If there is no anxiety, make a list to create anxiety, everyone wants to be a master, as soon as this list comes out, it is like a bait sprinkled in a calm pool." The fish that had been turning their bellies and not moving were all alive. ”

"The sports circle is too, I like to say who is the GOAT (the strongest in history), what James PK Kobe, Messi vs. Ronaldo." Don Juan continued to complain that the sports circle is now suffering from this kind of atmosphere, and it is about to become a fan circle, "The statistics are becoming more and more outrageous, I used to count a goal, and now even the goal scored by the part is also counted, I saw someone say a few days ago that Ronaldo's brother scored a goal." ”

Meng Fanqi was checking the information of the submission results, and was stunned for a moment at first glance, "What about Ronaldo and his younger brother?" ”

After thinking about it carefully, I came back to the taste, and I was the second brother for a long time.

In fact, on November 11th, the submission website was announced, and the submission window for this year was very short, and it was not like many later competitions to divide the verification set, and open the public list submission during the competition.

On November 13th, the deadline for submission of results will be.

Forty or fifty days passed unconsciously, and Meng Fanqi repeatedly polished these papers several times.

Not only that, but when he later found out that the experiment in the paper had been completed, he connected the detection algorithm to the classification model that had been trained for a long time, and started to detect the data of the event.

The detection task is an advanced step in the classification task, after your program has identified the category of the image, the next operation is to use a rectangular box to circle the position of the object in the image. That is, the frame on the face of the person who everyone was familiar with later.

The next step is segmentation, which does not use a large, regular shape like a rectangular box, but represents the detailed outline of an object on the picture at the pixel level, which is a kind of operation similar to automatic cutout.

Of course, whether it is detection or segmentation, it is necessary to manually label the original answer of the training set.

IMAGENET-2013'S DETECTION TRACK DATASET IS NOT TOO LARGE, WITH A TOTAL OF NEARLY 400,000 IMAGES, AND A TOTAL OF 200 CATEGORIES. This advanced type of data is much more difficult to label, so the amount of data and classification are not the same.

However, compared with 5,717 in 2012, a year has been a huge leap of 100 times.

"I didn't expect it to be so much time." Meng Fanqi remembers that most of the detections at this time were still based on traditional HOG and LBP methods, and the highest mAP on this dataset in 13 years was about 0.225.

Since I have time to complete the experiments on my thesis, I naturally have to take the time to reduce the dimension and attack these old antique methods.

Each participating team has three chances to submit for each task, and Meng Fanqi only needs one time.

Teams participating in competitions often train several versions of the model, and then do some permutations and combinations, and submit them multiple times to ensure that their results are not affected by some unstable factors.

It's also a way to pursue higher performance, because there's no guarantee of which result will perform best on the location's data.

Sometimes the difference between first and second place is just a millimeter, maybe just two or three decimal places.

It's just that Meng Fanqi doesn't have to do this at all.

There was no time to do anything in the rest of the time, Meng Fanqi wanted to submit the results early on the 11th, and more things were better than less.

But Don Juan stopped him, saying that the hero always had to be the last to arrive, so that it was particularly dramatic.

"This submission is not displayed in real time, but the results will be announced on the 14th." Meng Fanqi pointed out such an embarrassing problem.

"Uhhhh Don Juan had to explain forcefully, "Although others can't see it, the organizers can't see it." At the last minute, give them a little Chinese shock! ”

-------------------

Across the ocean, Stanford University's AI lab SAIL was founded in 1963 in the first wave of neural networks. Witnessed two booms and two declines in AI.

Today, it is directed by Li Feifei, a young Chinese scientist and organizer of IMAGENET.

WHILE AT PRINCETON WHEN I STARTED THE IMAGENET PROJECT IN '09, LI FEIFEI CAME TO STANFORD, WHERE SHE WAS PROMOTED TO TENURED PROFESSOR AND BEGAN LEADING STANFORD'S AI LAB THIS YEAR.

It is not easy to take over such a laboratory with a history, and in addition, this year's IMAGENET competition has just ended, Li Feifei is quite busy at the moment.

She glanced at the results for the new year yesterday, and it was expected.

I haven't seen any particularly groundbreaking papers this year, and everyone is still basically learning AlexNet and exploring new tracks.

Deep neural networks were the best last year, but who can be 100% sure that this is the right path?

Even the best-performing model still has a top-5 error rate of more than 11 points, and generally speaking, this result may be predicted by multiple network integrations, but it is easy to use, and this method has no practical application value. LI FEIFEI DOESN'T WANT THE IMAGENET HE CREATED TO BECOME A BRUSH PARADISE.

It's a long way to go.

At this moment, his phone rang suddenly, Li Feifei picked up the phone and glanced at it, it was Deng Jia.

"Holy shit, teacher, look at the verification results of the event."

Deng Jia's voice sounded very excited, and he said shit as soon as he came up.

"The result? What a result. "Li Feifei doesn't know what's going on, she has already seen the list yesterday, everyone's level is about the same, what can be counted today?

At the moment, it is not convenient to re-link the server, so Li Feifei said, "You can just take a screenshot and send it to me." ”

"Toot..."

Hung up directly over there, Li Feifei frowned slightly, what's wrong with this kid today, he is not usually so frizzy.

Soon, two pictures were transmitted.

Li Feifei clicked on it in turn, and in an instant, his pupils dilated slightly, and his breathing suddenly became rapid unconsciously.

I saw that the top rows of the two lists were the same team.

Team Name: Dream.

The commit description differs by only one letter: "A single DreamNet." and "A single DreamDet."

In a group of commits that integrate multiple models, the word single stands out.