VOC2008 RESULTS

Key to Abbreviations

AbbreviationContributors
BerlinFIRSTNikon Shinichi Nakajima1,3, Motoaki Kawanabe2, Christina Mueller1,2, Alexander Binder2
1Machine Learning Group, Technical University of Berlin; 2Intelligent Data Analysis Group, Fraunhofer Institute FIRST; 3Optical Research Laboratory, Nikon Corporation
BrookesMSRC Lubor Ladicky1, Phil Torr1, Pushmeet Kohli2
1Oxford Brookes University; 2Microsoft Research Cambridge
CASIA_Det Rongguo Zhang, Baihua Xiao, Chunheng Wang, Gang Cheng
Institute of Automation, Chinese Academy of Sciences
CASIA_LinSVM Aiwen Jiang, Gang Cheng, Linbo Zhang, Xinjie Li, Baihua Xiao, Chunheng Wang
Institute of Automation, Chinese Academy of Sciences
CASIA_NeuralNet Aiwen Jiang, Gang Cheng, Linbo Zhang, Xinjie Li, Baihua Xiao, Chunheng Wang
Institute of Automation, Chinese Academy of Sciences
CASIA_NonLinSVM Aiwen Jiang, Gang Cheng, Linbo Zhang, Xinjie Li, Baihua Xiao, Chunheng Wang
Institute of Automation, Chinese Academy of Sciences
ECPLIAMA Regis Behmo1,2, Veronique Prinet2, Nikos Paragios1
1MAS laboratory, Ecole Centrale Paris; 2LIAMA, Institute of Automation, Chinese Academy of Sciences
FIRST_SC1C Alexander Binder, Motoaki Kawanabe, Christina Mueller, Wojciech Wojcikiewicz
Fraunhofer Institute FIRST
FIRST_SCST Alexander Binder, Motoaki Kawanabe, Christina Mueller, Wojciech Wojcikiewicz
Fraunhofer Institute FIRST
INRIASaclay_CMA Fei Jiang, Hugues Berry, Marc Schoenauer, Oliver Temam
INRIA Saclay
INRIASaclay_MEVO Fei Jiang, Hugues Berry, Marc Schoenauer, Oliver Temam
INRIA Saclay
Jena Erik Rodner, Doaa Hegazy, Joachim Denzler
Friedrich Schiller University of Jena
LEAR_PlusClass Hedi Harzallah, Cordelia Schmid, Frederic Jurie, Adrien Gaidon
INRIA Rhone-Alpes
LEAR_flat Adrien Gaidon, Marcin Marszalek, Cordelia Schmid
LEAR, INRIA Rhone-Alpes
LEAR_shotgun Adrien Gaidon, Marcin Marszalek, Cordelia Schmid
LEAR, INRIA Rhone-Alpes
MPI_norank Sebastian Nowozin & Christoph Lampert
Max Planck Institute for Biological Cybernetics, Dept. Empirical Inference
MPI_single Sebastian Nowozin & Christoph Lampert
Max Planck Institute for Biological Cybernetics, Dept. Empirical Inference
MPI_struct Christoph Lampert
Max Planck Institute for Biological Cybernetics, Dept. Empirical Inference
Oxford Andrea Vedaldi & Andrew Zisserman
University of Oxford
SurreyUvA_SRKDA Muhammad Atif Tahir1, Koen van de Sande2, Jasper Uijlings2, Fei Yan1, Xirong Li2, Krystian Mikolajczyk1, Josef Kittler1, Theo Gevers2, Arnold Smeulders2
1University of Surrey; 2University of Amsterdam
TKK_ALL_SFBS Ville Viitaniemi & Jorma Laaksonen
Adaptive Informatics Research Centre, Helsinki University of Technology (TKK)
TKK_MAXVAL Ville Viitaniemi & Jorma Laaksonen
Adaptive Informatics Research Centre, Helsinki University of Technology (TKK)
UIUC_CMU Derek Hoiem1, Santosh Divvala2, James H. Hays2
1University of Illionois Urbana-Champaign; 2Carnegie Mellon University
UoCTTIUCI Pedro Felzenszwalb1, Ross Girshick1, David McAllester2, Deva Ramanan3
1University of Chicago; 2TTI Chicago; 3University of California, Irvine
UvA_0708Soft5ColorSift Koen van de Sande
University of Amsterdam
UvA_AdapTagRelDom Xirong Li & Koen van de Sande
University of Amsterdam
UvA_FullSFS Koen van de Sande
University of Amsterdam
UvA_Soft5ColorSift Koen van de Sande
University of Amsterdam
UvA_TreeSFS Jasper Uijlings & Koen van de Sande
University of Amsterdam
XRCE Florent Perronnin & Yan Liu
Xerox Research Centre Europe (XRCE), Textual and Visual Pattern Analysis Group
XRCE_Det Gabriela Czurka, Florent Perronnin, Yan Liu
Xerox Research Centre Europe (XRCE), Textual and Visual Pattern Analysis Group
XRCE_Seg Gabriela Czurka, Florent Perronnin, Yan Liu
Xerox Research Centre Europe (XRCE), Textual and Visual Pattern Analysis Group

Classification Results: VOC2008 data

Competition "comp1" (train on VOC2008 data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
BerlinFIRSTNikon 72.4 37.4 51.1 57.4 24.5 38.5 53.9 44.7 46.2 25.6 28.6 40.3 57.0 53.5 83.0 21.0 21.4 28.6 66.2 50.2
CASIA_LinSVM 50.8 21.5 31.8 41.8 18.3 7.7 35.7 40.6 30.6 4.9 7.1 31.5 29.3 22.0 73.4 7.9 13.7 11.6 18.9 31.0
CASIA_NeuralNet 47.6 12.7 28.4 35.3 12.7 5.8 31.6 34.5 22.7 3.7 6.0 23.6 21.9 16.1 69.1 8.1 5.0 8.7 15.9 15.1
CASIA_NonLinSVM 35.1 19.7 24.2 40.5 13.7 3.8 30.5 37.1 30.2 5.8 6.2 31.1 20.4 26.3 74.5 5.1 14.4 9.3 10.3 21.8
ECPLIAMA 54.9 25.3 31.1 29.8 18.9 21.4 33.7 25.5 28.4 - - 21.2 27.7 30.5 68.5 - - - 31.7 26.6
FIRST_SC1C 36.6 16.8 17.3 26.9 7.6 14.0 29.0 28.5 22.9 4.3 8.0 23.2 14.8 30.3 64.5 10.3 5.5 13.2 9.6 24.0
FIRST_SCST 36.6 16.8 17.3 26.9 7.6 14.0 29.0 28.5 22.9 4.3 8.0 23.2 14.8 30.3 64.5 10.3 5.5 13.2 9.6 24.0
INRIASaclay_CMA 52.4 15.0 23.6 33.9 10.2 10.3 32.7 32.4 26.4 13.2 16.0 22.2 18.5 27.3 64.8 8.6 4.4 7.9 20.0 30.4
INRIASaclay_MEVO 50.2 19.5 17.9 32.1 13.0 14.9 34.2 30.6 23.1 4.4 14.8 21.0 13.1 26.1 65.7 7.6 7.4 17.5 16.5 30.6
LEAR_flat 80.1 51.8 60.5 66.9 29.1 52.0 57.4 58.6 48.7 31.0 39.2 47.6 64.2 64.6 87.0 28.6 33.3 42.6 73.1 59.8
LEAR_shotgun 81.1 52.9 61.6 67.8 29.4 52.1 58.7 59.9 48.5 32.0 38.6 47.9 65.4 65.2 87.0 29.0 34.4 43.1 74.3 61.5
SurreyUvA_SRKDA 79.5 54.3 61.4 64.8 30.0 52.1 59.5 59.4 48.9 33.6 37.8 46.0 66.1 64.0 86.8 29.2 42.3 44.0 77.8 61.2
TKK_ALL_SFBS 77.9 47.3 52.4 61.0 27.9 45.5 53.5 55.5 47.6 26.8 40.8 46.1 58.6 58.3 83.5 26.4 24.3 39.2 70.3 56.9
TKK_MAXVAL 76.7 47.3 51.6 60.8 28.3 44.6 54.2 55.5 47.8 21.2 39.2 46.1 58.8 55.9 83.3 26.4 24.3 41.9 70.2 52.4
UvA_FullSFS 79.8 53.0 61.3 65.7 28.9 46.5 58.4 58.9 47.7 25.4 35.4 45.2 64.2 59.6 87.0 31.0 35.3 44.6 74.7 60.9
UvA_Soft5ColorSift 79.7 52.1 61.5 65.5 29.1 46.5 58.3 57.4 48.2 27.9 38.3 46.6 66.0 60.6 87.0 31.8 42.2 45.3 72.3 64.7
UvA_TreeSFS 80.8 53.2 61.6 65.6 29.4 49.9 58.5 59.4 48.0 30.1 39.6 45.0 67.3 60.4 87.1 30.1 41.5 45.4 74.3 59.8
XRCE 78.9 48.0 58.7 65.2 29.0 44.8 56.1 56.3 43.7 32.8 30.4 39.7 61.2 61.7 86.8 22.9 34.2 44.2 68.4 59.1

Precision/Recall Curves

Classification Results: VOC2008 data

Competition "comp2" (train on own data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
UIUC_CMU 67.8 52.1 38.4 49.7 44.0 38.6 58.4 38.5 41.2 24.9 31.2 27.1 46.2 60.3 85.4 19.2 22.6 31.5 51.3 58.3
UvA_0708Soft5ColorSift 81.9 55.0 67.7 68.8 33.7 51.8 64.0 61.4 51.2 40.1 48.6 50.8 69.4 64.3 88.2 37.6 48.2 46.4 76.8 66.3
UvA_AdapTagRelDom 79.7 49.7 62.4 65.2 28.7 48.8 60.1 57.0 47.4 35.4 36.6 46.3 66.5 58.8 86.9 30.3 42.8 43.0 73.9 63.5

Precision/Recall Curves

Detection Results: VOC2008 data

Competition "comp3" (train on VOC2008 data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
CASIA_Det 25.2 14.6 9.8 10.5 6.3 23.2 17.6 9.0 9.6 10.0 13.0 5.5 14.0 24.1 11.2 3.0 2.8 3.0 28.2 14.6
Jena 4.8 1.4 0.3 0.2 0.1 1.0 1.3 - 0.1 4.7 0.4 1.9 0.3 3.1 2.0 0.3 0.4 2.2 6.4 13.7
LEAR_PlusClass 36.5 34.3 10.7 11.4 22.1 23.8 36.6 16.6 11.1 17.7 15.1 9.0 36.1 40.3 19.7 11.5 19.4 17.3 29.6 34.0
MPI_struct 25.9 8.0 10.1 5.6 0.1 11.3 10.6 21.3 0.3 4.5 10.1 14.9 16.6 20.0 2.5 0.2 9.3 12.3 23.6 1.5
Oxford 33.3 24.6 - - - - 29.1 - - 12.5 - - 32.5 34.9 - - - - - -
UoCTTIUCI 32.6 42.0 11.3 11.0 28.2 23.2 32.0 17.9 14.6 11.1 6.6 10.2 32.7 38.6 42.0 12.6 16.1 13.6 24.4 37.1
XRCE_Det 26.4 10.5 1.4 4.5 0.0 10.8 4.0 7.6 2.0 1.8 4.5 10.5 11.8 13.6 9.0 1.5 6.1 1.8 7.3 6.8

Precision/Recall Curves

Detection Results: VOC2008 data

Competition "comp4" (train on own data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
UIUC_CMU 34.5 32.7 12.3 11.0 22.4 18.5 27.8 21.6 8.8 14.1 15.2 17.8 27.4 40.9 37.4 11.2 7.0 13.5 28.2 38.5

Precision/Recall Curves

Classification Results: VOC2007 data

Competition "comp1" (train on VOC2008 data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
LEAR_flat 77.1 56.2 47.9 68.1 30.6 52.3 68.0 56.4 53.6 34.9 50.2 42.7 54.9 61.3 84.0 36.5 36.6 45.9 73.2 52.5
UvA_Soft5ColorSift 77.1 58.0 53.2 67.8 27.0 55.3 70.1 56.4 55.1 34.7 46.0 42.5 65.0 59.5 84.8 37.4 45.1 50.4 76.9 52.9
XRCE 73.9 46.9 48.0 69.1 24.8 52.0 65.5 49.6 49.0 31.4 37.0 42.2 53.7 55.6 84.0 31.5 43.5 46.6 69.0 49.0

Precision/Recall Curves

Classification Results: VOC2007 data

Competition "comp2" (train on own data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
UvA_0708Soft5ColorSift 82.1 68.7 62.5 71.6 30.0 68.1 77.8 61.2 59.0 49.7 58.5 51.9 81.2 66.5 87.2 43.6 56.7 59.7 83.3 57.0

Precision/Recall Curves

Detection Results: VOC2007 data

Competition "comp3" (train on VOC2008 data)

Average Precision (AP %)

  aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
LEAR_PlusClass 28.5 39.0 10.7 11.2 20.2 41.0 48.4 15.2 16.1 25.7 10.1 11.5 34.9 39.7 16.8 10.3 21.8 22.8 37.0 36.3
Oxford 27.7 29.1 - - - - 41.5 - - 16.3 - - 31.9 33.8 - - - - - -

Precision/Recall Curves

Segmentation Taster (VOC2008 data)

Competition "comp5" (train on VOC2008 data)

Accuracy (%)

- Entries in parentheses are synthesized from detection results.

  [mean] back
ground		 aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
BrookesMSRC 20.1 75.0 36.9 4.8 22.2 11.2 13.7 13.8 20.4 10.0 8.7 3.6 28.3 6.6 17.1 22.6 30.6 13.5 26.8 12.1 20.1 24.8
(CASIA_Det) 13.5 44.7 8.5 9.4 10.7 7.5 13.3 37.3 10.3 14.4 2.0 5.4 4.0 8.0 9.0 15.9 10.5 8.2 16.8 4.1 21.6 22.1
Jena 8.0 47.8 7.2 3.1 4.6 5.6 2.2 0.6 13.4 0.0 0.7 7.5 0.7 5.7 4.4 8.9 8.7 5.0 9.2 3.4 12.2 17.8
(LEAR_PlusClass) 3.7 5.5 4.1 2.2 0.0 3.5 4.9 3.9 9.6 0.8 1.5 0.1 0.4 0.9 1.2 2.5 7.4 0.2 0.2 0.3 4.0 24.7
MPI_norank 7.0 66.3 6.7 1.2 2.1 3.1 2.5 5.8 2.6 2.9 1.1 1.7 4.0 2.6 3.7 5.1 10.5 0.8 5.8 2.1 8.4 8.0
MPI_single 12.9 75.4 19.1 7.7 6.1 9.4 3.8 11.0 12.1 5.6 0.7 3.7 15.9 3.6 12.2 16.1 15.9 0.6 19.7 5.9 14.7 12.5
(MPI_struct) 12.9 59.2 15.1 5.5 8.2 19.1 8.6 13.0 11.0 11.9 6.8 0.0 3.8 11.0 14.3 14.9 12.1 0.1 7.2 5.1 19.6 23.5
(UoCTTIUCI) 11.6 0.9 9.1 12.8 1.0 6.7 11.8 24.7 11.4 8.9 1.6 9.0 1.3 7.9 14.4 18.8 13.1 9.5 24.3 11.0 17.1 27.8
(XRCE_Det) 18.9 64.8 11.1 12.7 15.4 10.9 7.8 17.7 18.4 19.4 5.6 14.0 8.5 16.8 16.0 30.9 25.3 19.1 29.7 5.5 18.0 29.0
XRCE_Seg 25.4 75.9 25.8 15.7 19.2 21.6 17.2 27.3 25.5 24.2 7.9 25.4 9.9 17.8 23.3 34.0 28.8 23.2 32.1 14.9 25.9 37.3

Segmentation Taster (VOC2008 data)

Competition "comp6" (train on own data)

Note this competition was not officially defined. We include results of the submitted UIUC/CMU method, which used external data, to allow comparison.

Accuracy (%)

- Entries in parentheses are synthesized from detection results.

  [mean] back
ground		 aero
plane		 bicycle bird boat bottle bus car cat chair cow dining
table		 dog horse motor
bike		 person potted
plant		 sheep sofa train tv/
monitor
UIUC_CMU 19.5 79.3 31.9 21.0 8.3 6.5 34.3 15.8 22.7 10.4 1.2 6.8 8.0 10.2 22.7 24.9 27.7 15.9 4.3 5.5 19.0 32.1