Results for CD.net 2014

Results, all categories combined.

Click on method name for more details.

Method Average ranking across categories Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 34.09 33.86 0.6621 0.9542 0.0458 0.3379 5.5456 0.5141 0.5536
SuBSENSE [13] 10.09 9.86 0.8124 0.9904 0.0096 0.1876 1.6780 0.7408 0.7509
SOBS_CF [14] 27.36 27.43 0.7805 0.9442 0.0558 0.2195 6.0709 0.5883 0.5831
RMoG (Region-based Mixture of Gaussians) [15] 27.64 27.29 0.5940 0.9865 0.0135 0.4060 2.9638 0.5735 0.6965
SaliencySubsense [16] 12.09 12.71 0.7714 0.9914 0.0086 0.2286 1.8969 0.7176 0.7628
AAPSA [17] 24.45 23.57 0.6498 0.9905 0.0095 0.3502 2.0734 0.6179 0.6916
Multimode Background Subtraction Version 0 (MBS V0) [18] 17.36 14.14 0.7192 0.9929 0.0071 0.2808 1.3922 0.7139 0.7435
Multimode Background Subtraction [19] 15.45 12.57 0.7389 0.9927 0.0073 0.2611 1.2614 0.7288 0.7382
M4CD Version 2.0 [33] 12.09 15.29 0.7885 0.9841 0.0159 0.2115 2.3011 0.7038 0.7423
GraphCutDiff [20] 26.45 31.00 0.6297 0.9780 0.0220 0.3703 3.6774 0.5684 0.6666
Sample based background subtractor (SBBS) [34] 21.73 22.86 0.7073 0.9827 0.0173 0.2927 2.4315 0.6711 0.7221
EFIC [21] 17.73 18.57 0.7855 0.9779 0.0221 0.2145 2.7941 0.7088 0.7224
IUTIS-1 [22] 24.18 28.57 0.7654 0.9499 0.0501 0.2346 5.7503 0.5789 0.5928
IUTIS-2 [23] 22.36 25.86 0.6621 0.9838 0.0162 0.3379 3.1547 0.6026 0.7120
IUTIS-3 [24] 7.27 6.43 0.7779 0.9940 0.0060 0.2221 1.2985 0.7551 0.7875
Superpixel Strengthen Background Subtraction [25] 12.36 13.14 0.7416 0.9923 0.0077 0.2584 1.8902 0.7129 0.7754
M4CD Version 1.0 [26] 15.18 17.14 0.7750 0.9849 0.0151 0.2250 2.3609 0.6916 0.7320
SharedModel [31] 10.82 8.57 0.8098 0.9912 0.0088 0.1902 1.4996 0.7474 0.7503
C-EFIC [27] 15.27 14.86 0.7976 0.9782 0.0218 0.2024 2.6316 0.7307 0.7543
WeSamBE [36] 10.18 7.86 0.7955 0.9924 0.0076 0.2045 1.5105 0.7446 0.7679
DeepBS (supervised method) [37] 7.64 12.57 0.7545 0.9905 0.0095 0.2455 1.9920 0.7458 0.8332
BMOG [43] 22.45 24.00 0.7265 0.9813 0.0187 0.2735 2.9757 0.6543 0.6981
PAWCS [28] 8.36 6.43 0.7718 0.9949 0.0051 0.2282 1.1992 0.7403 0.7857
Cascade CNN(supervised method) [32] 1.45 1.00 0.9506 0.9968 0.0032 0.0494 0.4052 0.9209 0.8997
Euclidean distance [1] 35.18 34.71 0.6803 0.9449 0.0551 0.3197 6.5423 0.5161 0.5480
KDE - ElGammal [2] 28.91 30.29 0.7375 0.9519 0.0481 0.2625 5.6262 0.5688 0.5811
GMM | Stauffer & Grimson [3] 31.09 29.86 0.6846 0.9750 0.0250 0.3154 3.7667 0.5707 0.6025
GMM | Zivkovic [4] 32.36 32.29 0.6604 0.9725 0.0275 0.3396 3.9953 0.5566 0.5973
Mahalanobis distance [5] 27.09 24.86 0.1644 0.9931 0.0069 0.8356 3.4750 0.2267 0.7403
CwisarDRP [29] 12.64 13.29 0.7062 0.9947 0.0053 0.2938 1.7197 0.7095 0.7880
IUTIS-5 [30] 3.73 4.14 0.7849 0.9948 0.0052 0.2151 1.1986 0.7717 0.8087
CwisarDH [6] 17.27 15.57 0.6608 0.9948 0.0052 0.3392 1.5273 0.6812 0.7725
Spectral-360 [7] 21.55 20.43 0.7345 0.9861 0.0139 0.2655 2.2722 0.6732 0.7054
DCB [35] 31.73 28.43 0.3892 0.9897 0.0103 0.6108 2.8789 0.3975 0.6309
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 11.36 11.14 0.7657 0.9922 0.0078 0.2343 1.3763 0.7283 0.7696
SC_SOBS [9] 26.64 26.71 0.7621 0.9547 0.0453 0.2379 5.1498 0.5961 0.6091
AMBER [10] 22.91 24.86 0.7035 0.9794 0.0206 0.2965 2.9009 0.6577 0.7163
CP3-online [11] 30.91 28.86 0.7225 0.9705 0.0295 0.2775 3.4318 0.5805 0.5559

Results, for the bad weather category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 35.43 0.5964 0.9892 0.0108 0.4036 1.6752 0.6371 0.7680
SuBSENSE [13] 8.86 0.8213 0.9989 0.0011 0.1787 0.4527 0.8619 0.9091
SOBS_CF [14] 33.86 0.5791 0.9941 0.0059 0.4209 1.1926 0.6370 0.7762
RMoG (Region-based Mixture of Gaussians) [15] 23.29 0.5572 0.9991 0.0009 0.4428 0.8739 0.6826 0.8955
SaliencySubsense [16] 9.43 0.8142 0.9990 0.0010 0.1858 0.4565 0.8593 0.9129
AAPSA [17] 15.86 0.6738 0.9993 0.0007 0.3262 0.6650 0.7742 0.9255
Multimode Background Subtraction Version 0 (MBS V0) [18] 24.86 0.8211 0.9936 0.0064 0.1789 0.9449 0.7730 0.7571
Multimode Background Subtraction [19] 21.29 0.8341 0.9953 0.0047 0.1659 0.7634 0.7980 0.7828
M4CD Version 2.0 [33] 13.29 0.7391 0.9990 0.0010 0.2609 0.5037 0.8136 0.9067
GraphCutDiff [20] 10.14 0.8701 0.9979 0.0021 0.1299 0.4085 0.8787 0.8906
Sample based background subtractor (SBBS) [34] 28.43 0.7057 0.9958 0.0042 0.2943 0.8953 0.7403 0.8064
EFIC [21] 21.57 0.7647 0.9962 0.0038 0.2353 0.7164 0.7786 0.8373
IUTIS-1 [22] 33.86 0.6557 0.9906 0.0094 0.3443 1.4403 0.6705 0.7486
IUTIS-2 [23] 17.29 0.6226 0.9994 0.0006 0.3774 0.6450 0.7401 0.9415
IUTIS-3 [24] 15.57 0.7479 0.9987 0.0013 0.2521 0.5534 0.8032 0.8960
Superpixel Strengthen Background Subtraction [25] 9.43 0.8107 0.9990 0.0010 0.1893 0.4670 0.8580 0.9135
M4CD Version 1.0 [26] 16.00 0.7354 0.9988 0.0012 0.2646 0.5311 0.8069 0.8955
SharedModel [31] 13.29 0.8430 0.9978 0.0022 0.1570 0.5138 0.8480 0.8568
C-EFIC [27] 21.00 0.7352 0.9977 0.0023 0.2648 0.6600 0.7867 0.8719
WeSamBE [36] 9.43 0.8168 0.9989 0.0011 0.1832 0.4891 0.8608 0.9134
DeepBS (supervised method) [37] 6.29 0.7517 0.9996 0.0004 0.2483 0.3784 0.8301 0.9677
BMOG [43] 20.57 0.7635 0.9976 0.0024 0.2365 0.6243 0.7836 0.8152
PAWCS [28] 11.14 0.7181 0.9994 0.0006 0.2819 0.5319 0.8152 0.9474
Cascade CNN(supervised method) [32] 3.00 0.9312 0.9993 0.0007 0.0688 0.1911 0.9431 0.9555
Euclidean distance [1] 26.71 0.5567 0.9987 0.0013 0.4433 0.7824 0.6701 0.8846
KDE - ElGammal [2] 25.43 0.6941 0.9975 0.0025 0.3059 0.7192 0.7571 0.8486
GMM | Stauffer & Grimson [3] 27.57 0.7181 0.9971 0.0029 0.2819 0.7905 0.7380 0.7704
GMM | Zivkovic [4] 26.00 0.6863 0.9978 0.0022 0.3137 0.7707 0.7406 0.8138
Mahalanobis distance [5] 21.71 0.1749 1.0000 0.0000 0.8251 1.2575 0.2212 0.9975
CwisarDRP [29] 17.14 0.7531 0.9984 0.0016 0.2469 0.5760 0.8015 0.8718
IUTIS-5 [30] 9.29 0.7493 0.9993 0.0007 0.2507 0.5002 0.8248 0.9311
CwisarDH [6] 24.57 0.6288 0.9986 0.0014 0.3712 0.7475 0.6837 0.8762
Spectral-360 [7] 24.71 0.7032 0.9977 0.0023 0.2968 0.6804 0.7569 0.8211
DCB [35] 30.71 0.2588 0.9984 0.0016 0.7412 1.5795 0.3835 0.8261
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 11.43 0.7457 0.9991 0.0009 0.2543 0.5109 0.8228 0.9231
SC_SOBS [9] 30.00 0.5676 0.9976 0.0024 0.4324 0.8606 0.6620 0.8434
AMBER [10] 17.57 0.6661 0.9990 0.0010 0.3339 0.6164 0.7673 0.9169
CP3-online [11] 25.00 0.8365 0.9934 0.0066 0.1635 0.9364 0.7485 0.7001

Results, for the low framerate category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 34.00 0.6057 0.9608 0.0392 0.3943 4.7581 0.3365 0.2917
SuBSENSE [13] 17.00 0.8537 0.9938 0.0062 0.1463 0.9968 0.6445 0.6035
SOBS_CF [14] 28.29 0.8046 0.9173 0.0827 0.1954 8.7531 0.5148 0.4613
RMoG (Region-based Mixture of Gaussians) [15] 31.14 0.5805 0.9922 0.0078 0.4195 1.6809 0.5312 0.5916
SaliencySubsense [16] 14.29 0.8333 0.9950 0.0050 0.1667 0.9399 0.6515 0.6311
AAPSA [17] 29.86 0.5799 0.9943 0.0057 0.4201 1.6158 0.4942 0.5675
Multimode Background Subtraction Version 0 (MBS V0) [18] 20.71 0.6656 0.9942 0.0058 0.3344 0.8610 0.6279 0.6604
Multimode Background Subtraction [19] 22.14 0.6773 0.9942 0.0058 0.3227 0.8651 0.6350 0.6000
M4CD Version 2.0 [33] 15.57 0.7911 0.9949 0.0051 0.2089 0.8394 0.6275 0.6315
GraphCutDiff [20] 22.57 0.4713 0.9966 0.0034 0.5287 1.2794 0.5127 0.6814
Sample based background subtractor (SBBS) [34] 27.14 0.6997 0.9927 0.0073 0.3003 1.3551 0.5534 0.5461
EFIC [21] 7.29 0.7694 0.9982 0.0018 0.2306 0.5666 0.6632 0.7232
IUTIS-1 [22] 19.86 0.7316 0.9955 0.0045 0.2684 1.0088 0.5694 0.6260
IUTIS-2 [23] 18.00 0.7513 0.9951 0.0049 0.2487 0.9678 0.6034 0.6690
IUTIS-3 [24] 7.71 0.8213 0.9963 0.0037 0.1787 0.7267 0.7327 0.6995
Superpixel Strengthen Background Subtraction [25] 14.71 0.7446 0.9960 0.0040 0.2554 0.9733 0.6910 0.6816
M4CD Version 1.0 [26] 19.86 0.7860 0.9941 0.0059 0.2140 0.9243 0.6119 0.6010
SharedModel [31] 9.00 0.8430 0.9958 0.0042 0.1570 0.7450 0.7286 0.6839
C-EFIC [27] 5.43 0.8077 0.9976 0.0024 0.1923 0.5532 0.6806 0.7135
WeSamBE [36] 13.00 0.8842 0.9944 0.0056 0.1158 0.8216 0.6602 0.6134
DeepBS (supervised method) [37] 17.71 0.5924 0.9975 0.0025 0.4076 1.3564 0.6002 0.7018
BMOG [43] 15.86 0.6385 0.9971 0.0029 0.3615 0.8967 0.6102 0.6956
PAWCS [28] 12.29 0.7732 0.9963 0.0037 0.2268 0.7258 0.6588 0.6405
Cascade CNN(supervised method) [32] 1.86 0.8489 0.9993 0.0007 0.1511 0.1317 0.8370 0.8285
Euclidean distance [1] 31.00 0.5914 0.9868 0.0132 0.4086 2.2419 0.5015 0.6152
KDE - ElGammal [2] 25.00 0.7000 0.9931 0.0069 0.3000 1.3124 0.5478 0.6245
GMM | Stauffer & Grimson [3] 21.57 0.5823 0.9961 0.0039 0.4177 1.2951 0.5373 0.6894
GMM | Zivkovic [4] 22.57 0.5300 0.9970 0.0030 0.4700 1.3620 0.5065 0.6686
Mahalanobis distance [5] 21.71 0.0538 0.9999 0.0001 0.9462 2.5114 0.0797 0.7612
CwisarDRP [29] 8.14 0.7718 0.9972 0.0028 0.2282 0.6774 0.6858 0.7045
IUTIS-5 [30] 5.29 0.8398 0.9968 0.0032 0.1602 0.6766 0.7743 0.7424
CwisarDH [6] 20.00 0.6738 0.9951 0.0049 0.3262 1.0435 0.6406 0.6399
Spectral-360 [7] 20.86 0.7515 0.9941 0.0059 0.2485 0.8964 0.6437 0.5946
DCB [35] 31.29 0.1570 0.9948 0.0052 0.8430 2.9401 0.1412 0.5957
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 16.14 0.7517 0.9963 0.0037 0.2483 1.1823 0.6259 0.6550
SC_SOBS [9] 27.71 0.7874 0.9577 0.0423 0.2126 4.7727 0.5463 0.5272
AMBER [10] 33.43 0.5226 0.9911 0.0089 0.4774 2.2069 0.4689 0.5937
CP3-online [11] 31.00 0.6810 0.9854 0.0146 0.3190 2.1756 0.4742 0.5263

Results, for the night videos category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 28.86 0.5773 0.9574 0.0426 0.4227 5.8859 0.4164 0.4270
SuBSENSE [13] 12.43 0.6570 0.9766 0.0234 0.3430 3.7718 0.5599 0.5359
SOBS_CF [14] 24.86 0.6808 0.9463 0.0537 0.3192 6.5308 0.4482 0.4056
RMoG (Region-based Mixture of Gaussians) [15] 28.86 0.5524 0.9668 0.0332 0.4476 5.1606 0.4265 0.4345
SaliencySubsense [16] 13.71 0.5844 0.9819 0.0181 0.4156 3.5948 0.5348 0.5633
AAPSA [17] 21.14 0.3908 0.9871 0.0129 0.6092 3.7971 0.4161 0.5056
Multimode Background Subtraction Version 0 (MBS V0) [18] 18.57 0.5535 0.9773 0.0227 0.4465 3.6592 0.5158 0.4899
Multimode Background Subtraction [19] 18.57 0.5535 0.9773 0.0227 0.4465 3.6592 0.5158 0.4899
M4CD Version 2.0 [33] 19.57 0.6525 0.9696 0.0304 0.3475 4.6115 0.4946 0.4891
GraphCutDiff [20] 23.00 0.6435 0.9687 0.0313 0.3565 4.7439 0.4688 0.4292
Sample based background subtractor (SBBS) [34] 19.57 0.5186 0.9818 0.0182 0.4814 3.6968 0.5055 0.5343
EFIC [21] 4.14 0.6704 0.9893 0.0107 0.3296 2.5739 0.6548 0.6869
IUTIS-1 [22] 21.71 0.6056 0.9717 0.0283 0.3944 4.5767 0.4770 0.4709
IUTIS-2 [23] 16.14 0.5594 0.9825 0.0175 0.4406 3.6933 0.5154 0.5348
IUTIS-3 [24] 18.00 0.5664 0.9819 0.0181 0.4336 3.7085 0.4948 0.5130
Superpixel Strengthen Background Subtraction [25] 12.29 0.5683 0.9848 0.0152 0.4317 3.3363 0.5384 0.5648
M4CD Version 1.0 [26] 19.29 0.6495 0.9704 0.0296 0.3505 4.5511 0.4977 0.4880
SharedModel [31] 14.14 0.5995 0.9799 0.0201 0.4005 3.5758 0.5419 0.5250
C-EFIC [27] 4.43 0.7223 0.9866 0.0134 0.2777 2.5899 0.6677 0.6636
WeSamBE [36] 8.57 0.6370 0.9840 0.0160 0.3630 3.1305 0.5929 0.5827
DeepBS (supervised method) [37] 10.29 0.5315 0.9959 0.0041 0.4685 2.5754 0.5835 0.8366
BMOG [43] 19.71 0.6495 0.9694 0.0306 0.3505 4.4376 0.4982 0.4611
PAWCS [28] 17.86 0.3608 0.9932 0.0068 0.6392 3.3386 0.4152 0.6539
Cascade CNN(supervised method) [32] 1.29 0.9139 0.9964 0.0036 0.0861 0.6116 0.8965 0.8805
Euclidean distance [1] 32.57 0.4913 0.9653 0.0347 0.5087 5.5378 0.3859 0.4168
KDE - ElGammal [2] 27.71 0.5914 0.9640 0.0360 0.4086 5.2735 0.4365 0.4036
GMM | Stauffer & Grimson [3] 29.57 0.5261 0.9701 0.0299 0.4739 4.9179 0.4097 0.4128
GMM | Zivkovic [4] 29.43 0.4797 0.9739 0.0261 0.5203 4.7227 0.3960 0.4231
Mahalanobis distance [5] 19.29 0.0825 0.9978 0.0022 0.9175 3.7362 0.1374 0.6914
CwisarDRP [29] 20.14 0.5067 0.9819 0.0181 0.4933 3.8484 0.4970 0.5447
IUTIS-5 [30] 13.00 0.5852 0.9828 0.0172 0.4148 3.5684 0.5290 0.5438
CwisarDH [6] 23.00 0.4076 0.9852 0.0148 0.5924 3.9853 0.3735 0.5021
Spectral-360 [7] 20.14 0.6237 0.9739 0.0261 0.3763 4.4642 0.4832 0.4610
DCB [35] 23.71 0.1714 0.9939 0.0061 0.8286 3.8662 0.2305 0.4481
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 18.14 0.6107 0.9759 0.0241 0.3893 4.0052 0.5130 0.4904
SC_SOBS [9] 24.86 0.6496 0.9515 0.0485 0.3504 6.1567 0.4503 0.4241
AMBER [10] 31.71 0.5890 0.9375 0.0625 0.4110 7.8383 0.3802 0.3818
CP3-online [11] 29.71 0.6221 0.9381 0.0619 0.3779 7.6963 0.3919 0.3410

Results, for the ptz category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 29.57 0.7953 0.7282 0.2718 0.2047 27.1630 0.0364 0.0188
SuBSENSE [13] 14.71 0.8306 0.9629 0.0371 0.1694 3.8159 0.3476 0.2840
SOBS_CF [14] 27.43 0.8558 0.6796 0.3204 0.1442 31.9430 0.0368 0.0190
RMoG (Region-based Mixture of Gaussians) [15] 23.00 0.6409 0.9278 0.0722 0.3591 7.4757 0.2470 0.2212
SaliencySubsense [16] 15.43 0.8366 0.9592 0.0408 0.1634 4.1845 0.3399 0.2796
AAPSA [17] 20.14 0.5128 0.9638 0.0362 0.4872 3.9676 0.3302 0.3772
Multimode Background Subtraction Version 0 (MBS V0) [18] 12.29 0.5770 0.9945 0.0055 0.4230 0.7821 0.5118 0.4988
Multimode Background Subtraction [19] 10.71 0.5973 0.9963 0.0037 0.4027 0.5850 0.5520 0.5400
M4CD Version 2.0 [33] 20.71 0.8538 0.8984 0.1016 0.1462 10.2247 0.2322 0.1791
GraphCutDiff [20] 16.00 0.5798 0.9868 0.0132 0.4202 1.6312 0.3723 0.3325
Sample based background subtractor (SBBS) [34] 23.29 0.6567 0.9243 0.0757 0.3433 7.8369 0.2400 0.2239
EFIC [21] 11.86 0.9177 0.9217 0.0783 0.0823 7.8707 0.5842 0.5282
IUTIS-1 [22] 25.14 0.8803 0.6844 0.3156 0.1197 31.4561 0.0453 0.0237
IUTIS-2 [23] 22.57 0.8488 0.8881 0.1119 0.1512 11.2503 0.2198 0.1704
IUTIS-3 [24] 13.00 0.6644 0.9868 0.0132 0.3356 1.5649 0.3921 0.3474
Superpixel Strengthen Background Subtraction [25] 15.43 0.8242 0.9629 0.0371 0.1758 3.8192 0.3337 0.2745
M4CD Version 1.0 [26] 19.43 0.8643 0.9101 0.0899 0.1357 9.0552 0.2348 0.1804
SharedModel [31] 12.57 0.7969 0.9791 0.0209 0.2031 2.2166 0.3860 0.3121
C-EFIC [27] 13.29 0.8686 0.8947 0.1053 0.1314 10.5973 0.6207 0.6144
WeSamBE [36] 12.86 0.8145 0.9754 0.0246 0.1855 2.5692 0.3844 0.3121
DeepBS (supervised method) [37] 20.43 0.7459 0.9248 0.0752 0.2541 7.7228 0.3133 0.2855
BMOG [43] 23.86 0.7667 0.8891 0.1109 0.2333 11.2335 0.2350 0.2094
PAWCS [28] 10.00 0.6976 0.9912 0.0088 0.3024 1.1162 0.4615 0.4725
Cascade CNN(supervised method) [32] 1.00 0.9663 0.9990 0.0010 0.0337 0.1221 0.9168 0.8730
Euclidean distance [1] 31.00 0.7808 0.6614 0.3386 0.2192 33.8518 0.0395 0.0206
KDE - ElGammal [2] 30.43 0.8121 0.6761 0.3239 0.1879 32.3132 0.0365 0.0188
GMM | Stauffer & Grimson [3] 29.00 0.6475 0.8570 0.1430 0.3525 14.5321 0.1522 0.1185
GMM | Zivkovic [4] 30.57 0.6111 0.8330 0.1670 0.3889 16.9493 0.1046 0.0683
Mahalanobis distance [5] 28.14 0.0398 0.9574 0.0426 0.9602 4.9260 0.0374 0.1311
CwisarDRP [29] 11.29 0.7539 0.9883 0.0117 0.2461 1.2984 0.4292 0.3200
IUTIS-5 [30] 11.14 0.6749 0.9902 0.0098 0.3251 1.2166 0.4282 0.3833
CwisarDH [6] 14.86 0.3363 0.9977 0.0023 0.6637 0.6847 0.3218 0.4824
Spectral-360 [7] 22.14 0.5047 0.9416 0.0584 0.4953 6.0771 0.3653 0.3265
DCB [35] 23.71 0.2259 0.9867 0.0133 0.7741 1.9676 0.0804 0.1150
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 16.57 0.6730 0.9770 0.0230 0.3270 2.5519 0.3241 0.2861
SC_SOBS [9] 26.71 0.8403 0.7126 0.2874 0.1597 28.6809 0.0409 0.0212
AMBER [10] 30.00 0.5161 0.8880 0.1120 0.4839 11.5906 0.1348 0.1895
CP3-online [11] 20.71 0.6061 0.9711 0.0289 0.3939 3.1516 0.2660 0.1992

Results, for the turbulence category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 25.29 0.6796 0.9972 0.0028 0.3204 0.4151 0.5291 0.4926
SuBSENSE [13] 9.57 0.8050 0.9994 0.0006 0.1950 0.1527 0.7792 0.7814
SOBS_CF [14] 28.86 0.7391 0.9763 0.0237 0.2609 2.4806 0.4702 0.4685
RMoG (Region-based Mixture of Gaussians) [15] 29.43 0.5780 0.9952 0.0048 0.4220 0.7314 0.4578 0.5701
SaliencySubsense [16] 13.14 0.7759 0.9993 0.0007 0.2241 0.1749 0.7512 0.7941
AAPSA [17] 26.29 0.7401 0.9933 0.0067 0.2599 0.7735 0.4643 0.4289
Multimode Background Subtraction Version 0 (MBS V0) [18] 25.86 0.5571 0.9981 0.0019 0.4429 0.3086 0.5698 0.6804
Multimode Background Subtraction [19] 25.43 0.6037 0.9979 0.0021 0.3963 0.3180 0.5858 0.6198
M4CD Version 2.0 [33] 10.29 0.7248 0.9997 0.0003 0.2752 0.1639 0.7978 0.8941
GraphCutDiff [20] 24.14 0.7562 0.9945 0.0055 0.2438 0.6509 0.5143 0.4929
Sample based background subtractor (SBBS) [34] 15.43 0.6962 0.9996 0.0004 0.3038 0.1969 0.7362 0.8055
EFIC [21] 20.86 0.6667 0.9991 0.0009 0.3333 0.2502 0.6713 0.7270
IUTIS-1 [22] 18.57 0.8533 0.9954 0.0046 0.1467 0.5339 0.5829 0.5506
IUTIS-2 [23] 16.43 0.7827 0.9987 0.0013 0.2173 0.2259 0.7145 0.6923
IUTIS-3 [24] 9.43 0.6860 0.9998 0.0002 0.3140 0.1638 0.7857 0.9261
Superpixel Strengthen Background Subtraction [25] 16.86 0.6778 0.9995 0.0005 0.3222 0.1847 0.7139 0.8247
M4CD Version 1.0 [26] 9.43 0.7288 0.9997 0.0003 0.2712 0.1639 0.8026 0.8980
SharedModel [31] 14.57 0.7904 0.9990 0.0010 0.2096 0.1936 0.7339 0.7566
C-EFIC [27] 22.00 0.6494 0.9990 0.0010 0.3506 0.2542 0.6275 0.7047
WeSamBE [36] 12.00 0.7382 0.9996 0.0004 0.2618 0.1748 0.7737 0.8371
DeepBS (supervised method) [37] 4.86 0.7979 0.9998 0.0002 0.2021 0.0838 0.8455 0.9082
BMOG [43] 18.14 0.6879 0.9993 0.0007 0.3121 0.2043 0.6932 0.7685
PAWCS [28] 19.00 0.8117 0.9950 0.0050 0.1883 0.6378 0.6450 0.6809
Cascade CNN(supervised method) [32] 4.43 0.9303 0.9997 0.0003 0.0697 0.0584 0.9108 0.8935
Euclidean distance [1] 27.86 0.8340 0.9661 0.0339 0.1660 3.4759 0.4135 0.3565
KDE - ElGammal [2] 25.29 0.8492 0.9857 0.0143 0.1508 1.5119 0.4478 0.3908
GMM | Stauffer & Grimson [3] 25.14 0.7913 0.9882 0.0118 0.2087 1.2760 0.4663 0.4293
GMM | Zivkovic [4] 26.86 0.7786 0.9886 0.0114 0.2214 1.2460 0.4169 0.3494
Mahalanobis distance [5] 29.14 0.3521 0.9972 0.0028 0.6479 0.5272 0.3359 0.6578
CwisarDRP [29] 12.14 0.6221 0.9998 0.0002 0.3779 0.1572 0.7397 0.9273
IUTIS-5 [30] 9.71 0.6777 0.9999 0.0001 0.3223 0.1589 0.7836 0.9414
CwisarDH [6] 16.29 0.6068 0.9997 0.0003 0.3932 0.1853 0.7227 0.8942
Spectral-360 [7] 22.14 0.8815 0.9859 0.0141 0.1185 1.4375 0.5429 0.4982
DCB [35] 37.29 0.2257 0.9765 0.0235 0.7743 2.7228 0.1582 0.3337
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 15.14 0.6109 0.9998 0.0002 0.3891 0.1987 0.7127 0.9035
SC_SOBS [9] 28.43 0.7277 0.9839 0.0161 0.2723 1.7286 0.4880 0.4955
AMBER [10] 13.14 0.6997 0.9997 0.0003 0.3003 0.1793 0.7545 0.8374
CP3-online [11] 32.14 0.5732 0.9946 0.0054 0.4268 0.6797 0.3743 0.3711

Results, for the baseline category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 32.14 0.8137 0.9970 0.0030 0.1863 1.1478 0.8450 0.8870
SuBSENSE [13] 6.00 0.9520 0.9982 0.0018 0.0480 0.3574 0.9503 0.9495
SOBS_CF [14] 17.14 0.9347 0.9978 0.0022 0.0653 0.3912 0.9299 0.9254
RMoG (Region-based Mixture of Gaussians) [15] 27.29 0.7082 0.9981 0.0019 0.2918 1.5935 0.7848 0.9125
SaliencySubsense [16] 7.71 0.9484 0.9982 0.0018 0.0516 0.3774 0.9483 0.9491
AAPSA [17] 20.86 0.9092 0.9979 0.0021 0.0908 0.5826 0.9183 0.9286
Multimode Background Subtraction Version 0 (MBS V0) [18] 17.00 0.9158 0.9979 0.0021 0.0842 0.4361 0.9287 0.9431
Multimode Background Subtraction [19] 17.00 0.9158 0.9979 0.0021 0.0842 0.4361 0.9287 0.9431
M4CD Version 2.0 [33] 15.86 0.9540 0.9976 0.0024 0.0460 0.3927 0.9322 0.9123
GraphCutDiff [20] 36.57 0.7028 0.9960 0.0040 0.2972 1.9757 0.7147 0.8093
Sample based background subtractor (SBBS) [34] 22.14 0.9417 0.9973 0.0027 0.0583 0.4947 0.9192 0.8994
EFIC [21] 24.29 0.9349 0.9971 0.0029 0.0651 0.5223 0.9172 0.9023
IUTIS-1 [22] 17.14 0.9214 0.9979 0.0021 0.0786 0.4538 0.9298 0.9391
IUTIS-2 [23] 29.57 0.7452 0.9978 0.0022 0.2548 1.5115 0.7913 0.9100
IUTIS-3 [24] 6.43 0.9712 0.9981 0.0019 0.0288 0.3002 0.9546 0.9393
Superpixel Strengthen Background Subtraction [25] 10.29 0.9345 0.9982 0.0018 0.0655 0.4176 0.9410 0.9481
M4CD Version 1.0 [26] 21.00 0.9382 0.9976 0.0024 0.0618 0.4934 0.9204 0.9063
SharedModel [31] 5.86 0.9545 0.9982 0.0018 0.0455 0.3344 0.9522 0.9502
C-EFIC [27] 20.71 0.9455 0.9970 0.0030 0.0545 0.5201 0.9309 0.9170
WeSamBE [36] 12.14 0.9422 0.9981 0.0019 0.0578 0.4678 0.9413 0.9422
DeepBS (supervised method) [37] 4.00 0.9517 0.9987 0.0013 0.0483 0.2424 0.9580 0.9660
BMOG [43] 33.43 0.8553 0.9939 0.0061 0.1447 1.4545 0.8301 0.8196
PAWCS [28] 13.43 0.9408 0.9980 0.0020 0.0592 0.4491 0.9397 0.9394
Cascade CNN(supervised method) [32] 1.29 0.9898 0.9989 0.0011 0.0102 0.1405 0.9786 0.9678
Euclidean distance [1] 30.86 0.8385 0.9955 0.0045 0.1615 1.0260 0.8720 0.9114
KDE - ElGammal [2] 23.57 0.8969 0.9977 0.0023 0.1031 0.5499 0.9092 0.9223
GMM | Stauffer & Grimson [3] 34.29 0.8180 0.9948 0.0052 0.1820 1.5325 0.8245 0.8461
GMM | Zivkovic [4] 31.57 0.8085 0.9972 0.0028 0.1915 1.3298 0.8382 0.8993
Mahalanobis distance [5] 24.57 0.3154 0.9991 0.0009 0.6846 2.8698 0.4642 0.9270
CwisarDRP [29] 20.14 0.8580 0.9981 0.0019 0.1420 0.8778 0.8880 0.9347
IUTIS-5 [30] 4.00 0.9680 0.9983 0.0017 0.0320 0.3053 0.9567 0.9464
CwisarDH [6] 20.57 0.8972 0.9980 0.0020 0.1028 0.5679 0.9145 0.9337
Spectral-360 [7] 18.57 0.9616 0.9968 0.0032 0.0384 0.4265 0.9330 0.9065
DCB [35] 26.00 0.7123 0.9982 0.0018 0.2877 1.3771 0.7695 0.9070
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 17.43 0.9513 0.9975 0.0025 0.0487 0.4766 0.9330 0.9170
SC_SOBS [9] 14.14 0.9327 0.9980 0.0020 0.0673 0.3747 0.9333 0.9341
AMBER [10] 28.00 0.8784 0.9973 0.0027 0.1216 0.9233 0.8813 0.8980
CP3-online [11] 27.00 0.8501 0.9972 0.0028 0.1499 0.7725 0.8856 0.9252

Results, for the dynamic background category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 30.57 0.7392 0.9905 0.0095 0.2608 1.1365 0.5953 0.5515
SuBSENSE [13] 14.29 0.7768 0.9994 0.0006 0.2232 0.4042 0.8177 0.8915
SOBS_CF [14] 23.86 0.9014 0.9820 0.0180 0.0986 1.8391 0.6519 0.5953
RMoG (Region-based Mixture of Gaussians) [15] 19.57 0.7892 0.9978 0.0022 0.2108 0.4238 0.7352 0.7288
SaliencySubsense [16] 14.86 0.7676 0.9994 0.0006 0.2324 0.4106 0.8157 0.9000
AAPSA [17] 23.86 0.7083 0.9983 0.0017 0.2917 0.4992 0.6706 0.7336
Multimode Background Subtraction Version 0 (MBS V0) [18] 21.57 0.7637 0.9972 0.0028 0.2363 0.4848 0.7904 0.8606
Multimode Background Subtraction [19] 20.57 0.7641 0.9972 0.0028 0.2359 0.4845 0.7915 0.8651
M4CD Version 2.0 [33] 21.57 0.8518 0.9930 0.0070 0.1482 0.8043 0.6857 0.6841
GraphCutDiff [20] 33.86 0.7693 0.9063 0.0937 0.2307 9.2106 0.5391 0.5357
Sample based background subtractor (SBBS) [34] 13.14 0.7772 0.9994 0.0006 0.2228 0.2682 0.8128 0.9037
EFIC [21] 28.14 0.6667 0.9967 0.0033 0.3333 0.9154 0.5779 0.6849
IUTIS-1 [22] 28.57 0.8811 0.9487 0.0513 0.1189 5.1263 0.4189 0.3305
IUTIS-2 [23] 29.14 0.8027 0.9828 0.0172 0.1973 2.0051 0.5741 0.5564
IUTIS-3 [24] 6.29 0.8778 0.9993 0.0007 0.1222 0.1985 0.8960 0.9239
Superpixel Strengthen Background Subtraction [25] 9.86 0.7875 0.9996 0.0004 0.2125 0.3262 0.8391 0.9185
M4CD Version 1.0 [26] 22.00 0.8451 0.9931 0.0069 0.1549 0.8119 0.6811 0.6806
SharedModel [31] 13.71 0.7597 0.9995 0.0005 0.2403 0.3304 0.8222 0.9198
C-EFIC [27] 29.86 0.6556 0.9952 0.0048 0.3444 1.0825 0.5627 0.6993
WeSamBE [36] 18.29 0.6796 0.9995 0.0005 0.3204 0.6012 0.7440 0.8933
DeepBS (supervised method) [37] 10.14 0.8543 0.9988 0.0012 0.1457 0.2067 0.8761 0.9083
BMOG [43] 14.57 0.9006 0.9966 0.0034 0.0994 0.4040 0.7928 0.7582
PAWCS [28] 7.86 0.8868 0.9989 0.0011 0.1132 0.1917 0.8938 0.9038
Cascade CNN(supervised method) [32] 1.00 0.9798 0.9997 0.0003 0.0202 0.0522 0.9658 0.9528
Euclidean distance [1] 33.00 0.7757 0.9714 0.0286 0.2243 3.0095 0.5081 0.4487
KDE - ElGammal [2] 28.14 0.8012 0.9856 0.0144 0.1988 1.6393 0.5961 0.5732
GMM | Stauffer & Grimson [3] 25.43 0.8344 0.9896 0.0104 0.1656 1.2083 0.6330 0.5989
GMM | Zivkovic [4] 25.86 0.8019 0.9903 0.0097 0.1981 1.1725 0.6328 0.6213
Mahalanobis distance [5] 28.14 0.1237 0.9988 0.0012 0.8763 1.1753 0.1798 0.7451
CwisarDRP [29] 11.43 0.8291 0.9992 0.0008 0.1709 0.2892 0.8487 0.8723
IUTIS-5 [30] 4.86 0.8636 0.9996 0.0004 0.1364 0.1808 0.8902 0.9324
CwisarDH [6] 14.57 0.8144 0.9985 0.0015 0.1856 0.3270 0.8274 0.8499
Spectral-360 [7] 16.43 0.7819 0.9992 0.0008 0.2181 0.3513 0.7766 0.8456
DCB [35] 24.00 0.5803 0.9991 0.0009 0.4197 0.5921 0.6149 0.7632
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 7.43 0.8691 0.9993 0.0007 0.1309 0.1887 0.8792 0.9129
SC_SOBS [9] 23.00 0.8918 0.9836 0.0164 0.1082 1.6899 0.6686 0.6283
AMBER [10] 13.86 0.9177 0.9956 0.0044 0.0823 0.4837 0.8436 0.7990
CP3-online [11] 27.71 0.7260 0.9963 0.0037 0.2740 0.6613 0.6111 0.6122

Results, for the camera jitter category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 33.43 0.7171 0.9477 0.0523 0.2829 6.0218 0.5073 0.3979
SuBSENSE [13] 13.00 0.8243 0.9908 0.0092 0.1757 1.6469 0.8152 0.8115
SOBS_CF [14] 22.86 0.8218 0.9768 0.0232 0.1782 2.8437 0.7150 0.6405
RMoG (Region-based Mixture of Gaussians) [15] 28.00 0.6669 0.9864 0.0136 0.3331 2.6794 0.7010 0.7605
SaliencySubsense [16] 14.57 0.8258 0.9903 0.0097 0.1742 1.6992 0.8071 0.7951
AAPSA [17] 23.57 0.6637 0.9916 0.0084 0.3363 2.1753 0.7207 0.8021
Multimode Background Subtraction Version 0 (MBS V0) [18] 7.00 0.8321 0.9929 0.0071 0.1679 1.5408 0.8367 0.8443
Multimode Background Subtraction [19] 7.00 0.8321 0.9929 0.0071 0.1679 1.5408 0.8367 0.8443
M4CD Version 2.0 [33] 11.00 0.8159 0.9921 0.0079 0.1841 1.4478 0.8231 0.8403
GraphCutDiff [20] 34.00 0.6938 0.9222 0.0778 0.3062 8.4121 0.5489 0.5918
Sample based background subtractor (SBBS) [34] 22.71 0.7322 0.9874 0.0126 0.2678 2.1608 0.7347 0.7950
EFIC [21] 22.57 0.8201 0.9789 0.0211 0.1799 2.7134 0.7125 0.6389
IUTIS-1 [22] 28.14 0.7936 0.9480 0.0520 0.2064 5.8053 0.5997 0.5299
IUTIS-2 [23] 24.57 0.7209 0.9867 0.0133 0.2791 2.4236 0.7165 0.7184
IUTIS-3 [24] 11.57 0.7923 0.9924 0.0076 0.2077 1.5231 0.8139 0.8520
Superpixel Strengthen Background Subtraction [25] 16.14 0.7837 0.9918 0.0082 0.2163 1.6682 0.8004 0.8276
M4CD Version 1.0 [26] 15.71 0.8290 0.9888 0.0112 0.1710 1.7379 0.8051 0.7901
SharedModel [31] 13.43 0.7960 0.9920 0.0080 0.2040 1.6061 0.8141 0.8377
C-EFIC [27] 11.86 0.8458 0.9890 0.0110 0.1542 1.6653 0.8248 0.8157
WeSamBE [36] 16.00 0.7777 0.9921 0.0079 0.2223 1.7091 0.7976 0.8395
DeepBS (supervised method) [37] 2.57 0.8788 0.9957 0.0043 0.1212 0.8994 0.8990 0.9313
BMOG [43] 19.00 0.8363 0.9777 0.0223 0.1637 2.6753 0.7493 0.7293
PAWCS [28] 9.86 0.7840 0.9935 0.0065 0.2160 1.4220 0.8137 0.8660
Cascade CNN(supervised method) [32] 1.00 0.9885 0.9984 0.0016 0.0115 0.2105 0.9758 0.9635
Euclidean distance [1] 34.43 0.7115 0.9456 0.0544 0.2885 6.2957 0.4874 0.3753
KDE - ElGammal [2] 30.14 0.7375 0.9562 0.0438 0.2625 5.1349 0.5720 0.4862
GMM | Stauffer & Grimson [3] 29.14 0.7334 0.9666 0.0334 0.2666 4.2269 0.5969 0.5126
GMM | Zivkovic [4] 32.00 0.6900 0.9665 0.0335 0.3100 4.4057 0.5670 0.4872
Mahalanobis distance [5] 22.00 0.2157 0.9976 0.0024 0.7843 3.4663 0.3358 0.8564
CwisarDRP [29] 15.43 0.7049 0.9936 0.0064 0.2951 1.8416 0.7656 0.8713
IUTIS-5 [30] 8.57 0.8220 0.9925 0.0075 0.1780 1.4389 0.8332 0.8511
CwisarDH [6] 14.00 0.7437 0.9931 0.0069 0.2563 1.7058 0.7886 0.8516
Spectral-360 [7] 23.00 0.6696 0.9906 0.0094 0.3304 2.0855 0.7142 0.8387
DCB [35] 21.29 0.2796 0.9969 0.0031 0.7204 3.3105 0.3669 0.9107
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 21.86 0.7717 0.9866 0.0134 0.2283 2.0787 0.7513 0.7645
SC_SOBS [9] 24.43 0.8113 0.9768 0.0232 0.1887 2.8794 0.7051 0.6286
AMBER [10] 19.43 0.6505 0.9938 0.0062 0.3495 1.9125 0.7107 0.8493
CP3-online [11] 34.71 0.6629 0.9519 0.0481 0.3371 5.9333 0.5207 0.4562

Results, for the intermittent object motion category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 28.57 0.5661 0.9448 0.0552 0.4339 7.1430 0.4497 0.6016
SuBSENSE [13] 14.29 0.6578 0.9915 0.0085 0.3422 3.8349 0.6569 0.7957
SOBS_CF [14] 22.43 0.7641 0.9381 0.0619 0.2359 6.7454 0.5810 0.5464
RMoG (Region-based Mixture of Gaussians) [15] 19.86 0.4488 0.9950 0.0050 0.5512 4.6882 0.5431 0.8026
SaliencySubsense [16] 14.86 0.5626 0.9954 0.0046 0.4374 4.1265 0.6012 0.8149
AAPSA [17] 24.71 0.4912 0.9890 0.0110 0.5088 4.9776 0.5098 0.7139
Multimode Background Subtraction Version 0 (MBS V0) [18] 12.29 0.6386 0.9931 0.0069 0.3614 3.1858 0.7092 0.8201
Multimode Background Subtraction [19] 12.57 0.7418 0.9862 0.0138 0.2582 2.3008 0.7568 0.7827
M4CD Version 2.0 [33] 12.71 0.7153 0.9909 0.0091 0.2847 3.1601 0.6939 0.8055
GraphCutDiff [20] 20.14 0.2923 0.9977 0.0023 0.7077 5.1143 0.4019 0.8315
Sample based background subtractor (SBBS) [34] 20.00 0.7616 0.9399 0.0601 0.2384 6.3180 0.6795 0.6772
EFIC [21] 26.57 0.7416 0.8942 0.1058 0.2584 11.5448 0.5783 0.5634
IUTIS-1 [22] 29.86 0.6050 0.9280 0.0720 0.3950 9.5356 0.5073 0.5485
IUTIS-2 [23] 19.14 0.3735 0.9973 0.0027 0.6265 4.7669 0.4836 0.8374
IUTIS-3 [24] 11.43 0.6987 0.9946 0.0054 0.3013 3.2481 0.7136 0.8146
Superpixel Strengthen Background Subtraction [25] 17.71 0.4840 0.9956 0.0044 0.5160 4.5804 0.5400 0.8255
M4CD Version 1.0 [26] 15.00 0.6178 0.9915 0.0085 0.3822 3.7157 0.6393 0.8000
SharedModel [31] 15.71 0.7182 0.9867 0.0133 0.2818 4.0264 0.6727 0.7587
C-EFIC [27] 22.00 0.8107 0.9172 0.0828 0.1893 8.4615 0.6229 0.5823
WeSamBE [36] 11.86 0.7472 0.9891 0.0109 0.2528 3.2798 0.7392 0.7888
DeepBS (supervised method) [37] 14.86 0.5735 0.9949 0.0051 0.4265 4.1292 0.6098 0.8251
BMOG [43] 24.14 0.5095 0.9871 0.0129 0.4905 4.8434 0.5291 0.6818
PAWCS [28] 7.00 0.7487 0.9945 0.0055 0.2513 2.3536 0.7764 0.8392
Cascade CNN(supervised method) [32] 9.71 0.9840 0.9843 0.0157 0.0160 1.5416 0.8505 0.7821
Euclidean distance [1] 30.14 0.5919 0.9336 0.0664 0.4081 8.9975 0.4892 0.4995
KDE - ElGammal [2] 33.86 0.5035 0.9309 0.0691 0.4965 10.0695 0.4088 0.4609
GMM | Stauffer & Grimson [3] 26.43 0.5142 0.9835 0.0165 0.4858 5.1955 0.5207 0.6688
GMM | Zivkovic [4] 27.00 0.5467 0.9712 0.0288 0.4533 5.4986 0.5325 0.6458
Mahalanobis distance [5] 33.43 0.1607 0.9780 0.0220 0.8393 8.0275 0.2290 0.5098
CwisarDRP [29] 16.29 0.4614 0.9957 0.0043 0.5386 4.2319 0.5626 0.8543
IUTIS-5 [30] 7.29 0.7047 0.9963 0.0037 0.2953 3.0420 0.7296 0.8501
CwisarDH [6] 20.43 0.5549 0.9911 0.0089 0.4451 4.6560 0.5753 0.7417
Spectral-360 [7] 23.43 0.5878 0.9835 0.0165 0.4122 5.3734 0.5609 0.7374
DCB [35] 32.71 0.4415 0.9695 0.0305 0.5585 7.5659 0.3710 0.5291
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 4.29 0.7813 0.9950 0.0050 0.2187 1.6329 0.7891 0.8512
SC_SOBS [9] 22.14 0.7237 0.9613 0.0387 0.2763 5.2207 0.5918 0.5896
AMBER [10] 12.29 0.7617 0.9866 0.0134 0.2383 2.7784 0.7211 0.7530
CP3-online [11] 23.86 0.7826 0.8746 0.1254 0.2174 11.5284 0.6177 0.5631

Results, for the shadow category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision  Average   FPR-S 
Multiscale Spatio-Temporal BG Model [12] 26.14 0.7824 0.9910 0.0090 0.2176 1.6933 0.7918 0.8130 0.5282
SuBSENSE [13] 10.14 0.9419 0.9920 0.0080 0.0581 1.0120 0.8986 0.8646 0.5996
SOBS_CF [14] 29.29 0.8699 0.9828 0.0172 0.1301 2.2579 0.7721 0.7045 0.5899
RMoG (Region-based Mixture of Gaussians) [15] 23.71 0.6680 0.9936 0.0064 0.3320 2.1720 0.7212 0.8073 0.3097
SaliencySubsense [16] 8.86 0.9429 0.9921 0.0079 0.0571 1.0025 0.8994 0.8654 0.5977
AAPSA [17] 26.86 0.8589 0.9855 0.0145 0.1411 1.9218 0.7953 0.7452 0.5877
Multimode Background Subtraction Version 0 (MBS V0) [18] 25.14 0.7762 0.9918 0.0082 0.2238 1.5794 0.7784 0.8063 0.3481
Multimode Background Subtraction [19] 19.57 0.7920 0.9924 0.0076 0.2080 1.4940 0.7968 0.8262 0.3481
M4CD Version 2.0 [33] 10.14 0.9324 0.9922 0.0078 0.0676 1.0796 0.8969 0.8707 0.5749
GraphCutDiff [20] 24.86 0.6578 0.9936 0.0064 0.3422 2.3516 0.7228 0.8271 0.4260
Sample based background subtractor (SBBS) [34] 20.43 0.5981 0.9970 0.0030 0.4019 1.8693 0.7105 0.8934 0.1228
EFIC [21] 24.43 0.8543 0.9908 0.0092 0.1457 1.7066 0.8202 0.8056 0.4846
IUTIS-1 [22] 19.14 0.8748 0.9912 0.0088 0.1252 1.3512 0.8494 0.8291 0.6032
IUTIS-2 [23] 21.29 0.6636 0.9946 0.0054 0.3364 2.2199 0.7393 0.8621 0.4480
IUTIS-3 [24] 11.57 0.9478 0.9914 0.0086 0.0522 1.0410 0.8984 0.8585 0.6031
Superpixel Strengthen Background Subtraction [25] 10.00 0.9299 0.9922 0.0078 0.0701 1.0462 0.8958 0.8701 0.5874
M4CD Version 1.0 [26] 12.43 0.9246 0.9920 0.0080 0.0754 1.1154 0.8913 0.8668 0.5590
SharedModel [31] 14.71 0.9445 0.9910 0.0090 0.0555 1.0876 0.8898 0.8455 0.5937
C-EFIC [27] 15.57 0.9191 0.9920 0.0080 0.0809 1.1933 0.8778 0.8453 0.4791
WeSamBE [36] 8.14 0.9401 0.9923 0.0077 0.0599 1.0187 0.8999 0.8686 0.5532
DeepBS (supervised method) [37] 2.86 0.9584 0.9942 0.0058 0.0416 0.7403 0.9304 0.9092 0.4844
BMOG [43] 21.43 0.8590 0.9909 0.0091 0.1410 1.5975 0.8414 0.8396 0.5372
PAWCS [28] 9.57 0.9172 0.9932 0.0068 0.0828 1.0230 0.8913 0.8710 0.4815
Cascade CNN(supervised method) [32] 1.29 0.9781 0.9973 0.0027 0.0219 0.3500 0.9593 0.9414 0.1566
Euclidean distance [1] 34.00 0.8006 0.9783 0.0217 0.1994 2.8949 0.6786 0.6112 0.5763
KDE - ElGammal [2] 25.43 0.8541 0.9885 0.0115 0.1459 1.6844 0.8030 0.7660 0.6217
GMM | Stauffer & Grimson [3] 30.29 0.7960 0.9871 0.0129 0.2040 2.1951 0.7370 0.7156 0.5352
GMM | Zivkovic [4] 30.86 0.7774 0.9878 0.0122 0.2226 2.1908 0.7322 0.7232 0.5428
Mahalanobis distance [5] 22.71 0.2109 0.9980 0.0020 0.7891 3.6861 0.3353 0.8726 0.0644
CwisarDRP [29] 18.43 0.8298 0.9922 0.0078 0.1702 1.6625 0.8249 0.8551 0.5773
IUTIS-5 [30] 5.14 0.9492 0.9923 0.0077 0.0508 0.9484 0.9084 0.8766 0.5792
CwisarDH [6] 18.14 0.8786 0.9910 0.0090 0.1214 1.2770 0.8581 0.8476 0.5547
Spectral-360 [7] 20.86 0.8898 0.9893 0.0107 0.1102 1.5682 0.8519 0.8187 0.5815
DCB [35] 35.43 0.6635 0.9838 0.0162 0.3365 3.2273 0.6307 0.6612 0.3706
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 15.14 0.9214 0.9918 0.0082 0.0786 1.1305 0.8832 0.8535 0.5005
SC_SOBS [9] 29.71 0.8502 0.9834 0.0166 0.1498 2.3000 0.7786 0.7230 0.6035
AMBER [10] 23.43 0.8297 0.9914 0.0086 0.1703 1.7537 0.8128 0.8098 0.4658
CP3-online [11] 33.86 0.7840 0.9832 0.0168 0.2160 2.5175 0.7037 0.6539 0.5914

Results, for the thermal category.

Click on method name for more details.

Method Average ranking Average     Re  Average     Sp  Average   FPR  Average   FNR  Average   PWC     Average     F-Measure  Average   Precision 
Multiscale Spatio-Temporal BG Model [12] 32.57 0.4102 0.9929 0.0071 0.5898 3.9622 0.5103 0.8403
SuBSENSE [13] 20.43 0.8161 0.9908 0.0092 0.1839 2.0125 0.8171 0.8328
SOBS_CF [14] 20.00 0.6347 0.9953 0.0047 0.3653 1.8021 0.7140 0.8715
RMoG (Region-based Mixture of Gaussians) [15] 22.43 0.3441 0.9991 0.0009 0.6559 5.1222 0.4788 0.9365
SaliencySubsense [16] 23.29 0.5938 0.9955 0.0045 0.4062 3.8988 0.6857 0.8852
AAPSA [17] 18.86 0.6186 0.9957 0.0043 0.3814 1.8316 0.7030 0.8795
Multimode Background Subtraction Version 0 (MBS V0) [18] 20.14 0.8101 0.9908 0.0092 0.1899 1.5315 0.8115 0.8174
Multimode Background Subtraction [19] 17.86 0.8162 0.9920 0.0080 0.1838 1.4289 0.8194 0.8268
M4CD Version 2.0 [33] 13.43 0.6432 0.9981 0.0019 0.3568 2.0839 0.7448 0.9517
GraphCutDiff [20] 22.71 0.4899 0.9976 0.0024 0.5101 4.6735 0.5786 0.9111
Sample based background subtractor (SBBS) [34] 20.29 0.6929 0.9941 0.0059 0.3071 1.6545 0.7499 0.8579
EFIC [21] 12.86 0.8335 0.9944 0.0056 0.1665 1.3553 0.8388 0.8490
IUTIS-1 [22] 15.71 0.6171 0.9972 0.0028 0.3829 1.9653 0.7174 0.9245
IUTIS-2 [23] 21.57 0.4125 0.9987 0.0013 0.5875 4.9923 0.5306 0.9395
IUTIS-3 [24] 12.86 0.7832 0.9945 0.0055 0.2168 1.2552 0.8210 0.8922
Superpixel Strengthen Background Subtraction [25] 22.14 0.6129 0.9956 0.0044 0.3871 3.9737 0.6906 0.8809
M4CD Version 1.0 [26] 16.14 0.6069 0.9980 0.0020 0.3931 2.8701 0.7161 0.9452
SharedModel [31] 20.00 0.8618 0.9845 0.0155 0.1382 1.8656 0.8319 0.8072
C-EFIC [27] 13.57 0.8131 0.9943 0.0057 0.1869 1.3706 0.8349 0.8690
WeSamBE [36] 20.86 0.7727 0.9928 0.0072 0.2273 2.3538 0.7962 0.8554
DeepBS (supervised method) [37] 16.71 0.6637 0.9956 0.0044 0.3363 3.5773 0.7583 0.9257
BMOG [43] 22.71 0.5244 0.9960 0.0040 0.4756 4.3614 0.6348 0.9005
PAWCS [28] 16.43 0.8504 0.9910 0.0090 0.1496 1.4018 0.8324 0.8280
Cascade CNN(supervised method) [32] 12.43 0.9461 0.9931 0.0069 0.0539 1.0478 0.8958 0.8577
Euclidean distance [1] 30.29 0.5111 0.9907 0.0093 0.4889 3.8516 0.6313 0.8877
KDE - ElGammal [2] 16.57 0.6725 0.9955 0.0045 0.3275 1.6795 0.7423 0.8974
GMM | Stauffer & Grimson [3] 26.29 0.5691 0.9946 0.0054 0.4309 4.2642 0.6621 0.8652
GMM | Zivkovic [4] 28.00 0.5542 0.9942 0.0058 0.4458 4.3002 0.6548 0.8706
Mahalanobis distance [5] 22.14 0.0786 0.9999 0.0001 0.9214 6.0413 0.1383 0.9932
CwisarDRP [29] 14.57 0.6778 0.9969 0.0031 0.3222 3.4564 0.7619 0.9116
IUTIS-5 [30] 11.14 0.7990 0.9952 0.0048 0.2010 1.1484 0.8303 0.8969
CwisarDH [6] 15.71 0.7268 0.9949 0.0051 0.2732 1.6199 0.7866 0.8786
Spectral-360 [7] 16.86 0.7238 0.9939 0.0061 0.2762 1.6337 0.7764 0.9114
DCB [35] 31.57 0.5653 0.9882 0.0118 0.4347 2.5188 0.6258 0.8502
FTSG (Flux Tensor with Split Gaussian mdoels)) [8] 10.57 0.7357 0.9960 0.0040 0.2643 1.1823 0.7768 0.9088
SC_SOBS [9] 19.86 0.6003 0.9957 0.0043 0.3997 1.9841 0.6923 0.8857
AMBER [10] 20.43 0.7071 0.9939 0.0061 0.2929 1.6264 0.7597 0.8514
CP3-online [11] 21.00 0.8229 0.9894 0.0106 0.1771 1.6974 0.7917 0.7663

Results for methods [3, 5] have been obtained by the organizing committee using authors' original code. Results for methods [1, 2, 6] have been obtained by the organizing committee using their own implementation or OpenCV. A 5x5 median filter has been applied in a post-processing step.

Metrics:

  • Average ranking across categories : (rank:Bad Weather + rank:Low Framerate + rank:Night Videos + rank:PTZ + rank:Turbulence + rank:Baseline + rank:Dynamic Background + rank:Camera Jitter + rank:Intermittent Object Motion + rank:Shadow + rank:Thermal) / 11
  • Average ranking : (rank:Recall + rank:Spec + rank:FPR + rank:FNR + rank:PWC + rank:FMeasure + rank:Precision) / 7
  • TP : True Positive
  • FP : False Positive
  • FN : False Negative
  • TN : True Negative
  • Re (Recall) : TP / (TP + FN)
  • Sp (Specificity) : TN / (TN + FP)
  • FPR (False Positive Rate) : FP / (FP + TN)
  • FNR (False Negative Rate) : FN / (TP + FN)
  • PWC (Percentage of Wrong Classifications) : 100 * (FN + FP) / (TP + FN + FP + TN)
  • F-Measure : (2 * Precision * Recall) / (Precision + Recall)
  • Precision : TP / (TP + FP)
  • FPR-S : Average False positive rate in hard shadow areas


References:

  • 1. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger. Comparative study of background subtraction algorithms.J. of Elec. Imaging, 19(3):1–12, 2010.
  • 2. A. Elgammal, D. Harwood, and L. Davis, "Non-parametric model for background subtraction," in Proc. Eur. Conf. on Computer Vision, Lect. Notes Comput. Sci. 1843, 751-767 2000.
  • 3. C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking," inProc. Int. Conf. on Computer Vi-sion and Pattern Recognition, Vol. 2, IEEE, Piscataway, NJ (1999)
  • 4. Z. Zivkovic, "Improved adaptive Gaussian mixture model for back-ground subtraction," in Proc. Int. Conf. Pattern Recognition, pp. 28-31, IEEE, Piscataway, NJ 2004
  • 5. Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger. Comparative study of background subtraction algorithms.J. of Elec. Imaging, 19(3):1–12, 2010.
  • 6. M. De Gregorio and M. Giordano "Change Detection with Weightless Neural Networks", in proc of IEEE Workshop on Change Detection, 2014
  • 7. M.Sedky, M.Moniri and C. C. Chibelushi "Spectral-360: A physical-based technique for change detection", in proc of IEEE Workshop on Change Detection, 2014
  • 8. R. Wang, F. Bunyak, G. Seetharaman and K. Palaniappan "Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models", in proc of IEEE Workshop on Change Detection, 2014
  • 9. L. Maddalena, A. Petrosino, "The SOBS algorithm: what are the limits?", in proc of IEEE Workshop on Change Detection, CVPR 2012
  • 10. B. Wang and P. Dudek "A Fast Self-tuning Background Subtraction Algorithm", in proc of IEEE Workshop on Change Detection, 2014
  • 11. Dong Liang, Shun'ichi Kaneko, Manabu Hashimoto, Kenji Iwata, Xinyue Zhao, Co-occurrence probability-based pixel pairs background model for robust object detection in dynamic scenes, Pattern Recognition, Volume 48, Issue 4, April 2015, Pages 1374-1390.
  • 12. Xiqun Lu "A multiscale spatio-temporal background model for motion detection", ICIP 2014
  • 13. St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R., "SuBSENSE : A Universal Change Detection Method with Local Adaptive Sensitivity". Accepted for IEEE Transactions on Image Processing, Nov. 2014
  • 14. L. Maddalena, A. Petrosino "A Fuzzy Spatial Coherence-based Approach to Background/ Foreground Separation for Moving Object Detection". Neural Computing and Applications, Springer London, Vol. 19, pp. 179–186, 2010
  • 15. Varadarajan, S.; Miller, P.; Huiyu Zhou, "Spatial mixture of Gaussians for dynamic background modelling," Advanced Video and Signal Based Surveillance (AVSS), 2013 10th IEEE International Conference on , pp.63,68, 27-30 Aug. 2013
  • 16. Zhenkun huang Ruimin Hu and Shihong Chen, paper has been submitted to CVPR2015
  • 17. G. Ramirez-Alonso y M. Chacon-Murguia, Auto-Adaptive Parallel SOM Architecture with a modular analysis for dynamic object segmentation in videos, Neurocomputing, vol. 175, p. 990–1000. DOI:10.1016/j.neucom.2015.04.118, 2016
  • 18. Hasan Sajid and Sen-Ching Samson Cheung, "Background Subtraction for Static and Moving Camera," Accepted for IEEE International Conference on Image Processing(ICIP) 2015
  • 19. Hasan Sajid and Sen-Ching Samson Cheung, "Universal Multimode Background Subtraction ," Submitted to IEEE Transactions on Image Processing, 2015.
  • 20. “Change detection based on graph cuts”, Alina Miron, Atta Badii, IWSSIP 2015
  • 21. G. Allebosch, F. Deboeverie, P. Veelaert, W. Philips, "EFIC: Edge based Foreground background segmentation and Interior Classification for dynamic camera viewpoints", In Advanced Concepts for Intelligent Vision Systems (ACIVS), Catania, Italy, pp. Accepted, 2015
  • 22. S. Bianco, G. Ciocca, R. Schettini "How far can you get by combining change detection algorithms?". Submitted to IEEE Transactions on Image Processing, 2015. (arXiv:1505.02921)
  • 23. S. Bianco, G. Ciocca, R. Schettini "How far can you get by combining change detection algorithms?". Submitted to IEEE Transactions on Image Processing, 2015. (arXiv:1505.02921)
  • 24. S. Bianco, G. Ciocca, R. Schettini "How far can you get by combining change detection algorithms?". Submitted to IEEE Transactions on Image Processing, 2015. (arXiv:1505.02921)
  • 25. to be submitted
  • 26. To be submitted, 2016
  • 27. G. Allebosch, D. Van Hamme, F. Deboeverie, P. Veelaert, and W. Philips, "C-EFIC: Color and Edge based Foreground background segmentation with Interior Classification", submitted to Computer Vision, Imaging and Computer Graphics Theory and Applications 2015 - Revised Selected Papers
  • 28. P.-L. St-Charles, G.-A. Bilodeau, R. Bergevin, "A Self-Adjusting Approach to Change Detection Based on Background Word Consensus" in IEEE Winter Conference on Applications of Computer Vision (WACV). Big Island, Hawaii, USA, Jan. 6-9, 2015.
  • 29. M. De Gregorio and M. Giordano, "WiSARDRP for change detection in video sequences", submitted to CVPR 2016
  • 30. S. Bianco, G. Ciocca, R. Schettini "How far can you get by combining change detection algorithms?". Submitted to IEEE Transactions on Image Processing, 2015. (arXiv:1505.02921)
  • 31. Yingying Chen; Jinqiao Wang; Hanqing Lu, "Learning sharable models for robust background subtraction," in Multimedia and Expo (ICME), 2015 IEEE International Conference on, pp.1-6, 2015
  • 32. Wang Y., Luo Z. M., and Jodoin P. M. “Interactive Deep Learning Method for Segmenting Moving Objects”, Pattern Recognition Letters, 2016.
  • 33. To be submitted, 2016
  • 34. Arun Varghese, Sreelekha G., "Sample based integrated background subtraction and shadow detection", Submitted to Eurasip Journal of Image and Video Processing, 2017
  • 35. Anonymous
  • 36. Shengqin Jiang and Xiaobo Lu "WeSamBE: A Weight-Sample-Based Method for Background Subtraction" submitted to IEEE Transactions on Circuits and Systems for Video Technology
  • 37. Mohammadreza Babaee, Duc Tung Dinh, Gerhard Rigoll "A Deep Convolutional Neural Network for Background Subtraction". arXiv:1702.01731
  • 43. Martins, I., Carvalho, P., Corte-Real, L., Alba-Castro, J.L. (2017) "BMOG: Boosted Gaussian Mixture Model with Controlled Complexity", accepted at IbPRIA 2017, to be published in Pattern Recognition and Image Analysis. IbPRIA 2017. LNCS, Springer