Crowdsourcing Utilizing Subgroup Structure of Latent Factor Modeling

References

• Aydin, B. I., Yilmaz, Y. S., Li, Y., Li, Q., Gao, J., and Demirbas, M. (2014), “Crowdsourcing for Multiple-Choice Question Answering,” in AAAI, pp. 2946–2953. Citeseer. DOI: 10.1609/aaai.v28i2.19016.
   Google Scholar
• Berend, D., and Kontorovich, A. (2014), “Consistency of Weighted Majority Votes,” in Advances in Neural Information Processing Systems (vol. 27), pp. 3446–3454, Curran Associates, Inc.
   Google Scholar
• Bondell, H. D., and Reich, B. J. (2008), “Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with Oscar,” Biometrics, 64, 115–123. DOI: 10.1111/j.1541-0420.2007.00843.x.
   PubMed Web of Science ®Google Scholar
• Borromeo, R. M., and Toyama, M. (2015), “Automatic vs. Crowdsourced Sentiment Analysis,” in Proceedings of the 19th International Database Engineering & Applications Symposium, pp. 90–95, New York, NY. DOI: 10.1145/2790755.2790761.
   Google Scholar
• Cao, P., Xu, Y., Kong, Y., and Wang, Y. (2019), “Max-mig: An Information Theoretic Approach for Joint Learning from Crowds,” arXiv preprint arXiv:1905.13436.
   Google Scholar
• Cape, J., Tang, M., and Priebe, C. E. (2019), “The Two-to-Infinity Norm and Singular Subspace Geometry with Applications to High-Dimensional Statistics,” The Annals of Statistics, 47, 2405–2439. DOI: 10.1214/18-AOS1752.
   Web of Science ®Google Scholar
• Dawid, A. P., and Skene, A. M. (1979), “Maximum Likelihood Estimation of Observer Error-Rates using the EM Algorithm,” Journal of the Royal Statistical Society, Series C, 28, 20–28. DOI: 10.2307/2346806.
   Google Scholar
• Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009), “Imagenet: A Large-Scale Hierarchical Image Database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255.
   Google Scholar
• Deng, K., Han, S., Li, K. J., and Liu, J. S. (2014), “Bayesian Aggregation of Order-based Rank Data,” Journal of the American Statistical Association, 109, 1023–1039. DOI: 10.1080/01621459.2013.878660.
   Web of Science ®Google Scholar
• Engel, J. (1988), “Polytomous Logistic Regression,” Statistica Neerlandica, 42, 233–252. DOI: 10.1111/j.1467-9574.1988.tb01238.x.
   Google Scholar
• Fan, J., and Cheng, J. (2018), “Matrix Completion by Deep Matrix Factorization,” Neural Networks, 98, 34–41. DOI: 10.1016/j.neunet.2017.10.007.
   PubMed Web of Science ®Google Scholar
• Fisher, R. A. (1936), “The Use of Multiple Measurements in Taxonomic Problems,” Annals of Eugenics, 7, 179–188. DOI: 10.1111/j.1469-1809.1936.tb02137.x.
   Google Scholar
• Foncubierta Rodríguez, A., and Müller, H. (2012), “Ground Truth Generation in Medical Imaging: A Crowdsourcing-based Iterative Approach,” in Proceedings of the ACM Multimedia 2012 Workshop on Crowdsourcing for Multimedia, pp. 9–14, New York, NY.
   Google Scholar
• Ghosh, A., Kale, S., and McAfee, P. (2011), “Who Moderates the Moderators. Crowdsourcing Abuse Detection in User-Generated Content,” in Proceedings of the 12th ACM Conference on Electronic Commerce, pp. 167–176, New York, NY.
   Google Scholar
• Hoaglin, D. C., and Welsch, R. E. (1978), “The Hat Matrix in Regression and Anova,” The American Statistician, 32, 17–22. DOI: 10.2307/2683469.
   Web of Science ®Google Scholar
• Ibrahim, S., Fu, X., Kargas, N., and Huang, K. (2019), “Crowdsourcing via Pairwise Co-occurrences: Identifiability and Algorithms,” in Advances in Neural Information Processing Systems (Vol. 32), pp. 7847–7857.
   Google Scholar
• Kim, H.-C., and Ghahramani, Z. (2012), Bayesian Classifier Combination (Vol. 22), pp. 619–627, La Palma: Canary Islands.
   Google Scholar
• Ko, C.-Y., Lin, R., Li, S., and Wong, N. (2019), “Misc: Mixed Strategies Crowdsourcing,” arXiv preprint arXiv:1905.07394.
   Google Scholar
• Kovashka, A., Russakovsky, O., Fei-Fei, L., and Grauman, K. (2016), “Crowdsourcing in Computer Vision,” arXiv preprint arXiv:1611.02145.
   Google Scholar
• Li, Y., Du, N., Liu, C., Xie, Y., Fan, W., Li, Q., Gao, J., and Sun, H. (2017), “Reliable Medical Diagnosis from Crowdsourcing: Discover Trustworthy Answers from Non-experts,” in Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 253–261, New York, NY.
   Google Scholar
• Li, Y., Rubinstein, B., and Cohn, T. (2019), “Exploiting Worker Correlation for Label Aggregation in Crowdsourcing,” in Proceedings of the 36th International Conference on Machine Learning (Vol. 97), pp. 3886–3895, Long Beach, CA.
   Google Scholar
• Ma, S., and Huang, J. (2017), “A Concave Pairwise Fusion Approach to Subgroup Analysis,” Journal of the American Statistical Association, 112, 410–423. DOI: 10.1080/01621459.2016.1148039.
   Web of Science ®Google Scholar
• Maaten, L. v. d., and Hinton, G. (2008), “Visualizing Data using t-SNE,” Journal of Machine Learning Research, 9, 2579–2605.
   Web of Science ®Google Scholar
• MacLean, D. L., and Heer, J. (2013), “Identifying Medical Terms in Patient-Authored Text: A Crowdsourcing-based Approach,” Journal of the American Medical Informatics Association, 20, 1120–1127. DOI: 10.1136/amiajnl-2012-001110.
   PubMed Web of Science ®Google Scholar
• Mitry, D., Peto, T., Hayat, S., Blows, P., Morgan, J., Khaw, K.-T., and Foster, P. J. (2015), “Crowdsourcing as a Screening Tool to Detect Clinical Features of Glaucomatous Optic Neuropathy from Digital Photography,” PloS One, 10, e0117401. DOI: 10.1371/journal.pone.0117401.
   PubMed Web of Science ®Google Scholar
• Nitzan, S., and Paroush, J. (1982), “Optimal Decision Rules in Uncertain Dichotomous Choice Situations,” International Economic Review, 23, 289–297.
   Web of Science ®Google Scholar
• Rohe, K., Qin, T., and Yu, B. (2016), “Co-clustering Directed Graphs to Discover Asymmetries and Directional Communities,” Proceedings of the National Academy of Sciences, 113, 12679–12684.
   PubMed Web of Science ®Google Scholar
• Shen, X. (1998), “On the Method of Penalization,” Statistica Sinica, 8, 337–357.
   Web of Science ®Google Scholar
• Snow, R., O’Connor, B., Jurafsky, D., and Ng, A. Y. (2008), “Cheap and Fast–but is it Good. Evaluating Non-expert Annotations for Natural Language Tasks,” in Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 254–263, USA.
   Google Scholar
• Stone, M. (1978), “Cross-Validation: A Review,” Statistics: A Journal of Theoretical and Applied Statistics, 9, 127–139.
   Google Scholar
• Street, W. N., Wolberg, W. H., and Mangasarian, O. L. (1993), “Nuclear Feature Extraction for Breast Tumor Diagnosis,” in Biomedical Image Processing and Biomedical Visualization (Vol. 1905), pp. 861–870.
   Google Scholar
• Su, H., Deng, J., and Fei-Fei, L. (2012), “Crowdsourcing Annotations for Visual Object Detection,” in Human Computation - Papers from the 2012 AAAI Workshop, Technical Report, pp. 40–46.
   Google Scholar
• Sugar, C. A., and James, G. M. (2003), “Finding the Number of Clusters in a Dataset: An Information-Theoretic Approach,” Journal of the American Statistical Association, 98, 750–763.
   Web of Science ®Google Scholar
• Tang, X., Xue, F., and Qu, A. (2020), “Individualized Multidirectional Variable Selection,” Journal of the American Statistical Association, 116, 1280–1296.
   Web of Science ®Google Scholar
• Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005), “Sparsity and Smoothness via the Fused Lasso,” Journal of the Royal Statistical Society, Series B, 67, 91–108.
   Google Scholar
• Tibshirani, R., Walther, G., and Hastie, T. (2001), “Estimating the Number of Clusters in a Data Svia the Gap Statistic,” Journal of the Royal Statistical Society, Series B, 63, 411–423.
   Google Scholar
• Venanzi, M., Guiver, J., Kazai, G., Kohli, P., and Shokouhi, M. (2014), “Community-based Bayesian Aggregation Models for Crowdsourcing,” in Proceedings of the 23rd International Conference on World Wide Web, pp. 155–164.
   Google Scholar
• Wang, H., Li, R., and Tsai, C.-L. (2007), “Tuning Parameter Selectors for the Smoothly Clipped Absolute Deviation Method,” Biometrika, 94, 553–568.
   PubMed Web of Science ®Google Scholar
• Welinder, P., Branson, S., Perona, P., and Belongie, S. J. (2010), “The Multidimensional Wisdom of Crowds,” in Advances in Neural Information Processing Systems (Vol. 23), pp. 2424–2432.
   Google Scholar
• Wen, Z., and Yin, W. (2013), “A Feasible Method for Optimization with Orthogonality Constraints,” Mathematical Programming, 142, 397–434.
   Web of Science ®Google Scholar
• Whitehill, J., Wu, T.-f., Bergsma, J., Movellan, J. R., and Ruvolo, P. L. (2009), “Whose Vote Should Count More: Optimal Integration of Labels from Labelers of Unknown Expertise,” in Advances in Neural Information Processing Systems (Vol. 22), pp. 2035–2043.
   Google Scholar
• Zhang, J., He, X., and Wang, J. (2021), “Directed Community Detection with Network Embedding,” Journal of the American Statistical Association, 117, 1809–1819.
   Web of Science ®Google Scholar
• Zhang, Y., Chen, X., Zhou, D., and Jordan, M. I. (2016), “Spectral Methods Meet EM: A Provably Optimal Algorithm for Crowdsourcing,” The Journal of Machine Learning Research, 17, 3537–3580.
   Web of Science ®Google Scholar