Quantile regression for large-scale data via sparse exponential transform method

ABSTRACT

In recent decades, quantile regression has received much more attention from academics and practitioners. However, most of existing computational algorithms are only effective for small or moderate size problems. They cannot solve quantile regression with large-scale data reliably and efficiently. To this end, we propose a new algorithm to implement quantile regression on large-scale data using the sparse exponential transform (SET) method. This algorithm mainly constructs a well-conditioned basis and a sampling matrix to reduce the number of observations. It then solves a quantile regression problem on this reduced matrix and obtains an approximate solution. Through simulation studies and empirical analysis of a 5% sample of the US 2000 Census data, we demonstrate efficiency of the SET-based algorithm. Numerical results indicate that our new algorithm is effective in terms of computation time and performs well for large-scale quantile regression.

KEYWORDS:

• Quantile regression
• large-scale data
• sparse exponential transform
• sampling

AMS SUBJECT CLASSIFICATIONS:

• 62J07
• 62D05

Acknowledgements

The authors are grateful to the Editor, Associate Editor and two anonymous referees for the constructive and valuable comments which lead to a significant improvement of the paper.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

C. X. Jiang  http://orcid.org/0000-0002-6900-8049

Additional information

Funding

This research was supported by the National Natural Science Foundation of China (grant numbers 71671056 and 71490725), the National Social Science Foundation of China (grant number 15BJY008), the Humanity and Social Science Foundation of Ministry of Education of China (grant number 14YJA790015) and the Social Science Planning Fund Program of Shangdong Province (grant number 18DTJJ01).