Abstract
The problem of domain generalization is to learn, given data from different source distributions, a model that can be expected to generalize well on new target distributions which are only seen through unlabeled samples. In this paper, we study domain generalization as a problem of functional regression. Our concept leads to a new algorithm for learning a linear operator from marginal distributions of inputs to the corresponding conditional distributions of outputs given inputs. Our algorithm allows a source distribution-dependent construction of reproducing kernel Hilbert spaces for prediction, and, satisfies finite sample error bounds for the idealized risk. Numerical implementations and source code are availableFootnote1.
Acknowledgments
We acknowledge the ELLIS Unit Linz, the LIT AI Lab, and the Institute for Machine Learning at the University of Linz. In addition, the research reported in this paper has been partly funded by the Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK), the Federal Ministry for Digital and Economic Affairs (BMDW), and the Province of Upper Austria in the frame of the COMET–Competence Centers for Excellent Technologies Programme and the COMET Module S3AI managed by the Austrian Research Promotion Agency FFG.
Notes
1 Source code: https://github.com/wzell/FuncRegr4DomGen
2 If we equip with , the weakest topology on such that the mapping with is continuous for all bounded and continuous functions and denote by the associated Borel σ- algebra, then becomes a itself measurable space, cf. [5, 10].
3 It holds that , the space of square-integrable functions on . The mapping is well-defined if the kernel k is bounded and it is injective if k is universal [11, 12].
4 The regression function fP is well-defined since and are Polish spaces (as compact subsets of ) and, therefore, every can be factorized in a conditional probability measure and a marginal (w.r.t. ) probability measure , see [13, Theorem 10.2.1].