Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List

ETD Abstract Container

Abstract Header

Integral Equations For Machine Learning Problems

Abstract Details

2016, Doctor of Philosophy, Ohio State University, Computer Science and Engineering.
Supervised learning algorithms have achieved significant success in the last decade. To further improve learning performance, we still need to have a better understanding of semi-supervised learning algorithms for leveraging a large amount of unlabeled data. In this dissertation, a new approach for semi-supervised learning will be discussed, which takes advantage of unlabeled data information through an integral operator associated with a kernel function. More specifically, several problems in machine learning are formulated as a regularized Fredholm integral equation, which has been well studied in the literature of inverse problems. Under this framework, we propose several simple and easily implementable algorithms with sound theoretical guarantees. First, a new framework for supervised learning is proposed, referred as Fredholm learning. It allows a natural way to incorporate unlabeled data and is flexible on the choice of regularizations. In particular, we connect this new learning framework to the classical algorithm of radial basis function networks, and more specifically, analyze two common forms of regularization procedures for RBF networks, one based on the square norm of coefficients in a network and another one using centers obtained by the k-means clustering. We provide a theoretical analysis of these methods as well as a number of experimental results, pointing out very competitive empirical performance as well as certain advantages over the standard kernel methods in terms of both flexibility (incorporating unlabeled data) and computational complexity. Moreover, the Fredholm learning algorithm could be interpreted as a special form of kernel methods using a data-dependent kernel. Our analysis shows that Fredholm kernels achieve noise suppressing effects under a new assumption for semi-supervised learning, termed the "noise assumption". We also address the problem of estimating the probability density ratio function q/p, which could be used for solving the {\it covariate shift} problem in transfer learning, given the marginal distribution p for training data and q for testing data. Our approach is based on reformulating the problem of estimating q/p as an inverse problem in terms of a Fredholm integral equation. This formulation, combined with the techniques of regularization and kernel methods, leads to a principled kernel-based framework for constructing algorithms and for analyzing them theoretically. The resulting family of algorithms, termed the FIRE algorithm for the Fredholm Inverse Regularized Estimator, is flexible, simple and easy to implement. More importantly, several encouraging experimental results are presented, especially applications to classification and semi-supervised learning within the covariate shift framework. We also show how the hyper-parameters in the FIRE algorithm can be chosen in a completely unsupervised manner.
Mikhail Belkin (Advisor)
Wang Yusu (Committee Member)
Wang DeLiang (Committee Member)
Lee Yoonkyung (Committee Member)
154 p.

Recommended Citations

Citations

  • Que, Q. (2016). Integral Equations For Machine Learning Problems [Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998

    APA Style (7th edition)

  • Que, Qichao. Integral Equations For Machine Learning Problems. 2016. Ohio State University, Doctoral dissertation. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998.

    MLA Style (8th edition)

  • Que, Qichao. "Integral Equations For Machine Learning Problems." Doctoral dissertation, Ohio State University, 2016. http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998

    Chicago Manual of Style (17th edition)