Skip to Main Content
Frequently Asked Questions
Submit an ETD
Global Search Box
Need Help?
Keyword Search
Participating Institutions
Advanced Search
School Logo
Files
File List
main.pdf (4.71 MB)
ETD Abstract Container
Abstract Header
Integral Equations For Machine Learning Problems
Author Info
Que, Qichao
ORCID® Identifier
http://orcid.org/0000-0002-2676-1460
Permalink:
http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998
Abstract Details
Year and Degree
2016, Doctor of Philosophy, Ohio State University, Computer Science and Engineering.
Abstract
Supervised learning algorithms have achieved significant success in the last decade. To further improve learning performance, we still need to have a better understanding of semi-supervised learning algorithms for leveraging a large amount of unlabeled data. In this dissertation, a new approach for semi-supervised learning will be discussed, which takes advantage of unlabeled data information through an integral operator associated with a kernel function. More specifically, several problems in machine learning are formulated as a regularized Fredholm integral equation, which has been well studied in the literature of inverse problems. Under this framework, we propose several simple and easily implementable algorithms with sound theoretical guarantees. First, a new framework for supervised learning is proposed, referred as Fredholm learning. It allows a natural way to incorporate unlabeled data and is flexible on the choice of regularizations. In particular, we connect this new learning framework to the classical algorithm of radial basis function networks, and more specifically, analyze two common forms of regularization procedures for RBF networks, one based on the square norm of coefficients in a network and another one using centers obtained by the k-means clustering. We provide a theoretical analysis of these methods as well as a number of experimental results, pointing out very competitive empirical performance as well as certain advantages over the standard kernel methods in terms of both flexibility (incorporating unlabeled data) and computational complexity. Moreover, the Fredholm learning algorithm could be interpreted as a special form of kernel methods using a data-dependent kernel. Our analysis shows that Fredholm kernels achieve noise suppressing effects under a new assumption for semi-supervised learning, termed the "noise assumption". We also address the problem of estimating the probability density ratio function q/p, which could be used for solving the {\it covariate shift} problem in transfer learning, given the marginal distribution p for training data and q for testing data. Our approach is based on reformulating the problem of estimating q/p as an inverse problem in terms of a Fredholm integral equation. This formulation, combined with the techniques of regularization and kernel methods, leads to a principled kernel-based framework for constructing algorithms and for analyzing them theoretically. The resulting family of algorithms, termed the FIRE algorithm for the Fredholm Inverse Regularized Estimator, is flexible, simple and easy to implement. More importantly, several encouraging experimental results are presented, especially applications to classification and semi-supervised learning within the covariate shift framework. We also show how the hyper-parameters in the FIRE algorithm can be chosen in a completely unsupervised manner.
Committee
Mikhail Belkin (Advisor)
Wang Yusu (Committee Member)
Wang DeLiang (Committee Member)
Lee Yoonkyung (Committee Member)
Pages
154 p.
Subject Headings
Computer Science
Keywords
Machine Learning
;
RBF Networks
;
Supervised Learning
;
Kernel Methods
;
Fredholm Equations
;
Covariate Shift
Recommended Citations
Refworks
EndNote
RIS
Mendeley
Citations
Que, Q. (2016).
Integral Equations For Machine Learning Problems
[Doctoral dissertation, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998
APA Style (7th edition)
Que, Qichao.
Integral Equations For Machine Learning Problems.
2016. Ohio State University, Doctoral dissertation.
OhioLINK Electronic Theses and Dissertations Center
, http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998.
MLA Style (8th edition)
Que, Qichao. "Integral Equations For Machine Learning Problems." Doctoral dissertation, Ohio State University, 2016. http://rave.ohiolink.edu/etdc/view?acc_num=osu1461258998
Chicago Manual of Style (17th edition)
Abstract Footer
Document number:
osu1461258998
Download Count:
2,402
Copyright Info
© 2016, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.