JAGADEESAN-THESIS-2019.pdf (1021.28 kB)
Faster Inductive Training for Convex Two-Layer Models
thesis
posted on 2019-08-01, 00:00 authored by Chandrasekhara Ganesh JagadeesanDeep learning is currently one of the most effective approaches in machine learning with applications in image processing, computer vision, and natural language processing. The key technique underpinning its success is the automated learning of latent representation in data using neural networks that employ parametric hidden variables. However, these parameters are typically subject to a non-convex optimization, making the global optimum hard to find.
Inductive learning frameworks that guarantee global optimality have been recently developed for two-layer conditional models with a learning strategy based on parametric transfer functions. However, they require optimization over large kernel matrices, hence are slow in training and cannot be scaled to big datasets. In this thesis, we propose a novel optimization strategy that iteratively and greedily expands the subspace of kernels, interlaced with network parameter optimization in the low-rank subspace. The resulting approach significantly speeds up training, while maintaining optimality and accuracy. This allows convex neural networks to be scaled to 10,000 examples for the first time.
History
Advisor
Zhang, XinhuaChair
Zhang, XinhuaDepartment
Computer ScienceDegree Grantor
University of Illinois at ChicagoDegree Level
- Masters
Degree name
MS, Master of ScienceCommittee Member
Gmytrasiewicz, Piotr J Sun, XiaoruiSubmitted date
August 2019Thesis type
application/pdfLanguage
- en
Issue date
2019-05-21Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC