File(s) under embargo

284

days

3

hours

until file(s) become available

Faster Inductive Training for Convex Two-Layer Models

thesis
posted on 01.08.2019 by Chandrasekhara Ganesh Jagadeesan
Deep learning is currently one of the most effective approaches in machine learning with applications in image processing, computer vision, and natural language processing. The key technique underpinning its success is the automated learning of latent representation in data using neural networks that employ parametric hidden variables. However, these parameters are typically subject to a non-convex optimization, making the global optimum hard to find. Inductive learning frameworks that guarantee global optimality have been recently developed for two-layer conditional models with a learning strategy based on parametric transfer functions. However, they require optimization over large kernel matrices, hence are slow in training and cannot be scaled to big datasets. In this thesis, we propose a novel optimization strategy that iteratively and greedily expands the subspace of kernels, interlaced with network parameter optimization in the low-rank subspace. The resulting approach significantly speeds up training, while maintaining optimality and accuracy. This allows convex neural networks to be scaled to 10,000 examples for the first time.

History

Advisor

Zhang, Xinhua

Chair

Zhang, Xinhua

Department

Computer Science

Degree Grantor

University of Illinois at Chicago

Degree Level

Masters

Degree name

MS, Master of Science

Committee Member

Gmytrasiewicz, Piotr J Sun, Xiaorui

Submitted date

August 2019

Thesis type

application/pdf

Language

en

Issue date

21/05/2019

Exports