Learning Deep Low-Dimensional Models from High-Dimensional Data: From Theory to Practice

CVPR 2024 Tutorial

Date: Tuesday, June 18 (full day tutorial)

Location: Room 3 (Summit 442)

Representation Learning SVG Image


Over the past decade, the advent of machine learning and large-scale computing has immeasurably changed the ways we process, interpret, and predict with data in imaging and computer vision. The “traditional” approach to algorithm design, based around parametric models for specific structures of signals and measurements—say sparse and low-rank models—and the associated optimization toolkit, is now significantly enriched with data-driven learning-based techniques, where large-scale networks are pre-trained and then adapted to a variety of specific tasks. Nevertheless, the successes of both modern data-driven and classic model-based paradigms rely crucially on correctly identifying the low-dimensional structures present in real-world data, to the extent that we see the roles of learning and compression of data processing algorithms—whether explicit or implicit, as with deep networks—as inextricably linked.

As such, this tutorial provides a timely tutorial that uniquely bridges low-dimensional models with deep learning in imaging and vision. This tutorial will show how:

  1. Low-dimensional models and principles provide a valuable lens for formulating problems and understanding the behavior of modern deep models in imaging and computer vision; and how
  2. Ideas from low-dimensional models can provide valuable guidance for designing new parameter efficient, robust, and interpretable deep learning models for computer vision problems in practice.

We will begin by introducing fundamental low-dimensional models (e.g., basic sparse and low-rank models) with motivating computer vision applications. Based on these developments, we will discuss strong conceptual, algorithmic, and theoretical connections between low-dimensional structures and deep models, providing new perspectives to understand state-of-the-art deep models in terms of learned representations, generalizability, and transferability. Finally, we will demonstrate that these connections can lead to new principles for designing deep networks learning low-dimensional structures in computer vision, with both clear interpretability and practical benefits. We will conclude with a panel discussion with expert researchers from academia and industry on what role low-dimensional models can and should play in our current age of opaque large language models and foundation models for computer vision.


Yi Ma

UC Berkeley

Qing Qu


Yuqian Zhang


Zhihui Zhu

Ohio State


Wuyang Chen

UC Berkeley

Mojan Javaheripi

Microsoft Research

Liyue Shen


Atlas Wang

UT Austin


The tutorial will take place on Tuesday, June 18th.

Lecture Speaker Time (PT)
Session 1: Understanding Low-Dimensional Representations & Learning in Deep Networks
Lecture 1-1: Introduction to Basic Low-Dimensional Models
(Lecture Abstract)
Yi Ma 9:00-10:00
Lecture 1-2: Understanding Low-Dimensional Representation via Neural Collapse
(Lecture Abstract)
Zhihui Zhu 10:00-11:00
Lecture 1-3: Invariant Low-Dimensional Subspaces of Learning Dynamics
(Lecture Abstract)
Qing Qu 11:15-12:15
Session 2: Designing Deep Networks for Pursuing Low-Dimensional Structures
Lecture 2-1: Low-Dimensional Representation Learning for High-Dimensional Data via the Principle of Compression
(Lecture Abstract)
Yi Ma 13:30-14:30
Lecture 2-2: White-Box Architecture Design via Unrolled Optimization and Compression
(Lecture Abstract)
Yaodong Yu (Yuqian Zhang conflict) 14:45-15:45
Lecture 2-3: White-Box Transformers via Sparse Rate Reduction
(Lecture Abstract)
Sam Buchanan 15:45-16:45
Session 3: Panel Discussion 17:00-18:00


Slides for the tutorial are available at this Dropbox link.