Software by Mark Schmidt and Students

code.zip - Compilation of updated and interoperable versions of many of the Matlab codes on this webpage.

minFuncSO - Julia code implementing plane-search methods for minimizing functions of the form f(Ax) (also includes a Julia implementation of minFunc).

SLS - Python code implementing stochastic gradient with a stochastic line-search to set the step size.
VB-MixEF - Matlab code for variational Bayes with a mixture of exponential family approximating distribution.
FPEI - Python code for parameter estimation in nucleic acid reactions in the Multistrand simulator.
BCDLaplacian - Python code for block coordinate descent updates using Laplacian-structured Hessian approximations
GP_DRF - Python code for deep Gaussian proceess models with variable-sized inputs.

matLearn - Matlab implementations of a wide variety of the most commonly-used machine learning algorithms.
style-swap - LuaTorch code implementing a fast artistic style transfer method that allows new styles.
Kaczmarz - Python code for randomized and greedy Kaczmarz methods for solving linear systems.

SAG4CRF - Matlab mex files implementing a non-uniform stochastic average gradient method for L2-regularized chain-structured conditional random fields.
practicalSVRG - Matlab mex files for stochastic variance-reduced gradient with growing batch size and identifying support vectors.
GCD - Python code implementing Gauss-Southwell and Gauss-Southwell-Lipshcitz coordinate descent methods for linear models.

SSL - Python files implementing semi-supervised learning (Yarowksy and co-training) approaches for some text-labeling problems (word-sense disambiguation and named-entity recognition)

SAG - Matlab mex files implementing the stochastic average gradient method for L2-regularized logistic regression.
Updated: prettyPlot - Added the ability to directly place labels on lines, and show filled-in error area.

prettyPlot - A wrapper that uses Matlab's plot function to make plots that look nicer in papers.
Updated: minFunc - Faster core L-BFGS code, enhanced derivative-checking options, and some minor changes.
Updated: thesis - Added the canonical and spectral binary parameterizations to the log-linear model (LLM) code.
Examples: QNST - A set of examples showing how to use QNST to solve a variety of problems.
Examples: TMP - A set of examples showing how to use TMP to solve a variety of problems.

batching - An optimizer that combines an L-BFGS line-search method with a growing batch-size strategy.
alphaBeta - Functions for approximate pairwise energy minimization with attractive energies.
thesis - Quasi-Newton methods for non-differentiable optimization, and structure learning in graphical models.
inexactProx - Code to reproduce the experiments in the inexact proximal-gradient paper.
Updated: UGM - Major update, including a complete change to the way paramter estimation is performed.

Updated: L1General - New methods added, mostly focusing on limited-memory quasi-Newton and first-order methods.
Updated: PQN - Fixed an important bug regarding the initialization of the SPG iterations.

UGM - Functions implementing exact and approximate decoding, inference, sampling, and parameter estimation in discrete undirected graphical models.
minConf - Functions for optimization of differentiable real-valued multivariate functions with simple constraints.
DCG - A probabilistic model of interventional data that allows cycles.
PQN - Code for optimization of (costly) differentiable multivariate functions over simple convex sets.
Updated: minFunc - another set of minor changes in response to feedback, but also some non-trivial changes to the conjugate gradient and dense quasi-Newton methods.
Examples: PQN - A set of examples showing how to use PQN to solve a variety of problems.
Examples: L1General - A set of examples showing how to use L1General to solve a variety of problems.

crfChain - Learning/inference/decoding/sampling in chain-structured conditional random fields with categorical features.
pdfCropAll - Runs pdfcrop on every .pdf in the current directory and replaces the original files.
UGMlearn - Matlab code for structure learning in discrete-state undirected graphical models (Markov Random Fields and Conditional Random Fields) using Group L1-regularization.
Gsparse - Matlab functions implementing spectral projected gradient methods for optimization with a Group L1-norm constraint.
Updated: L1General - Added new methods and variants of all methods that don't require the exact Hessian.
Updated: lasso - Added a variant of the active set method that allows warm-starting.
Updated: minFunc - Most of the changes were minor and in response to user feedback.
Examples: minFunc - A set of examples showing how to use minFunc to solve a variety of problems.

L1General - Functions implementing strategies for minimizing differentiable functions subject to L1 regularization.
L1precision - Matlab function for L1-regularized estimation of the precision matrix in Gaussian graphical models.
DAGLearn Matlab functions for structure learning in Gaussian/Sigmoid directed acyclic graphical models.

CRF2D - Matlab code for inference and learning in 2D binary conditional random fields, using approximate likelihoods (made obsolete by UGM).
blogreg - Functions for MCMC simulation of binary probit/logistic regression posterior distributions over parameters.

lasso - Functions implementing a variety of the methods available to solve 'LASSO' regression (and basis selection) problems.
minFunc - Function for unconstrained optimization of differentiable real-valued multivariate functions.

Mark Schmidt > Software