Probabilistic Graphical Models(CMU)-6

Probabilistic_Graphical_Models CMU notes

The class link: Probabilistic Graphical Models(Spring 2014) - Eric Xing

Lecture notes 6

Recap

Linear Regression

Logistic Regression

Conditional Random Fields vs. HMM
the CRF actually undirect the HMM model and what's more join all the observations($X$) together.

* benefits: sometimes the properties of $X$ are very global and depending on the configuration of behavior of a single node or pair of nodes to be in a particular way. (eg: the $X$ is about faces. this property is hard to be included in HMM.)

Some tricks on how to define the potentials:

Exponential family

An example of explaining why Multivariate Gaussian Distribution belonging to Exponential family

and here is another example:

• The reason why people like exponential family:
  
  • one of the reason is Moment generating property
    
• Moment $\Leftrightarrow Canonical parameters$ 1-to-1 mapping
  

How to estimate the parameters of exponential family(极大似然)

Sufficiency

• Once you get the sufficient statistic $T(X)$ you can safely ignore the raw data $X$, since $T(X)$ separate the $X$ and $\theta$, like what follows shows:
  

Other examples

Conjugate prior

Conjugate prior: if the natural parameter has the same form of its sufficient statistic in its prior

Generalized Linear Models(GLIMS)

Others

• stochastic gradient ascent = least mean squares (LMS) algorithm:

--