Probabilistic Graphical Models(Stanford) - 4

notes Probabilistic_Graphical_Models

Week2 Probabilistic Graphical Models

5. Pairwise Markov Networks

Q: how do you parameterize the undirect graphs?

unnormalized measure: $\widetilde{P}$



An explanation of the question above is what we shown below:

$B$ and $C$ are very likely to agree with each other, and $A$ and $D$ are very likely to agree with each other while $D$ and $C$ are likely not to agree with each other, thus $A$ and $B$ are not likely to agree with each other so $P(A,B) \neq \Phi(A,B)$

Pairwise Markov Networks

2. General Gibbs Distributions

$C_n^2$ edges and each edge has $d^2$ values, so totally the answer is $O(n^2d^2)$, but in total we have $O(d^n)$ parameters.

Gibbs Distribution

Parameters:
$Z$: partition function $-$ change the unnormalized measure $\widetilde{P}$ to probability distribution $\widetilde{P}_\Phi$

Induced Markov Network

(Notice:$P = P_\Phi$ normalized product of factors)

• However, we cannot read the factorization from the graph.

Flow of Influence

Summary

3. Conditional Random Fields

Since sometimes we recompute the relevant features many times, we may not need a model that good. Thus, the correct independent assumptions, add a bunch of edges to capture the correlations.

CRF is just like Gibbs distribution.
partition function that is a function of X, which means for any given X I'm going to have the sum of all the Y's that correspond to that X and then I'm going to construct an additional distribution for Y over Y for any given X.

CRF is related to Logistic Model:

• Thus, the logistic model is a very simple kind model of CRF !!!

Examples:

Summary

4. Independencies in Markov Networks

Theorem:

Converse therom
but only hold for positive distribution $P$. that is $P(\overline{X}) > 0, \forall X$

5. I-maps and perfect maps

Capturing Independencies in P

• $P$ factorized over $G$ $\Rifhtarrow$ $G$ is an I-map for $P$
  
  

minimal I-map is not the best tools for capturing structure in a distribution.

Perfect Map

Example:

The last one is the I-map but it can only capture one of the independece relationship of original network not all.

Example:

Uniqueness

Summary

6. Log-Linear Models

Example:

Ising Model

Metric MRFs

Examples:

7. Shared Features in Log-Linear Models

Example:

Example:

Summary