Probabilistic Graphical Models(CMU)-1

Probabilistic_Graphical_Models CMU notes

The class link: Probabilistic Graphical Models(Spring 2014) - Eric Xing

Lecture notes

Directed graphical models -- basics

• Consider an arbitrary directed (acyclic) graph, where
  each node in the graph corresponds to a random
  variable (scalar or vector):
  
  We want to associate a probability distribution
  $P (A, B, C, D, E, F )$ with this graph, and we want
  all of our calculations to respect this distribution
  e.g., $$P (F|A, B) =\frac{\sum_C\sum_D\sum_EP(A, B, C, D, E, F)}{\sum_C\sum_D\sum_E\sum_FP(A, B, C, D, E, F)}$$

Examples:

The constants are from $X_1$ to $X_5$ since we can see them directly, and the variables are from $X_6$ to $X_9$ since we cannot see them directly and the structure of the model(how the nodes combine with each other also influence the loss function of the whole model so this can also be seen as variables of model.)

An example:

Some merits:

The plot shows what the MLE or bayesian learning do:

So what is a Graphical Model?

Two types of GMs:

Markov blanket: the green nodes

Markov Random Fields

The classification problem can be seen as the two nodes BM, one for data and the other node for label.

Relationship between different models:

Some questions:

Reading materials notes

Reading book An Introduction to Graphical Models

• There are two components to any graphical model:
  
  • the qualitative specification
  • the quantitative specification
• Where does the qualitative specication come from?
  
  • prior knowledge of causal relationships
  • prior knowledge of modular relationships
  • assessment from experts
  • learning from data
  • we simply like a certain architecture (e.g., a layered graph)
    

Reading Chapter 3 notes

• Bayesian assumption: