[关闭]
@nanmeng 2016-05-19T08:43:41.000000Z 字数 4208 阅读 1065

Probabilistic Graphical Models(Stanford) - 1

notes Probabilistic_Graphical_Models


Pre-Class

Why factors?
* Fundamental building block for defining distributions in high-dimensional spaces
* Set of basic operations for manipulating these probability distributions

Week1 Bayesian Network Fundamentals

1. Semantics & Factorization

An example for bayesian network:
PGM1_1

The calculation rule: chain rule

PGM1_2

The illustration of calculation the joint distribution:
(how to calculate with the value)
PGM1_3

Bayesian Network:

  • A directed acyclic graph(DAG)
  • For each node a CPD

A trick in BN:

As shown below, in the calculation of Bayesian Network, the summation can be calculated on ''part'' of the whole equation.
PGM1_4

P Factorizes over G:

PGM1_5
An example of Genetic Inheritance
PGM1_6

2. Reasoning Patterns

  • Causal Reasoning(top down)
  • Evidential Reasoning(bottom up)
  • Intercausal Reasoning(flow information between two causes)

An illustration of the Intercausal Reasoning is hard to figure out:
PGM1_7

PGM1_8
Student aces the SAT contribute to the increase prob. of and the prob. of .
PGM1_9

3. Flow of Probabilistic Influence

An example of active trail:
PGM1_10

A trail is active if: it has no v-structures like
Notice: v-structure is a structure that two nodes point to the same one. (like ).
Then the rules for what condition influence the information flow is like what shown below.
PGM1_11
(The final line in the table of the picture above is: and all of its descendants not in | either if or one of its descendants is in )

Summary

PGM1_12

Independencies in BNs
  • Types of three-variable structures: chain (aka causal trail or evidential trail), common parent
    (aka common cause), v-structure (aka common effect)
  • Property: A variable is independent of its non-descendants given its parents
  • Property: A variable is independent of all other variables in the network given its Markov blanket, which consists of its parents, its children, and its co-parents

materials:Probabilistic Graphical Models 10-708 Recitation 1 Handout

4. Conditional Independence

basic for conditional independence & symbol
PGM1_13
The definition of conditional independence
PGM1_14
An example of conditional independence:
PGM1_15
when people have not been told that the coin is a fair coin, then the prob of second time toss the coin and get head is higher given the first time get head. However, when people have been told that this is a fair coin, the two times tosses are independent with each other.

5. Independencies in Bayesian Networks

Recap:
PGM1_16
A new question:
PGM1_17

  • Theorem: If factorizes over , and then satisfies

PGM1_18

PGM1_19

  • red line: all the non-descendants of Letter
  • descendants of Letter are Job, Happy.

I-maps

PGM1_20
Example:
PGM1_21
G1 is the I-map of P1, while G2 is the I-map of P1 or P2.

I-Maps
• I-Map: A graph G is an I-map for a distribution P if
• Minimal I-Map: A graph G is a minimal I-map for a distribution P if you cannot remove any
edges from G and have it still be an I-map for P
• Perfect I-Map: A graph G is a perfect I-map for a distribution P if
• I-Equivalence: Two graphs G1 and G2 are I-equivalent if

PGM1_22

Illustrate one example in the picture: We know that when knowing the parrent of a node then it is independent with its non-descendants.Thus
Thus, the first equation in the picture is equall to the second equation.

Summary

PGM1_23

6. Naive Bayes

  • What independence assumption does the Naive Bayes model make?
    Given the class variable, each observed variable is independent of the other observed variables.

PGM1_24
If given the class, the variables are independent with each other.
PGM1_25
green: prior probabilities of two classes
blue: odds ratio
An example of Bernoulli Naive Bayes for text
PGM1_26
PGM1_27

Summary

PGM1_28

7. Medical Diagnosis

PGM1_29

8. Knowledge Engineering Example -SAMIAM

SIAMIAM

Relative materials

  • Markov Random Fields
  • 3.1 Independencies in MRFs
    • Two variables and are independent if there is no active trail between them; a trail is active if it doesn’t contain any observed variables.
    • Property: A variable is independent of all other variables in the network given its Markov blanket, which consists of its direct neighbors in the graph.
  • 3.2 Parameterization of MRFs
    • Markov random fields are parameterized by a set of factors defined over cliques in the graph; factors are not distributions as they do not have to sum to 1.
    • The joint probability distribution of the variables in an MRF can be written in factorized form as a normalized product of factors, i.e. where is the set of variables in the ith clique, and is the partition function.

SMO(Sequential Minimal Optimization)

添加新批注
在作者公开此批注前,只有你和作者可见。
回复批注