Bayesian Networks and Probabilistic DAG Inference in AI

A full joint probability distribution table over ( n ) boolean random variables requires ( 2^n - 1 ) independent probability values — an exponential storage and computational nightmare for real-world problems with hundreds of variables. Bayesian Networks (also known as Belief Networks or Causal Networks, pioneered by Judea Pearl) solve this by exploiting conditional independence relationships to represent full joint distributions compactly in polynomial space.

Formal Definition of a Bayesian Network

A Bayesian Network is a structured probabilistic representation composed of two parts:

1. A Directed Acyclic Graph (DAG): Nodes represent random variables (discrete or continuous), and directed edges represent direct causal or probabilistic dependencies.
2. Conditional Probability Tables (CPTs): For every node ( X_i ) in the graph, a local conditional distribution specifying the probability of ( X_i ) given each combination of values for its immediate parents: ( P(X_i | ext{Parents}(X_i)) ).

The Master Joint Distribution Factorization

The fundamental theorem of Bayesian networks states that the complete joint probability distribution over all variables ( X_1, dots, X_n ) is equal to the product of local conditional distributions:

The Classic Alarm Network (Judea Pearl)

Consider Pearl's famous burglar alarm scenario:

• A burglar (( B )) or an earthquake (( E )) can set off your home alarm (( A )).
• If the alarm sounds, your neighbors John (( J )) and Mary (( M )) may call you at work.

Calculating a Joint Probability: What is the probability that both John and Mary call, the alarm went off, but there was neither burglary nor earthquake?

D-Separation (Directional Separation)

D-Separation is a formal graph-theoretic criterion determining whether a set of variables ( X ) is conditionally independent of another set ( Y ) given observed evidence ( Z ). A path between ( X ) and ( Y ) is blocked if it contains:

1. Causal Chain (( X ightarrow Z ightarrow Y )): Blocked if ( Z ) is observed.
2. Common Cause / Fork (( X leftarrow Z ightarrow Y )): Blocked if ( Z ) is observed.
3. Common Effect / Collider / V-Structure (( X ightarrow Z leftarrow Y )): The path is blocked if neither ( Z ) nor any of ( Z )'s descendants are observed. Observing the collider ( Z ) activates the path, inducing dependence (an effect known as "explaining away").

The Markov Blanket

In a Bayesian network, a node ( X ) is conditionally independent of all other nodes in the entire network, given its Markov Blanket. The Markov Blanket of node ( X ) consists of:

• Its parents
• Its children
• All other parents of its children (co-parents)

def compute_joint_prob(burglary, earthquake, alarm, john, mary):
    # CPT Definitions
    p_b = 0.001 if burglary else 0.999
    p_e = 0.002 if earthquake else 0.998

    # Alarm CPT lookup
    alarm_table = {
        (True, True): 0.95,
        (True, False): 0.94,
        (False, True): 0.29,
        (False, False): 0.001
    }
    prob_a_true = alarm_table[(burglary, earthquake)]
    p_a = prob_a_true if alarm else (1.0 - prob_a_true)

    # JohnCalls CPT
    p_j = (0.90 if alarm else 0.05) if john else (0.10 if alarm else 0.95)
    # MaryCalls CPT
    p_m = (0.70 if alarm else 0.01) if mary else (0.30 if alarm else 0.99)

    # Master Chain Factorization Product
    joint_probability = p_b * p_e * p_a * p_j * p_m
    return joint_probability

# Case 1: Alarm triggers without burglary or earthquake, both neighbors call
prob_false_alarm = compute_joint_prob(False, False, True, True, True)
print(f"P(J, M, A, ~B, ~E) = {prob_false_alarm:.8f}")

# Case 2: Real Burglary occurs, alarm fires, both call
prob_true_burglary = compute_joint_prob(True, False, True, True, True)
print(f"P(J, M, A, B, ~E)  = {prob_true_burglary:.8f}")