Notes on Uncertainty

Assertions are made by Pearl ‘88 - Probabilistic Reasoning in Intelligent Systems.

Encoding knowledge into rules requires enumerating examples. Positive examples are difficult to satisfy, and ambiguously defined. As a compromise, exceptions can be summarized. Each proposition can be assigned a measure of uncertainty which is aggregated. This uncertainty value is not a truth value, but closer to a counter-example. There is a restrictive assumption of independence. Three schools appear, non-monotonic logic which is non-numerical, probability calculus that is numerical including Demspter-Schaefer, fuzzy logic, and certainty factors, and probability theory, Bayesian probability.

(A^B) -> C
What do these propositions say about the interaction of A and B, and what are their exceptions?

Extensional systems use productions, and Intensional systems use declarative knowledge. In an extensional system uncertainty is defined as a generalized truth value. Certainty values are composable, aggregated with weights. This system relies on the principal of modularity, which is made up of the principal of locality and detachment, which I would characterize as forms of universality. This treats all rules equally. Extensional systems have challenges in the areas of bidirectional inference, retracting conclusions, correlating sources of evidence, as well as abductive reasoning.

P(A|B) > 0
System cannot infer from B to A.
Evidence of A->B removed.

In order for extensional or production systems to work, there must be no cycles present in the graph. This removes any predictive ability, and focuses solely on prescriptive, or diagnostic ability. In order to remove cycles, exceptions can be enumerated, but the principals of locality and detachment in modularity must be removed.

B is true.
If C is true, NOT A is also more probable than A.

The conflict is between modularity and coherence, not the binary truth value of classical logic. Exceptions are not modular, and are not local or detached. P1 -> Q1 ignores P2, P3…Pn. In a graph of propositions, evidence towards an antecedent may be evidence against the consequent. Mitigations include attempts to correlate evidence by means of bounds propagation and user specified combination functions. However, higher order correlations are required beyond a pairwise correlation. Higher order dependencies imply a dynamic relationship dependent on evidence. This cannot be specified prior to experience. Given the formalism of a certainty factor, the domain of extensional systems can be calculated to apply only to Directed Acyclic Graphs.

In Intensional systems, certainty measures are given to sets of worlds and their associated weights. Connectives use set theoretic operations to relate the sets of worlds. These are not composable. Declarative knowledge is semantically sound (almost by definition), it is bidirectional and highly correlated. However, this knowledge cannot be acted upon in itself. A belief network falling into a category of a Bayesian Network, or Constraint / Qualitative Markov Network using Dempster-Schaefer must be used in order to act upon the semantic knowledge.

December 2019

Subscribe to updates here.


Now read this

Document Analysis using Support Vector Machines

This post details the Vector Space Kernel Model for document analysis outlined in Shawe-Taylor and Cristianini. Create Encoded Matrix for each Document # Select Dictionary of Terms Calculate Term Frequency Encode using Dictionary of... Continue →