• Perfect sampling via coupling from the past ( link)
• Stochastic processes on graphs (look on mathscinet for Russ Lyons, Robin Pemantle, Yuval Peres ).
• Stochastic calculus and integration (how does one integrate wrt a martingale?)
• Belief structures (look up Michael Goldstein in mathscinet. Warning - philosophical!).
• A study of finite exchangeability, and contrast to infinite exchangeability (start with Diaconis and Freedman, Annals of Probability 1980).
• Maximum entropy distributions and exponential families (start with Jaynes (1968) IEEE Trans. Systems, Science, and Cybernetics. There is a wealth of literature that follows, or search on the web).
• Consistency of nonparametric Bayesian procedures (Freedman 1963, 1965, Diaconis and Freedman 1986, Barron, Schervish, Wasserman 1999).
• limiting behvior of estimates in misspecified models (Berk 1966, White (in Econometrika) 1982).