¹School of Computer Science and Technology, Tianjin University, Tianjin 300072, China
²Information science and technology Department, Tianjin University of Finance and Economics, Tianjin 300222, China
³Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin 300072, China

---

Abstract—PAC-Bayes risk bound integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for deriving some of the tightest generalization bounds. A major issue in practical use of this bound is estimations of unknown prior and posterior distributions of the concept space. In this paper, by formulating the concept space as Reproducing Kernel Hilbert Space (RKHS) using the kernel method, we proposed a refined Markov Chain Monte Carlo (MCMC) sampling algorithm by incorporating feedback information of the simulated model over training examples for simulating posterior distributions of the concept space. Furthermore, we used a kernel density method to estimate their probability distributions in calculating the Kullback- Leibler divergence of the posterior and prior distributions. The experimental results on two artificial data sets show that the simulation is reasonable and effective in practice.

Index Terms—PAC-Bayes Bound, Reproducing Kernel Hilbert Space (RKHS), Markov Chain Monte Carlo (MCMC), Support Vector Machine (SVM)

[PDF]

Cite: Li Tang, Zheng Zhao, Xiu-Jun Gong, "A Refined MCMC Sampling from RKHS for PAC-Bayes Bound Calculation," Journal of Computers vol. 9, no. 4, pp. 930-937, 2014.