-
支持向量机建模中的一个关键和难点问题是自由参数的设置. 不同于以往应用残差的简单统计量选取最佳模型的方法, 本文提出通过检 验模型在训练集上的拟合残差是否不含冗余信息作为选择自由参数的依据. 进一步提出应用全向相关函数(omni-directional correlaton function, ODCF)检验残差信息冗余并给出应用方法,并从理论分析和数值仿真两 方面给出该方法正确性的证明.在两个典型的非线性时间序列(年 均太阳黑子数和Mackey-Glass数据)上进行了实验,实验结果优于相关 文献记载及基于校验集方法的预测性能.The selection of hyper-parameters is a crucial point in support vector machine modeling. Different from previous method of choosing an optimal model by using basic statistics of residuals in, the new approach selects hyper-parameters by checking whether there is redundant information in residual sequence. Furthermore, omni-directional correlation function (ODCF) is used to test redundancy in residual, and the accuracy of the method is proved by theoretical analysis and numerical simulation. Experiments conducted on benchmark time series, annual sunspot number and Mackey-Glass time series, indicating that the proposed method has better performance than the recorded in the literature.
-
Keywords:
- support vector machines /
- hyper-parameter /
- residual /
- ODCF
[1] Box G E P, Jenkins G M, Reinsel G C 2005 Time Series Analysis: Forecasting and control. (Posts & Telecom Press)
[2] Yang H, Wang R 2011 Acta Phys. Sin. 60 070508 (in Chinese) [杨红, 王瑞 2011 60 070508]
[3] Chen Q, Ren X M 2010 Acta Phys. Sin. 59 2310 (in Chinese) [陈强, 任雪梅 2010 59 2310]
[4] Shi Z W; Han M 2007 IEEE Trans. Neural Netw. 18 359
[5] Cai J W, Hu S S, Tao H F 2007 Acta Phys. Sin. 56 6820 (in Chinese) [蔡俊伟, 胡寿松, 陶洪峰 2007 56 6820]
[6] Cui W Z, Zhu C C, Bao W X, Liu J H 2005 Acta Phys. Sin. 54 3009 (in Chinese) [崔万照, 朱长纯, 保文星, 刘君华 2005 54 3009]
[7] Ye M Y, Wang X D 2004 Chin. Phys. 13 454
[8] Vapnik V N 1999 The Nature of Statistical Learning Theory (2nd Ed.) (New York: Springer)
[9] Vapnik V N 1998 Statistical Learning Theory (New York: Wiley)
[10] Sapankevych N I, Sankar R 2009 IEEE Comput. Intell. M. 5 28
[11] Bayro-Corrochano E J, Arana-Daniel N 2010 IEEE Trans. Neural Netw. 21 1731
[12] Grinblat G L, Uzal L C, Ceccatto H A, Granitto P M 2011 IEEE Trans. Neural netw. 22 37
[13] Yang X W, Zhang G Q, Lu J, Ma J 2011 IEEE Trans. on Fuzzy Syst. 19 105
[14] Cristianini N, Shawe-Taylor J (Translated by Li G Z, Wang M and Zeng H J) 2005 An introduction to support vector machines and other kernel-based learning methods (Beijing: Publishing House of Electronics Industry) (in Chinese) [Cristianini N, Shawe-Taylor J著 李国正, 王猛, 曾华军 译 2005 支持向量机导论 (北京:电子工业出版社)]
[15] Schölkopf B, Bartlett P, Smola A, and Williamson R Proceedings of ICANN'98, Perspectives in Neural Compution (Berlin: Springer L. Niklasson, M Bodén, and T Ziemke, Ed.) p111
[16] Kwok J T, Tsang I W 2003 IEEE Trans. Neural Netw. 14 544
[17] Smola A, Murata N, Schölkopf B and Muller K 1998 Proceedings of ICANN (Berlin: Springer Verlag) p105
[18] Mattera D, Haykin S 1999 Support Vector Machines for dynamic reconstruction of a chaotic system in: Advances in Kernel Methods: Support Vector Machine (Cambridge: MIT Press)
[19] herkassky V, Mulier F 1998 Learning from Data: Concepts, Theory and Methods (New York: John Wiley & Sons)
[20] Scholkopf B, Burges J, Smola A 1999 Advances in Kernel Methods: Support Vector Machine (Cambridge: MIT Press)
[21] Vladimir C, Ma Y Q 2004 Neural Networks 17 113
[22] Cristianini N, Kandola J, Elissee A, ShaweTaylor J 2006 J. Mach. Learn. Res. 194 205
[23] Zhang S Q, Jia J, Gao M, Han X 2010 Acta. Phys. Sin. 59 1576 (in Chinese) [张淑清, 贾健, 高敏, 韩叙 2010 59 1576]
[24] Rong H N, Zhang G X, Jin W D 2006 J. Sys. Simu. 18 3204 (in Chinese) [荣海娜, 张葛祥, 金炜东 2006 系统仿真学报 18 3204]
[25] Ljung L 1999 System Identification-Theory for the User (Prentice-Hall, Inc)
[26] Zhang L F, Zhu Q M, Longden A 2009 IEEE Trans. neural netw. 20 1
[27] Mao K Z, Billings S A 2000 Int. J. Control 73 132
[28] Yu Y H, Song J D 2009 Journal of Electronics & Information Technology 31 2220(in Chinese)[于艳华, 宋俊德 2009电子与信息学报 31 2220]
[29] Billings S A, Zhu Q M 1994 Int. J. Control 60 1107
[30] Prichard D, Theiler J 1995 Physica D 84 476
[31] Brock W A, Dechert W D, Scheinkman J 1996 Econometric Review 15 197
[32] Zhang L F, Zhu Q M, Longden A 2007 Int. J. Syst. Sci. 38 47
[33] Zhu Q M, Zhang L F, Longden A 2007 Automatica 43 1519
[34] Cao L J 2003 Neurocomputing 51 321
[35] Weigend A S, Huberman B A, Rumelhart D E 1990 Int. J. Neural Systems 1 193
[36] Tong H, Lim K S 1980 J. Roy. Statist. Soc. 42 245
[37] Ralavola L, Alche-Buc F 2003 Proceeding of NIPS Vancouver, Canada 2003 p981
-
[1] Box G E P, Jenkins G M, Reinsel G C 2005 Time Series Analysis: Forecasting and control. (Posts & Telecom Press)
[2] Yang H, Wang R 2011 Acta Phys. Sin. 60 070508 (in Chinese) [杨红, 王瑞 2011 60 070508]
[3] Chen Q, Ren X M 2010 Acta Phys. Sin. 59 2310 (in Chinese) [陈强, 任雪梅 2010 59 2310]
[4] Shi Z W; Han M 2007 IEEE Trans. Neural Netw. 18 359
[5] Cai J W, Hu S S, Tao H F 2007 Acta Phys. Sin. 56 6820 (in Chinese) [蔡俊伟, 胡寿松, 陶洪峰 2007 56 6820]
[6] Cui W Z, Zhu C C, Bao W X, Liu J H 2005 Acta Phys. Sin. 54 3009 (in Chinese) [崔万照, 朱长纯, 保文星, 刘君华 2005 54 3009]
[7] Ye M Y, Wang X D 2004 Chin. Phys. 13 454
[8] Vapnik V N 1999 The Nature of Statistical Learning Theory (2nd Ed.) (New York: Springer)
[9] Vapnik V N 1998 Statistical Learning Theory (New York: Wiley)
[10] Sapankevych N I, Sankar R 2009 IEEE Comput. Intell. M. 5 28
[11] Bayro-Corrochano E J, Arana-Daniel N 2010 IEEE Trans. Neural Netw. 21 1731
[12] Grinblat G L, Uzal L C, Ceccatto H A, Granitto P M 2011 IEEE Trans. Neural netw. 22 37
[13] Yang X W, Zhang G Q, Lu J, Ma J 2011 IEEE Trans. on Fuzzy Syst. 19 105
[14] Cristianini N, Shawe-Taylor J (Translated by Li G Z, Wang M and Zeng H J) 2005 An introduction to support vector machines and other kernel-based learning methods (Beijing: Publishing House of Electronics Industry) (in Chinese) [Cristianini N, Shawe-Taylor J著 李国正, 王猛, 曾华军 译 2005 支持向量机导论 (北京:电子工业出版社)]
[15] Schölkopf B, Bartlett P, Smola A, and Williamson R Proceedings of ICANN'98, Perspectives in Neural Compution (Berlin: Springer L. Niklasson, M Bodén, and T Ziemke, Ed.) p111
[16] Kwok J T, Tsang I W 2003 IEEE Trans. Neural Netw. 14 544
[17] Smola A, Murata N, Schölkopf B and Muller K 1998 Proceedings of ICANN (Berlin: Springer Verlag) p105
[18] Mattera D, Haykin S 1999 Support Vector Machines for dynamic reconstruction of a chaotic system in: Advances in Kernel Methods: Support Vector Machine (Cambridge: MIT Press)
[19] herkassky V, Mulier F 1998 Learning from Data: Concepts, Theory and Methods (New York: John Wiley & Sons)
[20] Scholkopf B, Burges J, Smola A 1999 Advances in Kernel Methods: Support Vector Machine (Cambridge: MIT Press)
[21] Vladimir C, Ma Y Q 2004 Neural Networks 17 113
[22] Cristianini N, Kandola J, Elissee A, ShaweTaylor J 2006 J. Mach. Learn. Res. 194 205
[23] Zhang S Q, Jia J, Gao M, Han X 2010 Acta. Phys. Sin. 59 1576 (in Chinese) [张淑清, 贾健, 高敏, 韩叙 2010 59 1576]
[24] Rong H N, Zhang G X, Jin W D 2006 J. Sys. Simu. 18 3204 (in Chinese) [荣海娜, 张葛祥, 金炜东 2006 系统仿真学报 18 3204]
[25] Ljung L 1999 System Identification-Theory for the User (Prentice-Hall, Inc)
[26] Zhang L F, Zhu Q M, Longden A 2009 IEEE Trans. neural netw. 20 1
[27] Mao K Z, Billings S A 2000 Int. J. Control 73 132
[28] Yu Y H, Song J D 2009 Journal of Electronics & Information Technology 31 2220(in Chinese)[于艳华, 宋俊德 2009电子与信息学报 31 2220]
[29] Billings S A, Zhu Q M 1994 Int. J. Control 60 1107
[30] Prichard D, Theiler J 1995 Physica D 84 476
[31] Brock W A, Dechert W D, Scheinkman J 1996 Econometric Review 15 197
[32] Zhang L F, Zhu Q M, Longden A 2007 Int. J. Syst. Sci. 38 47
[33] Zhu Q M, Zhang L F, Longden A 2007 Automatica 43 1519
[34] Cao L J 2003 Neurocomputing 51 321
[35] Weigend A S, Huberman B A, Rumelhart D E 1990 Int. J. Neural Systems 1 193
[36] Tong H, Lim K S 1980 J. Roy. Statist. Soc. 42 245
[37] Ralavola L, Alche-Buc F 2003 Proceeding of NIPS Vancouver, Canada 2003 p981
计量
- 文章访问数: 6693
- PDF下载量: 608
- 被引次数: 0
下载:
