CC BY-NC-ND 4.0 · Methods Inf Med 2020; 59(01): 001-008
DOI: 10.1055/s-0040-1701615
Original Article

A Method to Extract Feature Variables Contributed in Nonlinear Machine Learning Prediction

Mayumi Suzuki
1  Hitachi, Ltd. Research and Development Group, Tokyo, Japan
Takuma Shibahara
1  Hitachi, Ltd. Research and Development Group, Tokyo, Japan
Yoshihiro Muragaki
2  Faculty of Advanced Techno-Surgery, Institute of Advanced Biomedical Engineering and Science, Graduate School of Medicine, Department of Neurosurgery, Neurological Institute, Tokyo Women’s Medical University, Tokyo, Japan
› Author Affiliations


Background Although advances in prediction accuracy have been made with new machine learning methods, such as support vector machines and deep neural networks, these methods make nonlinear machine learning models and thus lack the ability to explain the basis of their predictions. Improving their explanatory capabilities would increase the reliability of their predictions.

Objective Our objective was to develop a factor analysis technique that enables the presentation of the feature variables used in making predictions, even in nonlinear machine learning models.

Methods A factor analysis technique was consisted of two techniques: backward analysis technique and factor extraction technique. We developed a factor extraction technique extracted feature variables that was obtained from the posterior probability distribution of a machine learning model which was calculated by backward analysis technique.

Results In evaluation, using gene expression data from prostate tumor patients and healthy subjects, the prediction accuracy of a model of deep neural networks was approximately 5% better than that of a model of support vector machines. Then the rate of concordance between the feature variables extracted in an earlier report using Jensen–Shannon divergence and the ones extracted in this report using backward elimination using Hilbert–Schmidt independence criteria was 40% for the top five variables, 40% for the top 10, and 49% for the top 100.

Conclusion The results showed that models can be evaluated from different viewpoints by using different factor extraction techniques. In the future, we hope to use this technique to verify the characteristics of features extracted by factor extraction technique, and to perform clinical studies using the genes, we extracted in this experiment.

Authors' Contributions

All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. M.S. and T. S., in particular, contributed equally to this work.

Publication History

Received: 07 May 2019

Accepted: 18 December 2019

Publication Date:
07 May 2020 (online)

© 2020. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (

Georg Thieme Verlag KG
Stuttgart · New York