Developing Deep Learning Applications for Life Science and Pharma Industry

Abstract

Deep Learning has boosted artificial intelligence over the past 5 years and is seen now as one of the major technological innovation areas, predicted to replace lots of repetitive, but complex tasks of human labor within the next decade. It is also expected to be ‘game changing’ for research activities in pharma and life sciences, where large sets of similar yet complex data samples are systematically analyzed. Deep learning is currently conquering formerly expert domains especially in areas requiring perception, previously not amenable to standard machine learning. A typical example is the automated analysis of images which are typically produced en-masse in many domains, e. g., in high-content screening or digital pathology. Deep learning enables to create competitive applications in so-far defined core domains of ‘human intelligence’. Applications of artificial intelligence have been enabled in recent years by (i) the massive availability of data samples, collected in pharma driven drug programs (=‘big data’) as well as (ii) deep learning algorithmic advancements and (iii) increase in compute power. Such applications are based on software frameworks with specific strengths and weaknesses. Here, we introduce typical applications and underlying frameworks for deep learning with a set of practical criteria for developing production ready solutions in life science and pharma research. Based on our own experience in successfully developing deep learning applications we provide suggestions and a baseline for selecting the most suited frameworks for a future-proof and cost-effective development.

Key words

drug research - imaging - screening

• References

• 1 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521: 436-444
• 2 Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. DTIC Document 1985: 1–34
• 3 LeCun Y, Bengio Y. Convolutional networks for images, speech, and time series. In: The handbook of brain theory and neural networks. 1995: 3316
• 4 Silver D, Huang A, Maddison CJ. et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016; 529: 484-489
• 5 He K, Zhang X, Ren S et al. Deep residual learning for image recognition. arXiv:1512.03385 2015
• 6 Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 2012; 1097-1105
• 7 Mohamed A, Sainath TN, Dahl G et al. Deep belief networks using discriminative features for phone recognition. Proc Speech and Signal Processing (ICASSP) IEEE Int Conf Acoustics: 5060-5063
• 8 Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions 2016; arXiv:1605.02688
• 9 Ekins S. The next era: Deep learning in pharmaceutical research. Pharmaceutical Research 2016; 33: 2594-2603
• 10 Angermueller C, Paernamaa T, Parts L. et al. Deep learning for computational biology. Molecular Systems Biology 2016; 12: 878
• 11 Gonczarek A, Tomczak JM, Zaręba S. et al. Interaction prediction in structure-based virtual screening using deep learning. Computers in Biology and Medicine 2017; S0010-4825: 30297-4
• 12 Pereira JC, Caffarena ER, dos Santos CN. Boosting docking-based virtual screening with deep learning. Journal of Chemical Information and Modeling 2016; 56: 2495-2506
• 13 Heffernan R, Paliwal K, Lyons J. et al. Improving prediction of secondary structure, local backbone angles, and solvent accessible surface area of proteins by iterative deep learning. Scientific Reports 2015; 5: 11476
• 14 Goh GB, Siegel C, Vishnu A et al. Chemception: A deep neural network with minimal chemistry knowledge matches the performance of expert-developed QSAR/QSPR Models. 2017; arXiv:1706.06689
• 15 Mayr A, Klambauer G, Unterthiner T. et al. Toxicity prediction using deep learning. Frontiers in Environmental Science 2016; 3: 80
• 16 Aliper A, Plis S, Artemov A. et al. Deep learning applications for predicting pharmacological properties of drugs and drug repurposing using transcriptomic data. Molecular Pharmaceutics 2016; 13: 2524-2530
• 17 Chen Y, Li Y, Narayan R. et al. Gene expression inference with deep learning. Bioinformatics 2016; 32: 1832-1839
• 18 Angermueller C, Lee HJ, Reik W. et al. DeepCpG: Accurate prediction of single-cell DNA methylation states using deep learning. Genome biology 2017; 18: 67
• 19 Alipanahi B, Delong A, Weirauch MT. et al. Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nature Biotechnology 2015; 33: 831-838
• 20 Dürr O, Sick B. Single-cell phenotype classification using deep convolutional neural networks. Journal of Biomolecular Screening 2016; 21: 998-1003
• 21 Kraus OZ, Ba JL, Frey BJ. Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics 2016; 32: i52-i59
• 22 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks For Biomedical Image Segmentation. 2015; arXiv:150504597
• 23 Sirinukunwattana K, Pluim JPW, Chen H et al. Gland segmentation in colon histology images: The GlaS challenge contest. 2016; arXiv:160300275
• 24 Stern U, He R, Yang CH. Analyzing animal behavior via classifying each video frame using convolutional neural networks. Scientific Reports 2015; 5: 14351
• 25 Ciresan DC, Meier U, Masci J. et al. Multi-column deep neural network for traffic sign classification. Neural Networks 2012; 32: 333-338
• 26 The HDF Group. Hierarchical Data Format, version 5, 1997-2017. http://www.hdfgroup.org/HDF5/
• 27 Cho K, van Merrienboer B, Gülcehre C et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. 2014; arXiv:1406.1078
• 28 Comparison of deep learning software. (n.d.). In Wikipedia. Retrieved June, 2017, from https://en.wikipedia.org/wiki/Comparison_of_deep_learning_software
• 29 Miguel JP, Mauricio D, Rodriguez G. A Review of software quality models for the evaluation of software products. International Journal of Software Engineering & Applications 2014; 5: 31-53
• 30 Adewumi A, Misra S, Omoregbe N. A review of models for evaluating quality in open source software. IERI Procedia 2013; 4: 88-92
• 31 Bahrampour S, Ramakrishnan N, Schott L et al. Comparative study of deep learning software frameworks. 2015; arXiv:151106435
• 32 Tran K. Deepframeworks – Evaluation of Deep Learning Frameworks GitHub repository 2017; https://github.com/zer0n/deepframeworks
• 33 Chintala S. Convnet-benchmarks – Easy benchmarking of all publicly accessible implementations of convnets. GitHub repository 2017; https://github.com/soumith/convnet-benchmarks
• 34 Chen T, Li M, Li Y et al. MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems. 2015; arXiv:151201274
• 35 Abadi M, Agarwal A, Barham P et al. TensorFlow: Large-scale machine learning on heterogeneous systems. 2015; arXiv:1603.04467
• 36 Jouppi NP, Young C, Patil N et al. In-Datacenter performance analysis of a tensor processing unit. 44th International Symposium on Computer Architecture (ISCA) 2017: 1–12
• 37 Zaharia M, Chowdhury M, Das T et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation 2012; 1–12
• 38 Kovalev V, Kalinovsky A, Kovalev S. Deep Learning with Theano, Torch, Caffe, Tensorflow, and Deeplearning4J: Which one is the best in speed and accuracy? XIII Int. Conf. on Pattern Recognition and Information Processing 2016; Conference Paper pp. 99–103
• 39 Dieleman S, Schlüter J, Raffel C et al. Lasagne: First release. 2015: https://zenodo.org/record/27878
• 40 Chollet F. Keras. GitHub repository 2015: https://github.com/keras-team/keras