Next >

Handbook on Neural Information Processing

Deep Learning of Representations Introduction Why Learn Representations?Why Distributed Representations?Why Deep?Why Semi-supervised or Unsupervised Learning?Deep Learning of Representations: A Review and Recent Trends Greedy Layerwise Pre-training Undirected Graphical Models and Boltzmann Machines The Restricted Boltzmann Machine The Zoo: Auto-Encoders, Sparse Coding, Predictive Sparse Decomposition, Denoising Auto-Encoders, Score Matching, and More Convolutional Architectures Local Receptive Fields and Weight Sharing Feature Pooling Learning Invariant Feature Sets Dealing with Factors of Variation: Invariant Features Invariance via Sparsity Teasing Apart Explanatory Factors via Slow Features Analysis Learning to Pool Features Beyond Learning Invariant Features Disentangling Factors of Variation On the Importance of Top-Down Connections Conclusion References Recurrent Neural Networks Introduction Architecture Connectionist Network Topologies Non-local Recurrent Globally Feedforward (NLRGF) Networks Specific Architectures Time Delay Neural Networks (TDNN)Williams-Zipser Recurrent Networks Partially-Connected Recurrent Networks State-Space Recurrent Networks Second-Order Recurrent Networks Nonlinear Autoregressive Model with Exogenous Inputs (NARX) Recurrent Networks Memory Delayed Activations as Memory Short-Term Memory and Generic Predictor Types of Memory Kernels Modular Components with Different Parameters Modular Components with Identical Parameters Learning Recurrent Back-Propagation: Learning with Fixed Points Traditional Back-Propagation Algorithm Back-Propagation through Time: Learning with Non-fixed Points Real-Time Recurrent Learning Long-Term Dependencies Modeling Finite State Automata Beyond Finite State Automata Applications Natural Language Processing Identification and Control of Dynamical Systems Conclusion References Supervised Neural Network Models for Processing Graphs Graphs Neural Models for Graph Processing The Graph Neural Network Model Processing DAGs with Recursive Neural Networks DPAGs DAGs-LE Supervised Learning for Graph Neural Networks Learning Objective Learning Procedure for GNNs Gradient Computation for Graph Neural Networks Output network State transition function Learning Procedure for Recursive Neural Networks Forward step Backward step Summary References Topics on Cellular Neural Networks The CNN Concept The Architecture Mathematical Description Other Tasks CNN’s Can Accomplish - The CNN Universal Machine A Particular Architecture The Architecture and the Equations The Decoupling Technique Particular Cases Implementation Issues Using OTA’s Using Log-Domain Techniques Log-Domain Transistor Level Simulations Comparison between 1D OTA and Log-Domain Implementations A “Toy” Application: 1D “Edge” Detection A 2D Log-Domain Architecture Comparison between 2D OTA and Log-Domain Implementations for Applications Using 64x64 Log-Domain Architectures Image Segmentation Two-Grid Coupled CNN’s The Architecture and the Equations The Cells The Interconnections The Equations The Decoupling Technique Boundary Conditions (BC’s) and Their Influence on Pattern Formation Dispersion Curve Turing Pattern Formation Mechanism Boundary Conditions in 2D CNN’s An Application Results Obtained with Ideal Circular Filters Results with CNN Spatial Filters References Approximating Multivariable Functions by Feedforward Neural Nets Introduction Dictionaries and Variable-Basis Approximation The Universal Approximation Property Quadratic Rates of Approximation Geometric Rates of Approximation Approximation of Balls in Variational Norms Best Approximation and Non-continuity of Approximation Tractability of Approximation A Shift in Point-of-View: Complexity and Dimension Measuring Worst-Case Error in Approximation Gaussian RBF Network Tractability Perceptron Network Tractability Discussion Summary of Main Notations References Bochner Integrals and Neural Networks Introduction Variational Norms and Completeness Bochner Integrals Spaces of Bochner Integrable Functions Main Theorem An Example Involving the Bessel Potential Application: A Gamma Function Inequality Tensor-Product Interpretation An Example Involving Bounded Variation on an Interval Pointwise-Integrals vs. Bochner Integrals Evaluation of Bochner Integrals Essential Boundedness Is Needed for the Main Theorem Connection with Sup Norm Some Concluding Remarks Appendix I: Some Banach Space Background Appendix II: Some Key Theorems References Semi-supervised Learning Introduction Semi-supervised Learning Self-Training SSL with Generative Models Semi-supervised SVMs (S3VMs)Semi-supervised Learning with Graphs Semi-supervised Learning with Committees (SSLC)SSLC with Multiple Views SSLC with Single View For Classification For Regression Combination with Active Learning SSL with Graphs SSL with Generative Models SSL with Committees Conclusion References Statistical Relational Learning Introduction Attribute-Value Learning Relational Learning Mapping Relational Data to Attribute-Value Data Summary of This Section Relational Learning: Tasks and Formalisms Inductive Logic Programming Learning from Graphs Multi-relational Data Mining Neural Network Based Approaches to Relational Learning CIL P -2 Relational Neural Networks -3 Graph Neural Networks Statistical Relational Learning Structuring Graphical Models Approaches in the Relational Database Setting Approaches in the Logical Setting Probabilistic Logics Examples of Formalisms Other Approaches General Remarks and Challenges Understanding Commonalities and Differences Parameter Learning and Structure Learning Scalability Recommended Reading References Kernel Methods for Structured Data A Gentle Introduction to Kernel Methods Mathematical Foundations Kernels Supervised Learning with Kernels Kernel Machines for Structured Input SVM for Binary Classification SVM for Regression Smallest Enclosing Hypersphere Kernel Principal Component Analysis Kernels on Structured Data Basic Kernels Kernel Combination Kernels on Discrete Structures Strings Trees Graphs Kernels from Generative Models Dynamic Alignment Kernels Fisher Kernel Marginalized Kernels Kernels on Logical Representations Kernels on Ground Terms Kernels on Proof Trees Learning Kernels Learning Kernel Combinations Learning Logical Kernels Supervised Kernel Machines for Structured Output Conclusions References Multiple Classifier Systems: Theory, Applications and Tools MCS Theory MCS Architectures Conditional Topology Hierarchical (Serial) Topology Multiple (Parallel) Topology Hybrid Topology Combining Rules Majority Voting Weighted Majority Voting Behavior-Knowledge Space Bayesian Combination The Dempster-Shafer Approach Strategies for Constructing a Classifier Ensemble Boosting Bagging Stacked Generalization Random Subspace Method Error-Correcting Output Codes (ECOC)Applications Remote-Sensing Data Analysis Document Analysis Biometrics Figure and Ground Medical Diagnosis Support Chemistry and Biology Time Series Prediction/Analysis Image and Video Analysis Computer and Network Security Miscellanea Tools Tool Categorization Weka KNIME PRTools Conclusions References Self Organisation and Modal Learning: Algorithms and Applications Introduction Snap-Drift Neural Network Description Architecture Algorithm Snap-Drift Self-Organising Map Description Architecture Algorithm Applications Applications of SDNN and SDSOM to Publicly Available Data DescriptionofData Experiments and Results Application of SDNN to E-Learning Conclusions and Future Work References Introduction Bayesian Networks Bayesian Network Theory Bayesian Network Modeling An Example Application: Medical Diagnosis Modeling Reasoning Knowledge Representation Diagnostic Reasoning Discussion Bonaparte: A Bayesian Network for Disaster Victim Identification Likelihood Ratio of Two Hypotheses DNA Profiles A Bayesian Network for Kinship Analysis Allele Probabilities Observations Inference The Application Summary A Petrophysical Decision Support System Probabilistic Modeling The Prior and the Observation Model Bayesian Inference Decision Support The Application Summary Discussion References Relevance Feedback in Content-Based Image Retrieval: A Survey Introduction Content-Based Image Retrieval Low-Level Feature Extraction Similarity Measure Minkowski-Form Distance Kullback-Leibler Divergence Earth Mover’s Distance Classification Methods Artificial Neural Networks SVM Ensemble Learning Bagging Boosting Current Databases Corel Image Gallery TinyImage ImageNet Short-Term Learning RF One-Class Two-Class Multi-class Long-Term Learning RF Latent Semantic Indexing-Based Techniques Correlation-Based Approaches Clustering-Based Algorithms Feature Representation-Based Methods Similarity Measure Modification-Based Approaches Others Summary References Learning Structural Representations of Text Documents in Large Document Collections Introduction Representation of Unstructured or Semi-structured Text Documents General Framework for Processing Graph Structured Data Self Organizing Maps for Structures Graph Neural Networks Clustering of the Wikipedia Dataset Discussion of Results Ranking of Documents Related Work Conclusions References Neural Networks in Bioinformatics Introduction Analyzing DNA Sequences Example Application Conclusion Peptide Sequence Analysis Example Application Conclusion Diagnostic Predictions Example Application Conclusion Conclusion References

Next >