Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure.

• Comparison of biological neurons and artificial neurons.
• Conceptual model of an ANN.
• Activation functions utilized in ANNs.
• Common classes of network architectures.

Mathematical Foundations and Learning mechanisms.

• Review of vector and matrix algebra.
• State-space concepts.
• Optimization principles.
• Error-correction learning methods.
• Memory-based learning approaches.
• Hebbian learning principles.
• Competitive learning mechanisms.

Single layer perceptrons.

• Architectural structure and learning process of perceptrons.
• Introduction to pattern classifiers and Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• Perceptron convergence behavior.
• Inherent limitations of perceptrons.

Feedforward ANN.

• Structure of Multi-layer feedforward networks.
• The Back propagation algorithm.
• Back propagation - training dynamics and convergence.
• Functional approximation via back propagation.
• Practical and design considerations in back propagation learning.

Radial Basis Function Networks.

• Pattern separability and interpolation techniques.
• Regularization Theory.
• Integration of Regularization and RBF networks.
• RBF network design and training procedures.
• Approximation capabilities of RBF networks.

Competitive Learning and Self organizing ANN.

• Standard clustering procedures.
• Learning Vector Quantization (LVQ).
• Competitive learning algorithms and associated architectures.
• Self organizing feature maps.
• Characteristics of feature maps.

Fuzzy Neural Networks.

• Neuro-fuzzy systems.
• Foundations of fuzzy sets and logic.
• Design of fuzzy systems.
• Design of fuzzy ANNs.

Applications

• A review of various Neural Network applications, highlighting their advantages and associated challenges.

DAY -2 MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • Generalities
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC – Dimension
• Bias - Variance tradeoff
• Regularisation techniques
• Over-fitting analysis
• Validation methods
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self Organisation Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and Kernel - induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

Content will be contextualized with topics covered on Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications