Welcome to Deep Bottleneck’s documentation!

Contents

• Background
  • An introduction to artificial neural networks
  • Statistical Dependence
  • How does ‘Statistical Dependance’ help understanding deep learning?
• Literature summary
  • 1. On the information bottleneck theory of deep learning (Saxe 2018)
  • 2. Estimating mutual information
  • 3. SVCCA: singular vector canonical correlation analysis
• Contributing
  • Extending the framework
  • Git workflow
  • Style Guide
  • Experiment workflow
  • Documentation
• User guide
  • Installation
  • How to use the framework
• Glossary
  • Information Theory Basics
  • Mathematical Terms in Tishby’s Experiments
• Experiments
  • Description of cohorts
  • Comparison of infoplanes for different estimators
  • EDGE individiual runs
  • Cohort 10
  • Effect of different initial bias settings for relu
  • Cohort 13
  • Cohort 13
  • Comparison of infoplanes for different estimators and their paramter settings
  • Cohort 4
  • Cohort 5
  • Cohort 9
• Indices and tables
• API Documentation
  • deep_bottleneck package
• Analyses
  • Minimal model
  • Standard vs. Weighted Binning
  • Tishby’s harmonics dataset
  • Explore bias towards entropy induced by activation function
  • Measuring “Information”
• Indices and tables