Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
467 changes: 261 additions & 206 deletions projects/project-notebooks/Microlearning.ipynb

Large diffs are not rendered by default.

38 changes: 38 additions & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,41 @@ git+https://github.com/neuromatch/pyBPL#egg=pybpl
git+https://github.com/neuromatch/MotorNet#egg=motornet
git+https://github.com/ctn-waterloo/sspspace@neuromatch#egg=sspspace
git+https://github.com/mitchellostrow/DSA#egg=dsa-metric

datatops

dm_acme

dm_control

graphviz

ipympl

ipysankeywidget

ipython

memory_profiler

mpl_interactions

nengo

nengo_gui

nengo_spa

pandas

pickleshare

tabulate

torch_optimizer

torchmetrics

trdg

xarray
23 changes: 9 additions & 14 deletions tutorials/Bonus_BonusContent/Bonus_Tutorial2.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@
"cells": [
{
"cell_type": "markdown",
"id": "b00e9b5d",
"metadata": {
"colab_type": "text",
"execution": {},
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
"<a href=\"https://colab.research.google.com/github/neuromatch/NeuroAI_Course/blob/main/tutorials/Bonus_BonusContent/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/neuromatch/NeuroAI_Course/main/tutorials/Bonus_BonusContent/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
]
},
{
Expand Down Expand Up @@ -79,7 +80,10 @@
"outputs": [],
"source": [
"# @title Install dependecies\n",
"!pip install plotly --quiet"
"\n",
"from os import environ as env\n",
"if any(var in env for var in [\"COLAB_RELEASE_TAG\", \"KAGGLE_KERNEL_RUN_TYPE\"]):\n",
" %pip install plotly --quiet"
]
},
{
Expand Down Expand Up @@ -957,15 +961,7 @@
"\n",
"The shallow autoencoder has a compression ratio of **784:2 = 392:1**. The first additional hidden layer has a compression ratio of **2:1**, followed by a hidden layer that sets the bottleneck compression ratio of **32:1**.\n",
"\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.\n",
"\n",
"&nbsp;\n",
"\n",
"This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.\n",
"\n",
"&nbsp;\n",
"\n",
"Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"\n",
"```python\n",
"model[:-2].apply(init_weights_kaiming_normal)\n",
Expand Down Expand Up @@ -1125,8 +1121,7 @@
"The encoder map shows well-separated clusters that correspond to the associated digits in the decoder grid. The decoder grid also shows that the network is robust to digit skewness, i.e., digits leaning to the left or the right are recognized in the same digit class.\n",
"\n",
"**Instructions:**\n",
"* Please execute the cells below\n",
"\n"
"* Please execute the cells below"
]
},
{
Expand Down Expand Up @@ -1730,7 +1725,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
"version": "3.10.20"
}
},
"nbformat": 4,
Expand Down
23 changes: 9 additions & 14 deletions tutorials/Bonus_BonusContent/instructor/Bonus_Tutorial2.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@
"cells": [
{
"cell_type": "markdown",
"id": "dee4365d",
"metadata": {
"colab_type": "text",
"execution": {},
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
"<a href=\"https://colab.research.google.com/github/neuromatch/NeuroAI_Course/blob/main/tutorials/Bonus_BonusContent/instructor/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/neuromatch/NeuroAI_Course/main/tutorials/Bonus_BonusContent/instructor/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
]
},
{
Expand Down Expand Up @@ -79,7 +80,10 @@
"outputs": [],
"source": [
"# @title Install dependecies\n",
"!pip install plotly --quiet"
"\n",
"from os import environ as env\n",
"if any(var in env for var in [\"COLAB_RELEASE_TAG\", \"KAGGLE_KERNEL_RUN_TYPE\"]):\n",
" %pip install plotly --quiet"
]
},
{
Expand Down Expand Up @@ -957,15 +961,7 @@
"\n",
"The shallow autoencoder has a compression ratio of **784:2 = 392:1**. The first additional hidden layer has a compression ratio of **2:1**, followed by a hidden layer that sets the bottleneck compression ratio of **32:1**.\n",
"\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.\n",
"\n",
"&nbsp;\n",
"\n",
"This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.\n",
"\n",
"&nbsp;\n",
"\n",
"Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"\n",
"```python\n",
"model[:-2].apply(init_weights_kaiming_normal)\n",
Expand Down Expand Up @@ -1127,8 +1123,7 @@
"The encoder map shows well-separated clusters that correspond to the associated digits in the decoder grid. The decoder grid also shows that the network is robust to digit skewness, i.e., digits leaning to the left or the right are recognized in the same digit class.\n",
"\n",
"**Instructions:**\n",
"* Please execute the cells below\n",
"\n"
"* Please execute the cells below"
]
},
{
Expand Down Expand Up @@ -1734,7 +1729,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
"version": "3.10.20"
}
},
"nbformat": 4,
Expand Down
29 changes: 11 additions & 18 deletions tutorials/Bonus_BonusContent/student/Bonus_Tutorial2.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,14 @@
"cells": [
{
"cell_type": "markdown",
"id": "eab90967",
"metadata": {
"colab_type": "text",
"execution": {},
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/github/NeuromatchAcademy/course-content/blob/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/NeuromatchAcademy/course-content/main/tutorials/Bonus_Autoencoders/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
"<a href=\"https://colab.research.google.com/github/neuromatch/NeuroAI_Course/blob/main/tutorials/Bonus_BonusContent/student/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/neuromatch/NeuroAI_Course/main/tutorials/Bonus_BonusContent/student/Bonus_Tutorial2.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
]
},
{
Expand Down Expand Up @@ -79,7 +80,10 @@
"outputs": [],
"source": [
"# @title Install dependecies\n",
"!pip install plotly --quiet"
"\n",
"from os import environ as env\n",
"if any(var in env for var in [\"COLAB_RELEASE_TAG\", \"KAGGLE_KERNEL_RUN_TYPE\"]):\n",
" %pip install plotly --quiet"
]
},
{
Expand Down Expand Up @@ -957,15 +961,7 @@
"\n",
"The shallow autoencoder has a compression ratio of **784:2 = 392:1**. The first additional hidden layer has a compression ratio of **2:1**, followed by a hidden layer that sets the bottleneck compression ratio of **32:1**.\n",
"\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.\n",
"\n",
"&nbsp;\n",
"\n",
"This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.\n",
"\n",
"&nbsp;\n",
"\n",
"Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"The choice of hidden layer size aims to reduce the compression rate in the bottleneck layer while increasing the count of trainable parameters. For example, if the compression rate of the first hidden layer doubles from **2:1** to **4:1**, the count of trainable parameters halves from 667K to 333K.This deep autoencoder's performance may be further improved by adding additional hidden layers and by increasing the count of trainable parameters in each layer. These improvements have a diminishing return due to challenges associated with training under high parameter count and depth. One option explored in the *Bonus* section is to add a first hidden layer with 2x - 3x the input size. This size increase results in millions of parameters at the cost of longer training time.Weight initialization is particularly important in deep networks. The availability of large datasets and weight initialization likely drove the deep learning revolution of 2010. We'll implement Kaiming normal as follows:\n",
"\n",
"```python\n",
"model[:-2].apply(init_weights_kaiming_normal)\n",
Expand Down Expand Up @@ -1035,8 +1031,7 @@
"execution": {}
},
"source": [
"[*Click for solution*](https://github.com/neuromatch/course-content-template/tree/main/tutorials/Bonus_BonusContent/solutions/Bonus_Tutorial2_Solution_0d50004f.py)\n",
"\n"
"[*Click for solution*](https://github.com/neuromatch/NeuroAI_Course/tree/main/tutorials/Bonus_BonusContent/solutions/Bonus_Tutorial2_Solution_0d50004f.py)"
]
},
{
Expand Down Expand Up @@ -1087,8 +1082,7 @@
"The encoder map shows well-separated clusters that correspond to the associated digits in the decoder grid. The decoder grid also shows that the network is robust to digit skewness, i.e., digits leaning to the left or the right are recognized in the same digit class.\n",
"\n",
"**Instructions:**\n",
"* Please execute the cells below\n",
"\n"
"* Please execute the cells below"
]
},
{
Expand Down Expand Up @@ -1356,8 +1350,7 @@
"execution": {}
},
"source": [
"[*Click for solution*](https://github.com/neuromatch/course-content-template/tree/main/tutorials/Bonus_BonusContent/solutions/Bonus_Tutorial2_Solution_f82e3b9c.py)\n",
"\n"
"[*Click for solution*](https://github.com/neuromatch/NeuroAI_Course/tree/main/tutorials/Bonus_BonusContent/solutions/Bonus_Tutorial2_Solution_f82e3b9c.py)"
]
},
{
Expand Down Expand Up @@ -1661,7 +1654,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.18"
"version": "3.10.20"
}
},
"nbformat": 4,
Expand Down
14 changes: 9 additions & 5 deletions tutorials/W1D1_Generalization/W1D1_Intro.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
"cells": [
{
"cell_type": "markdown",
"id": "4e3d2b26-a059-4683-8bfd-2499a50eb346",
"id": "7dba9cef",
"metadata": {
"execution": {}
"colab_type": "text",
"execution": {},
"id": "view-in-github"
},
"source": [
"<a href=\"https://colab.research.google.com/github/neuromatch/NeuroAI_Course/blob/main/tutorials/W1D1_Generalization/student/W1D1_Intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/neuromatch/NeuroAI_Course/main/tutorials/W1D1_Generalization/student/W1D1_Intro.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
"<a href=\"https://colab.research.google.com/github/neuromatch/NeuroAI_Course/blob/main/tutorials/W1D1_Generalization/W1D1_Intro.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a> &nbsp; <a href=\"https://kaggle.com/kernels/welcome?src=https://raw.githubusercontent.com/neuromatch/NeuroAI_Course/main/tutorials/W1D1_Generalization/W1D1_Intro.ipynb\" target=\"_parent\"><img src=\"https://kaggle.com/static/images/open-in-kaggle.svg\" alt=\"Open in Kaggle\"/></a>"
]
},
{
Expand All @@ -33,7 +35,9 @@
"source": [
"# @title Install and import feedback gadget\n",
"\n",
"!pip install vibecheck datatops --quiet\n",
"from os import environ as env\n",
"if any(var in env for var in [\"COLAB_RELEASE_TAG\", \"KAGGLE_KERNEL_RUN_TYPE\"]):\n",
" %pip install vibecheck datatops --quiet\n",
"\n",
"from vibecheck import DatatopsContentReviewContainer\n",
"def content_review(notebook_section: str):\n",
Expand Down Expand Up @@ -232,7 +236,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.19"
"version": "3.10.20"
}
},
"nbformat": 4,
Expand Down
Loading