Commit df87d84e authored by christian.foerster's avatar christian.foerster

restructering tutorial, and adding better description

parent a5c79194
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Observer Pattern"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"from __future__ import annotations\n",
"from abc import ABC, abstractmethod\n",
"from random import randrange\n",
"from typing import List\n",
"\n",
"\n",
"class Subject(ABC):\n",
" \"\"\"\n",
" The Subject interface declares a set of methods for managing subscribers.\n",
" \"\"\"\n",
"\n",
" @abstractmethod\n",
" def attach(self, observer: Observer) -> None:\n",
" \"\"\"\n",
" Attach an observer to the subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
" @abstractmethod\n",
" def detach(self, observer: Observer) -> None:\n",
" \"\"\"\n",
" Detach an observer from the subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
" @abstractmethod\n",
" def notify(self) -> None:\n",
" \"\"\"\n",
" Notify all observers about an event.\n",
" \"\"\"\n",
" pass\n",
"\n",
"\n",
"class ConcreteSubject(Subject):\n",
" \"\"\"\n",
" The Subject owns some important state and notifies observers when the state\n",
" changes.\n",
" \"\"\"\n",
"\n",
" _state: int = None\n",
" \"\"\"\n",
" For the sake of simplicity, the Subject's state, essential to all\n",
" subscribers, is stored in this variable.\n",
" \"\"\"\n",
"\n",
" _observers: List[Observer] = []\n",
" \"\"\"\n",
" List of subscribers. In real life, the list of subscribers can be stored\n",
" more comprehensively (categorized by event type, etc.).\n",
" \"\"\"\n",
"\n",
" def attach(self, observer: Observer) -> None:\n",
" print(\"Subject: Attached an observer.\")\n",
" self._observers.append(observer)\n",
"\n",
" def detach(self, observer: Observer) -> None:\n",
" self._observers.remove(observer)\n",
"\n",
" \"\"\"\n",
" The subscription management methods.\n",
" \"\"\"\n",
"\n",
" def notify(self) -> None:\n",
" \"\"\"\n",
" Trigger an update in each subscriber.\n",
" \"\"\"\n",
"\n",
" print(\"Subject: Notifying observers...\")\n",
" for observer in self._observers:\n",
" observer.update(self)\n",
"\n",
" def some_business_logic(self) -> None:\n",
" \"\"\"\n",
" Usually, the subscription logic is only a fraction of what a Subject can\n",
" really do. Subjects commonly hold some important business logic, that\n",
" triggers a notification method whenever something important is about to\n",
" happen (or after it).\n",
" \"\"\"\n",
"\n",
" print(\"\\nSubject: I'm doing something important.\")\n",
" self._state = randrange(0, 10)\n",
"\n",
" print(f\"Subject: My state has just changed to: {self._state}\")\n",
" self.notify()\n",
"\n",
"\n",
"class Observer(ABC):\n",
" \"\"\"\n",
" The Observer interface declares the update method, used by subjects.\n",
" \"\"\"\n",
"\n",
" @abstractmethod\n",
" def update(self, subject: Subject) -> None:\n",
" \"\"\"\n",
" Receive update from subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
"\n",
"\"\"\"\n",
"Concrete Observers react to the updates issued by the Subject they had been\n",
"attached to.\n",
"\"\"\"\n",
"\n",
"\n",
"class ConcreteObserverA(Observer):\n",
" def update(self, subject: Subject) -> None:\n",
" if subject._state < 3:\n",
" print(\"ConcreteObserverA: Reacted to the event\")\n",
"\n",
"\n",
"class ConcreteObserverB(Observer):\n",
" def update(self, subject: Subject) -> None:\n",
" if subject._state == 0 or subject._state >= 2:\n",
" print(\"ConcreteObserverB: Reacted to the event\")\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Subject: Attached an observer.\n",
"Subject: Attached an observer.\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 7\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 8\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 9\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n"
]
}
],
"source": [
"subject = ConcreteSubject()\n",
"\n",
"observer_a = ConcreteObserverA()\n",
"subject.attach(observer_a)\n",
"\n",
"observer_b = ConcreteObserverB()\n",
"subject.attach(observer_b)\n",
"\n",
"subject.some_business_logic()\n",
"subject.some_business_logic()\n",
"\n",
"subject.detach(observer_a)\n",
"\n",
"subject.some_business_logic()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
This diff is collapsed.
......@@ -4,273 +4,205 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Numpy\n",
"## Speeding up your python code\n",
"\n",
"For all of you that love matlab numpy will be very familiar. (https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html)\n",
"If you design your own python algorithms, they tend to be very slow. Mainly because python ist not a typed language.\n",
"\n",
"Why use numpy? Because it's very fast! (https://stackoverflow.com/questions/7596612/benchmarking-python-vs-c-using-blas-and-numpy)\n",
"You have different options to improve upon performance:\n",
"- write your algorithm in c/c++ or fortran and call it from python\n",
"- use Numba for small projects (Numba uses LLVM to power Just-In-Time compilation of array oriented Python code)\n",
"- use Cython for bigger projects, as it is more versatile\n",
"\n",
"**A more comprehensive tutorial:** (https://docs.scipy.org/doc/numpy/user/quickstart.html)"
"A more detailled comparison can be found [here](https://rushter.com/blog/numba-cython-python-optimization/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Arrays Examples"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"arr = np.array([3,4,5,6])\n",
"rng = np.arange(100)\n",
"ones = np.ones(1000000)\n",
"zeros = np.zeros((100,100)) # 2d array shape passed as tuple\n",
"repeat = np.repeat(2,1000)"
"import timeit\n",
"\n",
"import numpy as np\n",
"import numba\n",
"import cython"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Shapes"
"### Simple example!"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"repeat.shape"
"def my_algorithm(array):\n",
" summe = 0\n",
" for num in array:\n",
" summe += num\n",
" return summe / len(array)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# 3d array\n",
"repeat.reshape((10,10,10))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# flatten array to 1d\n",
"repeat.flatten()"
"array = np.array(range(100_000))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Indexing and Slicing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rng[:5]"
"This is the slow plain python vesion"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"rng[79]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"27.8 ms ± 1.9 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
]
}
],
"source": [
"rng[3::5]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rng_2d = rng.reshape((10,10))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# all lines, starting with column 5\n",
"rng_2d[:,4:] "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# first 5 lines, all column\n",
"rng_2d[:-5,:] "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rng_2d[3:5,6:9]"
"%timeit my_algorithm(array)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Boolean Indexing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# 2d boolean array\n",
"boolean_arr = rng_2d > 45\n",
"boolean_arr"
"Using **Numba** to speed things up"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"# losing shape of course!\n",
"rng_2d[boolean_arr]"
"my_algorithm_numba = numba.jit(my_algorithm)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 21,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"74.4 µs ± 1.3 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
]
}
],
"source": [
"rng_2d[rng_2d[:,0] > 40, 0]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rng = np.arange(10000000)\n",
"# symbols: & and, | or, ~ not\n",
"%timeit rng[(rng<90) & (rng>67)] # \n",
"%timeit rng[np.logical_and(rng<90,rng>67)]"
"%timeit my_algorithm_numba(array)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Performance Comparison"
"Using **Cython** to write our algorithm"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 22,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The Cython extension is already loaded. To reload it, use:\n",
" %reload_ext Cython\n"
]
}
],
"source": [
"vector_len = 10000000"
"%load_ext Cython"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"square = []\n",
"for i in range(vector_len):\n",
" square.append(i**2)"
"%%cython\n",
"\n",
"def cython_mean(long[:] x):\n",
" cdef double total = 0\n",
" for i in range(len(x)):\n",
" total += x[i]\n",
" return total / len(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 24,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"192 µs ± 7.54 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
]
}
],
"source": [
"%%time\n",
"square_np = np.square(np.arange(vector_len))"
"%timeit cython_mean(array)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"square_even = []\n",
"for i in range(vector_len):\n",
" i_2 = i**2\n",
" if i_2 < (vector_len / 2) ** 2:\n",
" square_even.append(i_2)\n",
" else:\n",
" pass"
"### Complexer Example\n",
"\n",
"[This](https://jakevdp.github.io/blog/2012/08/24/numba-vs-cython/) example is quite well explained. But let's go through one ourselfs. "
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"square_even_np = np.square(np.arange(vector_len))\n",
"square_even_np = square_even_np[square_even_np < (vector_len / 2) ** 2]"
"def mean(array):\n",
" summe = 0\n",
" for num in array:\n",
" summe += num\n",
" return summe / len(array)\n",
" \n",
"\n",
"def nse(modelled, observed):\n",
" numerator = 0\n",
" denominator = 0\n",
" observed_mean = mean(observed)\n",
" for i in range(len(modelled)):\n",
" numerator += (modelled[i] - observed[i]) ** 2\n",
" denominator += (observed[i] - observed_mean) ** 2\n",
" \n",
" return 1 - (numerator / denominator)"
]
},
{
......@@ -311,9 +243,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
"version": "3.8.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}
This diff is collapsed.
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Loops\n",
"\n",
"\n",
"### For Loop\n",
"\n",
"This kind of loop is designed to iterate over data structures (lists, tuple, dict, set,...) that have a length."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# summing up the values of a list\n",
"summe=0\n",
"for num in [1,2,3,4]:\n",
" # everything that is indeted is within the loop\n",
" summe+=num\n",
" print(num)\n",
" \n",
"print(\"Summe: \"+str(summe))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for number in [1,2,3,4,[],\"ffffdd\"]:\n",
" print(number)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### While Loop\n",
"\n",
"Designed to repeat the execution of code until a certain condition is met."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# while loop example\n",
"start=1\n",
"end=6\n",
"while start<end:\n",
" print(start)\n",
" start+=1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Be careful with while loop, this one would run endlessly. Uncomment and run if you dare! ;) "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#while True:\n",
"# a=1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Creating a simple matrix in a loop!..."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[['0,0', '0,1', '0,2', '0,3', '0,4', '0,5', '0,6', '0,7'],\n",
" ['1,0', '1,1', '1,2', '1,3', '1,4', '1,5', '1,6', '1,7'],\n",
" ['2,0', '2,1', '2,2', '2,3', '2,4', '2,5', '2,6', '2,7'],\n",
" ['3,0', '3,1', '3,2', '3,3', '3,4', '3,5', '3,6', '3,7']]"
]
},
"execution_count": 2,