Commit df87d84e authored by christian.foerster's avatar christian.foerster
Browse files

restructering tutorial, and adding better description

parent a5c79194
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Observer Pattern"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"from __future__ import annotations\n",
"from abc import ABC, abstractmethod\n",
"from random import randrange\n",
"from typing import List\n",
"\n",
"\n",
"class Subject(ABC):\n",
" \"\"\"\n",
" The Subject interface declares a set of methods for managing subscribers.\n",
" \"\"\"\n",
"\n",
" @abstractmethod\n",
" def attach(self, observer: Observer) -> None:\n",
" \"\"\"\n",
" Attach an observer to the subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
" @abstractmethod\n",
" def detach(self, observer: Observer) -> None:\n",
" \"\"\"\n",
" Detach an observer from the subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
" @abstractmethod\n",
" def notify(self) -> None:\n",
" \"\"\"\n",
" Notify all observers about an event.\n",
" \"\"\"\n",
" pass\n",
"\n",
"\n",
"class ConcreteSubject(Subject):\n",
" \"\"\"\n",
" The Subject owns some important state and notifies observers when the state\n",
" changes.\n",
" \"\"\"\n",
"\n",
" _state: int = None\n",
" \"\"\"\n",
" For the sake of simplicity, the Subject's state, essential to all\n",
" subscribers, is stored in this variable.\n",
" \"\"\"\n",
"\n",
" _observers: List[Observer] = []\n",
" \"\"\"\n",
" List of subscribers. In real life, the list of subscribers can be stored\n",
" more comprehensively (categorized by event type, etc.).\n",
" \"\"\"\n",
"\n",
" def attach(self, observer: Observer) -> None:\n",
" print(\"Subject: Attached an observer.\")\n",
" self._observers.append(observer)\n",
"\n",
" def detach(self, observer: Observer) -> None:\n",
" self._observers.remove(observer)\n",
"\n",
" \"\"\"\n",
" The subscription management methods.\n",
" \"\"\"\n",
"\n",
" def notify(self) -> None:\n",
" \"\"\"\n",
" Trigger an update in each subscriber.\n",
" \"\"\"\n",
"\n",
" print(\"Subject: Notifying observers...\")\n",
" for observer in self._observers:\n",
" observer.update(self)\n",
"\n",
" def some_business_logic(self) -> None:\n",
" \"\"\"\n",
" Usually, the subscription logic is only a fraction of what a Subject can\n",
" really do. Subjects commonly hold some important business logic, that\n",
" triggers a notification method whenever something important is about to\n",
" happen (or after it).\n",
" \"\"\"\n",
"\n",
" print(\"\\nSubject: I'm doing something important.\")\n",
" self._state = randrange(0, 10)\n",
"\n",
" print(f\"Subject: My state has just changed to: {self._state}\")\n",
" self.notify()\n",
"\n",
"\n",
"class Observer(ABC):\n",
" \"\"\"\n",
" The Observer interface declares the update method, used by subjects.\n",
" \"\"\"\n",
"\n",
" @abstractmethod\n",
" def update(self, subject: Subject) -> None:\n",
" \"\"\"\n",
" Receive update from subject.\n",
" \"\"\"\n",
" pass\n",
"\n",
"\n",
"\"\"\"\n",
"Concrete Observers react to the updates issued by the Subject they had been\n",
"attached to.\n",
"\"\"\"\n",
"\n",
"\n",
"class ConcreteObserverA(Observer):\n",
" def update(self, subject: Subject) -> None:\n",
" if subject._state < 3:\n",
" print(\"ConcreteObserverA: Reacted to the event\")\n",
"\n",
"\n",
"class ConcreteObserverB(Observer):\n",
" def update(self, subject: Subject) -> None:\n",
" if subject._state == 0 or subject._state >= 2:\n",
" print(\"ConcreteObserverB: Reacted to the event\")\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Subject: Attached an observer.\n",
"Subject: Attached an observer.\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 7\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 8\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n",
"\n",
"Subject: I'm doing something important.\n",
"Subject: My state has just changed to: 9\n",
"Subject: Notifying observers...\n",
"ConcreteObserverB: Reacted to the event\n"
]
}
],
"source": [
"subject = ConcreteSubject()\n",
"\n",
"observer_a = ConcreteObserverA()\n",
"subject.attach(observer_a)\n",
"\n",
"observer_b = ConcreteObserverB()\n",
"subject.attach(observer_b)\n",
"\n",
"subject.some_business_logic()\n",
"subject.some_business_logic()\n",
"\n",
"subject.detach(observer_a)\n",
"\n",
"subject.some_business_logic()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Packaging\n",
"\n",
"Jupyter Notebook is not ideal for packaging.\n",
"We'll be negelecting the setup of a virtual environment for now, as it might only confuse you and it's setup if different from ide to ide. \n",
"\n",
"Setting up virtual environments:\n",
" - [Anaconda](https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/20/conda/)\n",
" - [virtualenv python module](https://realpython.com/python-virtual-environments-a-primer/)\n",
" - Pycharm simply does it for you! (also uses virtualenv)\n",
"\n",
"---\n",
"### Set up Folders and Files\n",
"We're gonna go through all the steps now. The low level basic way!\n",
"- create a folder called after your package (tanks)\n",
"- go into that folder\n",
"- create \n",
" - readme.md\n",
" - setup.py\n",
" - license.txt\n",
" - package folder\n",
" - doc folder\n",
" - test folder\n",
"\n",
"**Terminal commands make our lives easier!**\n",
"\n",
"Linux and Mac commands for setup\n",
"```shell\n",
"mkdir tanks\n",
"cd tanks\n",
"mkdir tanks\n",
"mkdir doc\n",
"mkdir test\n",
"touch readme.md\n",
"touch setup.py\n",
"touch license.txt\n",
"```\n",
"\n",
"Windows commands for setup (not sure whetherthes commands work on all windows versions!)\n",
"```shell\n",
"mkdir tanks\n",
"cd tanks\n",
"mkdir tanks\n",
"mkdir doc\n",
"mkdir test\n",
"type NUL > readme.md\n",
"type NUL > setup.py\n",
"type NUL > license.txt\n",
"```\n",
"---\n",
"### Fill Folders and Files\n",
"The folder structure has been set up. Let's fill files and folders:\n",
"\n",
"- write your readme.md\n",
"- add licence eg. copy paste (https://opensource.org/licenses/MIT)\n",
"- enter the test folder and add test.py with our tests (the following snipped needs to be added)\n",
"\n",
"```python\n",
"import os\n",
"import sys\n",
"sys.path.append(os.path.abspath(\".\"))\n",
"\n",
"from tanks.tanks import Tank\n",
"```\n",
"\n",
"- enter the package folder and create two files \n",
" - \\__init__.py (2 underscores front and back)\n",
" - tanks.py (add our code)\n",
"- fill in the setup.py with the following\n",
"\n",
"```python\n",
"from setuptools import setup, find_packages\n",
"\n",
"setup(name='tanks',\n",
" version='1.0',\n",
" description='A tool for simulating water tanks.',\n",
" long_description='Our very long explanation on what our package does',\n",
" classifiers=[\n",
" 'Development Status :: early stage',\n",
" 'Programming Language :: Python :: 3',\n",
" ],\n",
" install_requires=[], # this will normally be containg depencies from packages\n",
" packages=find_packages(exclude=(\"doc\",\".git\",\".idea\",\"venv\", \"test\")),\n",
" keywords='Water Management',\n",
" author='Christian Foerster',\n",
" author_email='Christian.Foerster@eawag.ch',\n",
" license='MIT')\n",
"```\n",
"\n",
"---\n",
"### Generate Documentation\n",
"\n",
"First we'll install Sphinx.\n",
"```sh\n",
"pip3 install -U sphinx sphinx_rtd_theme\n",
"```\n",
"\n",
"Now we want to open a terminal in the doc folder we created before and run (defaults will do):\n",
"\n",
"```sh\n",
"sphinx-quickstart\n",
"```\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Tweak Sphinx Settings\n",
"\n",
"\n",
"The doctrings in our code are written in the NumPy style. Other styles are available, if you want to check some out ( ([stackflow overview](https://stackoverflow.com/questions/3898572/what-is-the-standard-python-docstring-format), [sphinx overview](https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html)) )\n",
"\n",
"In order for spinx to handle the NumPy style we must include the `sphinx.ext.napoleon` extension and add a few other settings to the conf.py. \n",
"\n",
"```python\n",
"# Configuration file for the Sphinx documentation builder.\n",
"#\n",
"# This file only contains a selection of the most common options. For a full\n",
"# list see the documentation:\n",
"# http://www.sphinx-doc.org/en/master/config\n",
"\n",
"# -- Path setup --------------------------------------------------------------\n",
"\n",
"# If extensions (or modules to document with autodoc) are in another directory,\n",
"# add these directories to sys.path here. If the directory is relative to the\n",
"# documentation root, use os.path.abspath to make it absolute, like shown here.\n",
"#\n",
"import os\n",
"import sys\n",
"sys.path.insert(0, os.path.abspath('.'))\n",
"sys.path.insert(0, os.path.abspath('../'))\n",
"\n",
"\n",
"# -- Project information -----------------------------------------------------\n",
"\n",
"project = 'tanks'\n",
"copyright = '2019, Christian Foerster'\n",
"author = 'Christian Foerster'\n",
"\n",
"# The full version, including alpha/beta/rc tags\n",
"release = '1.0'\n",
"\n",
"\n",
"# -- General configuration ---------------------------------------------------\n",
"\n",
"# Add any Sphinx extension module names here, as strings. They can be\n",
"# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n",
"# ones.\n",
"extensions = [\n",
" 'sphinx.ext.autodoc',\n",
" 'sphinx.ext.viewcode',\n",
" 'sphinx.ext.napoleon'\n",
"]\n",
"\n",
"napoleon_google_docstring = False\n",
"napoleon_use_param = False\n",
"napoleon_use_ivar = True\n",
"\n",
"\n",
"# Add any paths that contain templates here, relative to this directory.\n",
"templates_path = ['_templates']\n",
"\n",
"# List of patterns, relative to source directory, that match files and\n",
"# directories to ignore when looking for source files.\n",
"# This pattern also affects html_static_path and html_extra_path.\n",
"exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n",
"\n",
"\n",
"# -- Options for HTML output -------------------------------------------------\n",
"\n",
"# The theme to use for HTML and HTML Help pages. See the documentation for\n",
"# a list of builtin themes.\n",
"#\n",
"html_theme = 'sphinx_rtd_theme'\n",
"\n",
"# Add any paths that contain custom static files (such as style sheets) here,\n",
"# relative to this directory. They are copied after the builtin static files,\n",
"# so a file named \"default.css\" will overwrite the builtin \"default.css\".\n",
"html_static_path = ['_static']\n",
"\n",
"numpydoc_show_class_members = False\n",
"\n",
"```\n",
"\n",
"#### Generate .rst Files\n",
"\n",
"In this step we're going to generate `.rst` files from our python doctrings.\n",
"\n",
"Simply run this command.\n",
"\n",
"```shell\n",
"sphinx-apidoc -o source/ ../tanks\n",
"```\n",
"\n",
"Sphinx will have created a folder called source in your doc folder. This folder contains the `.rst` files.\n",
"\n",
"#### Define Website Appearance\n",
"\n",
"Finally we can start telling Shpinx what the we on our website. To do that open the `index.rst`.\n",
"\n",
"You should in include the files from the source folder and any other content you might desire.\n",
"\n",
"\n",
"**This will be the most laborious step of the whole process.**\n",
"\n",
"For further information on this check out [Documenting Your Project Using Sphinx](https://pythonhosted.org/an_example_pypi_project/sphinx.html).\n",
"\n",
"\n",
"I created a few rst files more to add information regarding installation, examples, issues ...\n",
"\n",
"and added all these to the source directory / **index.rst**:\n",
"\n",
"```rst\n",
".. tanks documentation master file, created by\n",
" sphinx-quickstart on Thu Nov 7 15:49:27 2019.\n",
" You can adapt this file completely to your liking, but it should at least\n",
" contain the root `toctree` directive.\n",
"\n",
"Welcome to tanks's documentation!\n",
"=================================\n",
"\n",
"Welcome to this minimal example on how to create your own python package. \n",
"\n",
".. note:: This is how you can format a note.\n",
"\n",
"\n",
".. toctree::\n",
" :maxdepth: 2\n",
" :caption: Contents:\n",
"\n",
" \n",
" ./source/description\n",
" ./source/installation\n",
" ./source/example\n",
" ./source/issues\n",
" ./source/tanks\n",
"\n",
"\n",
"Indices and tables\n",
"==================\n",
"\n",
"* :ref:`genindex`\n",
"* :ref:`modindex`\n",
"* :ref:`search`\n",
"\n",
"```\n",
"\n",
"\n",
"#### Generate HTML\n",
"\n",
"To generate the html all you need to do is run `make html` in your terminal.\n",
"\n",
"The command's output will state where the files are saved. In our package in the folder `./doc/_build/html`\n",
"contains the files. Open to the `index.html` to see what you have produced.\n",
"\n",
"---\n",
"\n",
"### Put it all on GitLab\n",
"\n",
"Initiate a git repository called **tanks** on your hosting platform (GitLab).\n",
"\n",
"Init a local git repository in the tanks folder.\n",
"```sh\n",
"git init\n",
"```\n",
"\n",
"Add remote to local repo.\n",
"```sh\n",
"git remote add origin git@gitlab.switch.ch:christian.foerster/tanks.git\n",
"```\n",
"\n",
"Finally create your first commit and push it.\n",
"```sh\n",
"git add .\n",
"git commit -m \"Initial commit\"\n",
"git push -u origin master\n",
"```\n",
"\n",
"---\n",
"\n",
"### FYI\n",
"For the **minimalistic** setup you would only need the **setup.py** and **package folder**. This is bad though! No generated documentation, no tests, and so on! But that's enough to pip install ;)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Typical Order of things\n",
"\n",
"**Use your IDE of choice!** \n",
"For trying out stuff I use jupyter notebook, it's light weight and interactive! As soon as my project is starting to take shape I switch to PyCharm.\n",
"\n",
"**Set up your Project!** \n",
"Start your IDE set up a new project with the _right name_! Set up a clean python environment (with anaconda, venv, or your IDE). Create the default file needed in your package! \n",
"That means at least:\n",
"- readme.md\n",
"- setup.py\n",
"- license.txt\n",
"- package folder\n",
"- doc folder \n",
"- test folder\n",
"\n",
"Initialize a Git repository and run your first commit to your hosting platform!\n",
"\n",
"**------------------------------------------------------** \n",
"**Now your read for some serious coding!** \n",
"**------------------------------------------------------**\n",
"\n",
"\n",
"**Document functions, methods and classes as you write them - NOT weeks later!** \n",
"\n",
"**For every module, class your code write tests (using the unittest package) as soon as you finished coding it**\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Performance\n",
"\n",
"- Numpy is fast! Why?\n",
" - try to stick with one variable type. Don't use lists and arrays and mix it all up\n",
"- what needs to be in a loop what not. \n",
"- trade off memory, performance"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
......@@ -4,273 +4,205 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Numpy\n",
"## Speeding up your python code\n",
"\n",
"For all of you that love matlab numpy will be very familiar. (https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html)\n",
"If you design your own python algorithms, they tend to be very slow. Mainly because python ist not a typed language.\n",
"\n",
"Why use numpy? Because it's very fast! (https://stackoverflow.com/questions/7596612/benchmarking-python-vs-c-using-blas-and-numpy)\n",
"You have different options to improve upon performance:\n",
"- write your algorithm in c/c++ or fortran and call it from python\n",
"- use Numba for small projects (Numba uses LLVM to power Just-In-Time compilation of array oriented Python code)\n",
"- use Cython for bigger projects, as it is more versatile\n",
"\n",
"**A more comprehensive tutorial:** (https://docs.scipy.org/doc/numpy/user/quickstart.html)"
"A more detailled comparison can be found [here](https://rushter.com/blog/numba-cython-python-optimization/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Arrays Examples"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"arr = np.array([3,4,5,6])\n",
"rng = np.arange(100)\n",